Publications

Existing image-based pest counting methods rely on single static images and often produce inaccurate results under occlusion. To address this issue, this paper proposes an automated pest counting method in water traps through active robotic stirring. First, an automated robotic arm-based stirring system is developed to redistribute pests and reveal occluded individuals for counting. Then, the effects of different stirring patterns on pest counting performance are investigated. Six stirring patterns are designed and evaluated across different pest density scenarios to identify the optimal one. Finally, a heuristic counting confidence-driven closed-loop control system is proposed for adaptive-speed robotic stirring, adjusting the stirring speed based on the average change rate of counting confidence between consecutive frames. Experimental results show that the four circles is the optimal stirring pattern, achieving the lowest overall mean absolute counting error of 4.384 and the highest overall mean counting confidence of 0.721. Compared with constant-speed stirring, adaptive-speed stirring reduces task execution time by up to 44.7% and achieves more stable performance across different pest density scenarios. Moreover, the proposed pest counting method reduces the mean absolute counting error by up to 3.428 compared to the single static image counting method under high-density scenarios where occlusion is severe.

Accurate pest population monitoring and tracking their dynamic changes are crucial for precision agriculture decision-making. A common limitation in existing vision-based automatic pest counting research is that models are typically evaluated on datasets with ground truth but deployed in real-world scenarios without assessing the reliability of counting results due to the lack of ground truth. To this end, this paper proposed a method for comprehensively evaluating pest counting confidence in the image, based on information related to counting results and external environmental conditions. First, a pest detection network is used for pest detection and counting, extracting counting result-related information. Then, the pest images undergo image quality assessment, image complexity assessment, and pest distribution uniformity assessment. And the changes in image clarity caused by stirring during image acquisition are quantified by calculating the average gradient magnitude. Notably, we designed a hypothesis-driven multi-factor sensitivity analysis method to select the optimal image quality assessment and image complexity assessment methods. And we proposed an adaptive DBSCAN clustering algorithm for pest distribution uniformity assessment. Finally, the obtained information related to counting results and external environmental conditions is input into a regression model for prediction, resulting in the final pest counting confidence. To the best of our knowledge, this is the first study dedicated to comprehensively evaluating counting confidence in counting tasks, and quantifying the relationship between influencing factors and counting confidence through a model. Experimental results show our method reduces MSE by 31.7% and improves R² by 15.2% on the pest counting confidence test set, compared to the baseline built primarily on information related to counting results.

In order to improve the learning ability of robot in factory transportation system and speed up the learning cycle for robots to deal with the changing environment, on the basis of investigating the current research work of robot navigation in factory transportation system, this paper designs three different interactive reinforcement learning (IRL) algorithms and apply them to the navigation task for robots in factory transportation system. We verify the designed algorithms in the discrete grid world. From the experiment results, we can see that the third designed IRL algorithm using modified exploration strategy with the human suggestion and the feedback achieves the best effect, the success rate of completing the task is 90%, and the learning cycle is between other two algorithms. The project code is released at: https://github.com/XuminGaoGithub/Human-Robot-Interactive-Reinforcement-Learning-for-Factory-Robot-Navigation.

The current vision-based aphid counting methods in water traps suffer from undercounts caused by occlusions and low visibility arising from dense aggregation of insects and other objects. To address this problem, we propose a novel aphid counting method through interactive stirring actions. We use interactive stirring to alter the distribution of aphids in the yellow water trap and capture a sequence of images which are then used for aphid detection and counting through an optimized small object detection network based on Yolov5. We also propose a counting confidence evaluation system to evaluate the confidence of count-ing results. The final counting result is a weighted sum of the counting results from all sequence images based on the counting confidence. Experimental results show that our proposed aphid detection network significantly outperforms the original Yolov5, with improvements of 33.9% in AP@0.5 and 26.9% in AP@[0.5:0.95] on the aphid test set. In addition, the aphid counting test results using our proposed counting confidence evaluation system show significant improvements over the static counting method, closely aligning with manual counting results.

Aphids can cause direct damage and indirect virus transmission to crops. Timely monitoring and control of their populations are thus critical. However, the manual counting of aphids, which is the most common practice, is labor-intensive and time-consuming. Additionally, two of the biggest challenges in aphid counting are that aphids are small objects and their density distributions are varied in different areas of the field. To address these challenges, we proposed a hybrid automatic aphid counting network architecture which integrates the detection network and the density map estimation network. When the distribution density of aphids is low, it utilizes an improved Yolov5 to count aphids. Conversely, when the distribution density of aphids is high, it switches to CSRNet to count aphids. To the best of our knowledge, this is the first framework integrating the detection network and the density map estimation network for counting tasks. Through comparison experiments of counting aphids, it verified that our proposed approach outperforms all other methods in counting aphids. It achieved the lowest MAE and RMSE values for both the standard and high-density aphid datasets: 2.93 and 4.01 (standard), and 34.19 and 38.66 (high-density), respectively. Moreover, the AP of the improved Yolov5 is 5 % higher than that of the original Yolov5. Especially for extremely small aphids and densely distributed aphids, the detection performance of the improved Yolov5 is significantly better than the original Yolov5. This work provides an effective early warning caused by aphids in sugar beet fields, offering protection for sugar beet growth and ensuring sugar beet yield. The datasets and project code are released at: https://github.com/JunfengGaolab/Counting-Aphids.

In order to promote agricultural automatic picking and yield estimation technology, this project designs a set of automatic detection, positioning and counting algorithms for grape bunches, and applies it to agricultural robots. The Yolov3 detection network is used to realize the accurate detection of grape bunches, and the local tracking algorithm is added to eliminate relocation. Then it obtains the accurate 3D spatial position of the central points of grape bunches using the depth distance and the spatial restriction method. Finally, the counting of grape bunches is completed. It is verified using the agricultural robot in the simulated vineyard environment. The project code is released at: https://github.com/XuminGaoGithub/Grape_bunches_count_using_robots.

This work proposes a lightweight object detection network model ShuffleNet-SSD (S-SSD) to solve the problem of single shot multibox detector (SSD) network model where it cannot meet the real-time performance requirement in the task of object detection and recognition of indoor mobile robot. This model is suitable for indoor mobile robot by improving the SSD network model based on ShuffleNet network. The main idea of the improvement is that S-SSD replaces VGG-16 network as the basic feature extraction network of SSD network model with ShuffleNet network. The proposed model is based on the design of deep separable convolution, point-by-point grouping convolution, and channel rearrangement. It retains the design idea of multiscale feature graph detection of SSD network model. This model ensures a slight decline in detection accuracy while greatly reduces the amount of computation generated by the network operation, thereby greatly improving the detection rate. A data set for the task of object detection and recognition of indoor mobile robot is made. The S-SSD lightweight network model is superior to the original SSD network model and tiny-YOLO lightweight network model in terms of detection accuracy and detection rate, and can simultaneously meet the requirement of detection accuracy and real-time performance in the task of indoor object detection and recognition of mobile robot. These findings are verified through the comparative experiments of object detection accuracy and detection rate and real-time object detection and recognition of mobile robot under the actual indoor scene.

Aiming at the lack of semantic information of indoor objects in the grid map commonly used in the indoor mobile robot, this paper proposes a real-time indoor semantic map construction method combined with the lightweight object detection network: firstly, we proposes a lightweight object detection network which is called S-SSD (ShuffleNet-SSD) by improving the SSD (Single Shot MultiBox Detector) network, it can be used to extract the semantic information of indoor objects in real time; then the semantic information of indoor objects is transformed into the grid map which is created by mobile robot, so the indoor semantic map is constructed. The effectiveness and superiority of S-SSD in the indoor object detection of mobile robot are verified by the comparative experiment, and the effectiveness of the real-time indoor semantic map construction proposed in this paper is verified by the experiment of semantic map construction.

Aiming at the multi-scale characteristics of satellite imagery, and the adhesion phenomenon in the farmland segmentation results which is caused by the close distance between different farmland blocks, this paper proposes a robust and effective network based on U-Net for farmland segmentation of satellite imagery, which is called MMUU-Net. On the basis of adopting the encoder with higher classification accuracy network, adding ASPP (Atrous Spatial Pyramid Pooling) layer in the middle, and designing the multi-scale feature fusion module in the decoder, so that the multi-scale feature information is fully utilized; in order to better fuse multi-scale information, a more robust loss function is designed; finally, we propose a segmentation strategy of the coarse and refined two-stage to eliminate the adhesion phenomenon. Through the comparative experiments, it is verified that MMUU-Net is better than other segmentation networks, and can be effectively applied to the task of farmland segmentation of satellite imagery.

Aiming at the phenomenon that the fixed segmentation threshold leads to under segmentation and over segmentation in extracting line feature of laser radar data using the IEPF algorithm, a separate and merged extraction algorithm of line feature combined with support vector machine was proposed. In separate stage,the IEPF algorithm was used to segment the data preliminarily. In the merged stage, the threshold value was adjusted to eliminate the line segments of under segmentation as far as possible, and the three features from over segmented line segments and normal line segments which included the adjacent degree, the collinear degree and the overlap degree were extracted respectively as feature vectors to train the SVM model. The SVM model was applied to the actual test, and the classification results which were over segmented line segments were merged. Experimental results show that the proposed algorithm effectively eliminates most of the over segmented line segments and under segmented line segments produced by line segment extraction using IEPF algorithm.