publications

2023

1. arxiv

   Recent Advances of Deep Robotic Affordance Learning: A Reinforcement Learning Perspective

   Yang, Xintong, Ji, Ze, Wu, Jing, and Lai, Yu-kun

   2023
2. IEEE Access

   Affordance-Based Human–Robot Interaction With Reinforcement Learning

   Munguia-Galeano, Francisco, Veeramani, Satheeshkumar, Hernández, Juan David, Wen, Qingmeng, and Ji, Ze

   IEEE Access 2023
3. WACV

   AFPSNet: Multi-Class Part Parsing based on Scaled Attention and Feature Fusion

   Alsudays, Njuod, Wu, Jing, Lai, Yu-Kun, and Ji, Ze

   In 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) 2023

2022

1. ROBIO

   Localisation-Safe Reinforcement Learning for Mapless Navigation

   Lin, Feiqiang, Ji, Ze, Wei, Changyun, and Grech, Raphael

   In 2022 IEEE International Conference on Robotics and Biomimetics (ROBIO) 2022
2. Mater. Des

   Predicting laser powder bed fusion defects through in-process monitoring data and machine learning

   Feng, Shuo, Chen, Zhuoer, Bircher, Benjamin, Ji, Ze, Nyborg, Lars, and Bigot, Samuel

   Materials & Design 2022

   Abs

   Industry application of additive manufacturing demands strict in-process quality control procedures and high product quality. Feedback loop control is a reasonable solution and a necessary tool. This paper demonstrated our preliminary work on the laser powder-bed fusion feedback loop: predict local porosity through in-process monitoring images and machine learning. 3D models were rebuilt from in-situ optical tomography monitoring images and post-build X-ray CT images. They were registered to the original CAD. Dataset for machine learning was assembled from those registered 3D models. The trained machine learning model can precisely predict local porosity caused by lack of fusion and keyhole with multi-layer monitoring images. It also indicates the optimal processing window. It is impossible to be sure about the occurrence of defects in a layer based only on the abnormality of a single layer, and vice versa. Defects in a layer can be caused by improper parameters or anomalies in current layer or subsequent layers; defects in one layer can also be eliminated by proper parameters in the following layers. The work laid the basis for the next step feedback loop control of pore defect.

3. TASE

   Anisotropic GPMP2: A Fast Continuous-Time Gaussian Processes Based Motion Planner for Unmanned Surface Vehicles in Environments With Ocean Currents

   Meng, Jiawei, Liu, Yuanchang, Bucknall, Richard, Guo, Weihong, and Ji, Ze

   IEEE Transactions on Automation Science and Engineering 2022
4. OE

   Learning manipulation skills with demonstrations for the swing process control of dredgers

   Wei, Changyun, Bai, Haonan, Wei, Yi, and Ji, Ze

   Ocean Engineering 2022
5. AIM

   Design and development of a robust vision-based tactile sensor

   Rayamane, Prasad, Ji, Ze, and Packianather, Michael

   In 2022 IEEE/ASME International Conference on Advanced Intelligent Mechatronics 2022
6. CASE

   FRobs_RL: A flexible robotics reinforcement learning library

   Fajardo, Jose Manuel, González Roldan, Felipe, Realpe, Sebastian, Hernandez, Juan D., Ji, Ze, and Cardenas, Pedro-F

   In IEEE International Conference on Automation Science and Engineering (CASE) 2022

2021

1. Int.J.Intell.Syst.

   Safe distance prediction for braking control of bridge cranes considering anti-swing

   Chen, Huili, Liu, Guoliang, Tian, Guohui, Zhang, Jianhua, and Ji, Ze

   International Journal of Intelligent Systems 2021
2. Int.J.Intell.Syst.

   Online human action recognition with spatial and temporal skeleton features using a distributed camera network

   Liu, Guoliang, Zhang, Qinghui, Cao, Yichao, Tian, Guohui, and Ji, Ze

   International Journal of Intelligent Systems 2021
3. TAROS

   Reinforcement Learning-based Mapless Navigation with Fail-safe Localisation

   Lin, F., Ji, Z., Wei, C., and Niu, H.

   In Towards Autonomous Robotic Systems Conference (TAROS) 2021
4. TAROS

   Designing a Multi-Locomotion Modular Snake Robot

   Djoudi, S., Frampton, J.M., Murphy, A., Scammell, T., Wright, T., and Ji, Z

   In Towards Autonomous Robotic Systems Conference (TAROS) 2021
5. Precis. Agric

   RS-Net: robust segmentation of green overlapped apples

   Jia, Weikuan, Zhang, Zhonghua, Shao, Wenjiang, Ji, Ze, and Hou, Sujuan

   Precision Agriculture 2021
6. Comput. Electron. Agric.

   FoveaMask: A fast and accurate deep learning model for green fruit instance segmentation

   Jia, Weikuan, Zhang, Zhonghua, Shao, Wenjing, Hou, Sujuan, Ji, Ze, Liu, Guoliang, and Yin, Xiang

   Computers and Electronics in Agriculture 2021

   Abs

   In the process of agricultural automation production, efficient and accurate segmentation of target fruit is the basis and guarantee for numerous applications including crop growth monitoring, yield prognosis, and machine picking. Green fruit is apt to affect by complicated scenes such as occlusions and overlaps, as well as the homo-chromatic background, which brings great challenges to the recognition and segmentation. In this paper, we propose a novel methodology named FoveaMask for improving the robustness and generalization of green fruit segmentation framework. Features of input images are firstly extracted by ResNet and fused by FPN. The classification and bounding-box regression of each spatial position on the feature maps are then carried out directly by the way of full convolution. RoI Align layer is applied to fix feature region of proposals to the same size yet preserve exact spatial locations. Finally, instance-level fruit segmentation is realized by pixel-level classification on each proposal using embedded mask branches. The whole network architecture of the new model does not involve the related design and operation of anchor, which greatly improves the generalization ability for different shape fruits, alleviates the computing and storage resources, and balances the contradiction between accuracy and efficiency simultaneously. Additionally, a position attention module(PAM) is introduced into the embedding mask branch to aggregate the effective information pixels, which can improve the robustness of the segmentation model in the actual complex environment of the orchard. We test the model on green apple and immature persimmon data sets and the experimental results show that FoveaMask performs best in terms of both recognition accuracy and model complexity compared with other 11 different types of detection and segmentation models.

7. IROS

   ShorelineNet: An Efficient Deep Learning Approach for Shoreline Semantic Segmentation for Unmanned Surface Vehicles

   Yao, L., Dimitrios, K., Ji, Z., and Liu, Y.

   In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems 2021
8. TAROS

   An Open-Source Multi-Goal Reinforcement Learning Environment for Robotic Manipulation with Pybullet

   Yang, Xintong, Ji, Ze, Wu, Jing, and Lai, Yu-Kun

   In Towards Autonomous Robotic Systems Conference (TAROS) 2021
9. TNNLS

   Hierarchical reinforcement learning with universal policies for multi-step robotic manipulation

   Yang, Xintong., Ji, Z., Wu, J., Lai, Y.K., Wei, C., Liu, G., and Setch, R.

   IEEE Transaction on Neural Networks and Learning Systems 2021
10. OE

    Model predictive control for slurry pipeline transportation of a cutter suction dredger

    Wei, Changyun, Wei, Yi, and Ji, Ze

    Ocean Engineering 2021

    Abs

    Cutter Suction Dredgers (CSDs) are a special type of ships designed for construction and maintenance projects of ocean and offshore engineering. During the dredging operation, CSDs can excavate nearly all kinds of soil on the sea bed, and then the dredged materials with coarse particles need to be sucked up by a slurry pump and transported to a disposal area through a long-distance pipeline. In order to avoid sedimentation of slurry in pipeline transportation, the flow rate must be maintained within a reasonable range. Otherwise, the pipeline can be blocked when the slurry density is too high. In this paper, we present a Model Predictive Control (MPC) approach to manipulate the flow rate of slurry in pipeline transportation for a CSD. To demonstrate the advantages of our proposed approach, we also implement three Proportional Integral Derivative (PID) controllers (i.e., conventional PID, Fuzzy-PID, and LQR-PID) to make a direct comparison. Moreover, in order to evaluate the effectiveness of our proposed approach in real scenarios, we have, in particular, built a slurry pipeline transportation platform. Both the simulation and experimental results show that our proposed MPC approach is more effective than other PID controllers in controlling the flow rate in the slurry pipeline transportation problem. The proposed approach can provide a guideline for the automated control of the slurry pump for a CSD.

11. Sens. Actuator

    Self-validating high-g accelerometers through data-driven methods

    Wen, Jingjing, Yao, Houpu, Ji, Ze, Wu, Bin, and Xu, Feng

    Sensors and Actuators A: Physical 2021

    Abs

    In the aerospace industry, pyroshock testing is an indispensable step in designing space electronics. Yet, damages in high-g accelerometers, the core measuring instruments in pyroshock test systems, could result in various failures of pyroshock tests. To ensure the reliability of pyroschock tests for space electronics, a machine learning system is proposed to perform self-validation for high-g accelerometers. In this work, self-validation refers to the capability of identifying five key parameters, namely the validated shock signal, the validated uncertainty, the measurement status, the raw shock signal and the fault type, synchronously during measuring shock signals in pyroshock tests. To achieve the highest performance, we accomplish these tasks through combining an ensemble learning model and a deep neural network (DNN). The ensemble learning model, which integrates several k-nearest neighbors with different k values, is used to identify the sensorsâ health conditions from their measurements and diagnose their fault types synchronously if damaged. The DNN, a deep autoencoder-based neural network, is designed to correct corrupted measurements through constructing the mapping between faulty signals and their corresponding reference counterparts. Experimental results show that the proposed machine learning system is capable of not only accurately identifying the health conditions and fault types of the damaged high-g accelerometers from their measurements, but also recovering the corrupted shock signals to a large extent, and, meanwhile, outputting the five self-validation parameters.

12. SII

    Accelerated Sim-to-Real Deep Reinforcement Learning: Learning Collision Avoidance from Human Player

    Niu, H., Ji, Z., Arvin, F., Lennox, B., and Carrasco, J.

    In 2021 IEEE/SICE International Symposium on System Integration 2021
13. SII

    Model Checking for Decision Making System of Long Endurance Unmanned Surface Vehicle

    Niu, H., Ji, Z., Savvaris, A., Tsourdos, A., and Carrasco, J.

    In 2021 IEEE/SICE International Symposium on System Integration 2021
14. Automat Constr

    Deep learning-based prediction of piled-up status and payload distribution of bulk material

    Yao, ZongWei, Huang, Qiuping, Ji, Ze, Li, XueFei, and Bi, Qiushi

    Automation in Construction 2021

2020

1. JAMT

   A novel image feature descriptor for SLM spattering pattern classification using a consumable camera

   Ji, Ze, and Han, Quanquan

   Int J Adv Manuf Technol 2020
2. OE

   Energy efficient path planning for Unmanned Surface Vehicle in spatially-temporally variant environment

   Niu, Hanlin, Ji, Ze, Savvaris, Al, and Tsourdos, Antonios

   Ocean Engineering 2020

   Abs

   Unmanned Surface Vehicles (USVs) are increasingly used for ocean missions, which typically require long duration of operations under strict energy constraints. Consequently, there is an increased interest in energy efficient path planning for USVs. This work proposes a novel energy efficient path planning algorithm to address the challenges with the presence of spatially-temporally variant sea current and complex geographic map data, by integrating the following algorithms, namely Voronoi roadmap, Dijkstra’s searching, coastline expanding and genetic algorithm (GA). The selection, crossover and mutation operators are employed as part of the GA algorithm. The dividing, smoothing and exchanging operators are proposed to improve the quality of the path and adapt to the Voronoi-Visibility roadmap. The Global Self-Consistent Hierarchical High-Resolution Shorelines dataset and historical sea current dataset are applied to demonstrate the flexibility and practicability of the proposed algorithm. To evaluate the performance, the Voronoi-GA energy efficient algorithm and Voronoi-Visibility energy efficient path re-planning algorithm are also implemented to provide the baseline for comparison. The proposed algorithm generates the most energy efficient paths in ten USV missions, while keeping a configurable clearance from the coastlines. The practicability and scalability of this algorithm is also demonstrated by analysing the computational time in these ten missions.

3. IEEE Sensors

   On Fault Diagnosis for High-g Accelerometers via Data-Driven Models

   Wen, J., Yao, H., Ji, Z., Wu, B., and Xia, M.

   IEEE Sensors Journal 2020
4. RA-L

   Particle Swarm Optimization for Cooperative Multi-Robot Task Allocation: A Multi-Objective Approach

   Wei, C., Ji, Z., and Cai, B.

   IEEE Robotics and Automation Letters 2020
5. IEEE Sensors

   Efficient Multi-View Multi-Target Tracking Using a Distributed Camera Network

   He, L., Liu, G., Tian, G., Zhang, J., and Ji, Z.

   IEEE Sensors Journal 2020
6. IEEE Sensors

   A Deep Learning Approach to Recover High-g Shock Signals From the Faulty Accelerometer

   Wen, J., Yao, H., Wu, B., Ren, Y., and Ji, Z.

   IEEE Sensors Journal 2020
7. AIM

   Online Collision Avoidance for Human-Robot Collaborative Interaction Concerning Safety and Efficiency

   Liu, G., He, H., Tian, G., Zhang, J., and Ji, Z.

   In 2020 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM) 2020

2019

1. IEEE Robio

   Visual SLAM Based on Dynamic Object Removal

   Liu, H., Liu, G., Tian, G., Xin, S., and Ji, Z.

   In 2019 IEEE International Conference on Robotics and Biomimetics (ROBIO) 2019
2. IEEE Robio

   Distributed Human 3D Pose Estimation and Action Recognition

   Liu, G., Liu, T., Tian, G., and Ji, Z.

   In 2019 IEEE International Conference on Robotics and Biomimetics (ROBIO) 2019
3. IEEE Robio

   Simultaneous Monocular Visual Odometry and Depth Reconstruction with Scale Recovery

   Luo, Y., Liu, G., Liu, H., Liu, T., Tian, G., and Ji, Z.

   In 2019 IEEE International Conference on Robotics and Biomimetics (ROBIO) 2019
4. taros

   An Auto-Correction Teleoperation Method for a Mobile Manipulator Using Gaze Tracking and Hand Motion Detection

   Chen, Junshen, Ji, Ze, Niu, Hanlin, Setchi, Rossitza, and Yang, Chenguang

   In Towards Autonomous Robotic Systems 2019

   Abs

   Situational awareness in remote environments is crucial for human operators to teleoperate mobile manipulators confidently and reliably. Visual feedback is the most common way for environment perception, providing rich information to human operators. This paper proposes an intuitive teleoperation method by combining gaze tracking and hand motion detection to teleoperate a mobile manipulator. A camera is fixed on the end-effector of the mobile robot’s arm to provide visual feedback, acting as the eye of the teleoperator. Rather than direct remote control of the robot, an adaptive auto-correction mechanism is introduced for helping human operators to achieve better hand-eye coordination of the teleoperation experiences. The mobile manipulator can adjust its behaviours, such as speed, while gaze and hand movements of the operator are in different states. The experiments carried out demonstrated the effectiveness of the proposed algorithm and the survey evaluation verified the practical application value of the system.

5. Voronoi-Visibility Roadmap-based Path Planning Algorithm for Unmanned Surface Vehicles

   Niu, Hanlin, Savvaris, Al, Tsourdos, Antonios, and Ji, Ze

   Journal of Navigation 2019
6. ICASSP

   Low-cost Measurement of Industrial Shock Signals via Deep Learning Calibration

   Yao, H., Wen, J., Ren, Y., Wu, B., and Ji, Z.

   In ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) 2019
7. JNE

   Feature extraction of four-class motor imagery EEG signals based on functional brain network

   Ai, Qingsong, Chen, Anqi, Chen, Kun, Liu, Quan, Zhou, Tichao, Xin, Sijin, and Ji, Ze

   Journal of Neural Engineering 2019

   Abs

   Objective. A motor-imagery-based brain–computer interface (MI-BCI) provides an alternative way for people to interface with the outside world. However, the classification accuracy of MI signals remains challenging, especially with an increased number of classes and the presence of high variations with data from multiple individual people. This work investigates electroencephalogram (EEG) signal processing techniques, aiming to enhance the classification performance of multiple MI tasks in terms of tackling the challenges caused by the vast variety of subjects. Approach. This work introduces a novel method to extract discriminative features by combining the features of functional brain networks with two other feature extraction algorithms: common spatial pattern (CSP) and local characteristic-scale decomposition (LCD). After functional brain networks are established from the MI EEG signals of the subjects, the measures of degree in the binary networks are extracted as additional features and fused with features in the frequency and spatial domains extracted by the CSP and LCD algorithms. A real-time BCI robot control system is designed and implemented with the proposed method. Subjects can control the movement of the robot through four classes of MI tasks. Both the BCI competition IV dataset 2a and real-time data acquired in our designed system are used to validate the performance of the proposed method. Main results. As for the offline data experiment results, the average classification accuracy of the proposed method reaches 79.7%, outperforming the majority of popular algorithms. Experimental results with real-time data also prove the proposed method to be highly promising in its real-time performance. Significance. The experimental results show that our proposed method is robust in extracting discriminative brain activity features when performing different MI tasks, hence improving the classification accuracy in four-class MI tasks. The high classification accuracy and low computational demand show a considerable practicality for real-time rehabilitation systems.

2018

1. Autonomous 3D scene understanding and exploration in cluttered workspaces using point cloud data

   Ji, Z., Schruoffeneger, R., and Setchi, R.

   In 2018 IEEE 15th International Conference on Networking, Sensing and Control (ICNSC) 2018
2. taros

   Seeing the Unseen: Locating Objects from Reflections

   Wu, Jing, and Ji, Ze

   In Towards Autonomous Robotic Systems 2018

   Abs

   Inspired by the ubiquitous use of reflections in human vision system, in this paper, we present our first step exploration of using reflections to extend the FOV of cameras in computer vision applications. We make use of a stereo camera, and establish mathematical models for locating objects from their mirror reflections. We also propose a pipeline to track and locate moving objects from their reflections captured in videos. Experimental results demonstrated the efficiency and effectiveness of the proposed method, and verified the potential use of reflections in locating non-line of sight (NLOS) objects.

2017

1. Autonomous boat dynamics: How far away is simulation from the high sea?

   Huang, Y., and Ji, Z.

   In OCEANS 2017 - Aberdeen 2017