Publications

2026

1. 

   Incremental 3D Crop Model Association for Real-Time Counting in Dense Orchards

   Daesung Park, KwangEun Ko, Dongbum Pyo, and Jaehyeon Kang

   IEEE Robotics and Automation Letters (RA-L), 2026

   Abs DOI Video

   Accurate real-time crop counting is essential for autonomous agricultural systems. However, existing methods often fail in dense plantings due to heavy foliage, irregular planting patterns, and frequent occlusions. While 2D tracking suffers from double-counting and 3D reconstruction requires offline processing, we propose a real-time crop counting framework that incrementally constructs global 3D crop instances during data collection. Each crop is modeled as a 3D oriented bounding box, initialized upon detection and updated with subsequent observations. To ensure robust association across frames, we employ 3D Generalized Intersection over Union (GIoU) for spatial matching and confidence-based filtering for validation, effectively reducing double-counting in dense orchards. Unlike prior methods, our approach supports on-the-fly counting without post-hoc reconstruction and performs reliably in unstructured field conditions. Experimental results demonstrate the accuracy and real-time capability of the proposed system in dense agricultural settings.

2025

1. 

   Retraining-Free Camera Localization in Indoor Point Clouds Using Edges and Normals

   Jaehyeon Kang

   IEEE Transactions on Instrumentation and Measurement (TIM), 2025

   Abs DOI

   Estimating a 6-DoF camera pose in a prebuilt map is essential for spatial perception in robotics and autonomous driving. While structure-from-motion (SfM) maps provide visual context, they lack geometric precision and scale. In contrast, point cloud maps from LiDAR or depth cameras offer accurate geometry for metric-level perception. This article presents a novel camera localization method that enhances pose estimation accuracy in indoor point cloud maps that do not contain visual information. By leveraging low-level features such as edges and normals from both the query image and the point cloud, the proposed approach operates across diverse environments without scene-specific training. Specifically, we introduce an edge alignment cost that measures the edge point reprojection errors and a normal distribution cost using normalized information distance (NID) to quantify normal vector similarity. An adaptive weighting scheme integrates these costs in an optimization framework. Experiments on simulated and real-world datasets demonstrate that our method outperforms state-of-the-art feature-matching algorithms in localization accuracy.

2024

1. 

   A Measure of Semantic Class Difference of Point Reprojection Pairs in Camera Pose Estimation

   Jaehyeon Kang and Changjoo Nam

   IEEE Transactions on Industrial Informatics (TII), 2024

   Abs DOI

   In this article, we propose a new measure that evaluates the semantic errors of camera poses in visual odometry (VO) and visual simultaneous localization and mapping (VSLAM). Traditionally, VO/VSLAM methods have used photometric images to estimate camera poses, but they suffer from varying illumination and viewpoint changes. Thus, methods using semantic images have been an alternative to increase consistency, as semantic information has shown its robustness even in hostile environments. Our measure compares semantic classes of map point reprojection pairs between images to improve the camera pose estimation accuracy in VO/VSLAM. To evaluate the difference between semantic classes, we adopt the normalized information distance from information theory. Furthermore, we suggest a weight parameter to balance the existing error of VO/VSLAM with the semantic error introduced by our approach. Our experimental results, obtained from the VKITTI and KITTI benchmark datasets, show that the proposed semantic error measure reduces both the relative pose error and absolute trajectory error of camera pose estimation compared to the existing photometric image-based errors of indirect and direct VO/VSLAM.

2022

1. 

   Spatial template-based geometric complexity reduction method for photo-realistic modeling of large-scale indoor spaces

   Janghun Hyeon, Joohyung Kim, Hyunga Choi, Bumchul Jang, Jaehyeon Kang, and Nakju Doh

   Engineering Applications of Artificial Intelligence, 2022

   Abs DOI

   Recent progresses in image-based rendering (IBR) have demonstrated the feasibility of photo-realistic modeling in room-scale indoor spaces. However, it is difficult to extend the method to large-scale indoor spaces, because the computational complexity increases exponentially as the geometric complexity increases. In this study, we propose a framework that automatically generates photo-realistic model of large-scale indoor spaces. We first define primary factors that increase geometrical complexity as geometrically excluded objects (GEOs). The proposed framework removes GEOs in images and point clouds to efficiently represent large-scale indoor spaces. To this end, we introduce a segmentation method to segment GEOs from every image coherently. In addition, we also introduce an image inpainting method to fill in the segmented images for photo-realistic indoor modeling. Experiments are conducted in three small-scale spaces and two large-scale indoor spaces. In the experiments, the proposed modules are validated thoroughly. In addition, the experimental results show that the proposed method enables to generate photo-realistic indoor models automatically and efficiently.

2. 

   Tomato harvesting robotic system based on Deep-ToMaToS: Deep learning network using transformation loss for 6D pose estimation of maturity classified tomatoes with side-stem

   JoonYoung Kim, HyeRan Pyo, Inhoon Jang, Jaehyeon Kang, ByeongKwon Ju, and KwangEun Ko

   Computers and Electronics in Agriculture, 2022

   Abs DOI

   This paper presents the development of an autonomous harvesting robot system for tomato, a representative crop cultivated in the facility horticulture smart farm. The automated harvesting work using a robotic system is very challenging because of the appearance, environmental features, such as the atypical directions of the peduncles or their growing form in a bunch. Also, the robot system should enable the harvesting of the target fruit only without damaging other fruits, stems, and branches. Hence, this paper presents a deep learning network pipeline, Deep-ToMaToS, capable of three-level maturity classification and 6D pose (3D translation + 3D rotation) estimation of the target fruit simultaneously. Due to the difficulties encountered in building a large-scale dataset to train and test the deep learning model for the 6D pose estimation in the real world, we presented an automatic data collection scheme based on a photo-realistic 3D simulator environment. The robotic harvesting system includes a harvesting motion control algorithm based on the result of the 6D pose estimation. The overall process of the motion control phase is described along with the decision way of the appropriate final posture of the harvesting module mounted at the end-effector of the robot manipulator via removal of invalid motions getting out of the valid workspace or redundant motions. We conducted experiments on the 6D pose estimation based on the Deep-ToMaToS and the harvesting motion control in virtual and real smart farm environments. The experimental results showed a 6D pose estimation accuracy of 96 % based on the ADD_S metric, and the proposed harvesting motion control algorithm achieves the harvesting success rate of 84.5 % on average. The experimental results reveal that the harvesting robot system has significant potential to extend to harvesting works for other fruits and crops.

3. 

   Precise Camera–LiDAR Extrinsic Calibration Based on a Weighting Strategy Using Analytic Plane Covariances

   Gunhee Koo, Jaehyeon Kang, Bumchul Jang, and Nakju Doh

   IEEE Transactions on Instrumentation and Measurement (TIM), 2022

   Abs DOI

   Planarity is a practical feature for estimating the camera–LiDAR extrinsic parameters. LiDAR directly measures the 3-D plane in the form of a point cloud, but the camera captures the 2-D plane image which requires perspective-n-point (PnP) techniques for 3-D plane estimation. Likewise, these 3-D plane acquisition processes by LiDAR and camera are quite different; hence, their plane covariances from each sensor also differ. The plane measurement covariance originates from the LiDAR measurement noise; it can be directly calculated from a 3-D point cloud. The plane parameter covariances originate from the corner point errors in a 2-D image; however, their direct formulas have not been studied. In this study, we analytically derive the plane parameter covariances under the assumption of linear error propagation. To validate the proposed covariances, we implemented a simulation to observe how effectively the proposed covariances reflect on the true plane parameter covariances. Next, we validated that the proposed covariances could increase the performance of the camera–LiDAR extrinsic calibration. We compared our method with other calibration methods by varying the pixel noise levels and the number of board measurements in calibration simulations. In calibration field tests, we calibrated a scanning system equipped with a Ladybug5+ camera and a VLP-16 LiDAR and compared our method with other calibration methods.

2020

1. 

   Analytic Plane Covariances Construction for Precise Planarity-based Extrinsic Calibration of Camera and LiDAR

   Gunhee Koo, Jaehyeon Kang, Bumchul Jang, and Nakju Doh

   In IEEE International Conference on Robotics and Automation (ICRA), 2020

   Abs DOI

   IPlanarity of checkerboards is a widely used feature for extrinsic calibration of camera and LiDAR. In this study, we propose two analytically derived covariances of (i) plane parameters and (ii) plane measurement, for precise extrinsic calibration of camera and LiDAR. These covariances allow the graded approach in planar feature correspondences by exploiting the uncertainty of a set of given features in calibration. To construct plane parameter covariance, we employ the error model of 3D corner points and the analytically formulated plane parameter errors. Next, plane measurement covariance is directly derived from planar regions of point clouds using the out-of-plane errors. In simulation validation, our method is compared to an existing uncertainty-excluding method using the different number of target poses and the different levels of noise. In field experiment, we validated the applicability of the proposed analytic plane covariances for precise calibration using the basic planarity-based method and the latest planarity-and-linearity-based method.

2. 

   Modeling of architectural components for large-scale indoor spaces from point cloud measurements

   Gahyeon Lim, Youjin Oh, Dongwoo Kim, ChangHyun Jun, Jaehyeon Kang, and Nakju Doh

   IEEE Robotics and Automation Letters (RA-L), 2020

   Abs DOI

   In this letter, we propose a method to model architectural components in large-scale indoor spaces from point cloud measurements. The proposed method enables the modeling of curved surfaces, cylindrical pillars, and slanted surfaces, which cannot be modeled using existing approaches. It operates by constructing the architectural points from the raw point cloud after removing non-architectural (objects) points and filling in the holes caused by their exclusion. Then, the architectural points are represented using a set of piece-wise planar segments. Finally, the adjacency graph of the planar segments is constructed to verify the fact that every planar segment is closed. This ensures a watertight mesh model generation. Experimentation using 14 different real-world indoor space datasets and 2 public datasets, comprising spaces of various sizes-from room-scale to large-scale (12,557 m2), verify the accuracy of the proposed method in modeling environments with curved surfaces, cylindrical pillars, and slanted surfaces.

3. 

   Automatic targetless camera–LIDAR calibration by aligning edge with Gaussian mixture model

   Jaehyeon Kang and Nakju Doh

   Journal of Field Robotics (JFR), 2020

   Abs DOI

   This paper presents a calibration algorithm that does not require an artificial target object to precisely estimate a rigid-body transformation between a camera and a light detection and ranging (LIDAR) sensor. The proposed algorithm estimates calibration parameters by minimizing a cost function that evaluates the edge alignment between two sensor measurements. In particular, the proposed cost function is constructed using a projection model-based many-to-many correspondence of the edges to fully exploit measurements with different densities (dense photometry and sparse geometry). The alignment of the many-to-many correspondence is represented using the Gaussian mixture model (GMM) framework. Here, each component of the GMM, including weight, displacement, and standard deviation, is derived to suitably capture the intensity, location, and influential range of the edge measurements, respectively. The derived cost function is optimized by the gradient descent method with an analytical derivative. A coarse-to-fine scheme is also applied by gradually decreasing the standard deviation of the GMM to enhance the robustness of the algorithm. Extensive indoor and outdoor experiments validate the claim that the proposed GMM strategy improves the performance of the proposed algorithm. The experimental results also show that the proposed algorithm outperforms previous methods in terms of precision and accuracy by providing calibration parameters of standard deviations less than 0.6° and 2.1 cm with a reprojection error of 1.78 for a 2.1-megapixel image (2,048 × 1,024) in the best case.

2019

1. 

   Automatic spatial template generation for realistic 3D modeling of large-scale indoor spaces

   Janghun Hyeon, Hyunga Choi, JooHyung Kim, Bumchul Jang, Jaehyeon Kang, and Nakju Doh

   In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2019

   Abs DOI

   This paper proposes a realistic indoor modeling framework for large-scale indoor spaces. The proposed framework reduces the geometric complexity of an indoor model to efficiently represent large-scale environments for image-based rendering (IBR) approaches. For this purpose, the proposed framework removes geometrically excluded objects (GEOs) in point cloud and images, which represent the primary factors in high geometric complexity. In particular, GEOs are coherently removed from all images using a global geometry model. Then, the remaining holes are inpainted using globally consistent guidelines, to achieve accurate image blending in IBR approaches. The experimental results verify that the proposed GEO removal framework provides efficient point clouds and images for realistic indoor modeling in large-scale indoor spaces.

2017

1. 

   Towards a realistic indoor world reconstruction: Preliminary results for an object-oriented 3D RGB-D mapping

   ChangHyun Jun, Jaehyeon Kang, Suyong Yeon, Hyunga Choi, Tae-Young Chung, and Nakju Doh

   Intelligent Automation & Soft Computing, 2017

   Abs DOI

   A real world reconstruction that generates cyberspace not from a computer graphics tool, but from the real world, has been one of the main issues in two different communities of robotics and computer vision under different names of Simultaneous Localization And Mapping (SLAM) and Structure from Motion (SfM). However, there have been few trials that actively integrate SLAM and SfM for possible synergy. This paper shows the real world reconstruction can be enabled through this integration. As a result, the preliminary map has been generated of which five subgoals are: Realistic view (RGB), accurate geometry (depth), applicability to multi-floor indoor building, initial classification of a possible set of objects, and full automation. To this end, an engineering framework of “Acquire-Build-Comprehend (ABC)” is proposed, through which a sensor system acquires an RGB-Depth point cloud from the real world, builds a three-dimensional map, and comprehends this map to yield the possible set of objects. Its performance is demonstrated by building a map for three levels of indoor building of which volume is 1,408 m3.

2016

1. 

   Full-DOF calibration of a rotating 2-D LIDAR with a simple plane measurement

   Jaehyeon Kang and Nakju Doh

   IEEE Transactions on Robotics (T-RO), 2016

   Abs DOI

   This paper proposes a calibration method that accurately estimates six parameters between the two centers of 2-D light detection and ranging (LIDAR) and a rotating platform. This method uses a simple plane, and to the best of our knowledge, it is the first to enable full-degree-of-freedom (DOF) estimation without additional hardware. The key concept behind this method is a decoupling property, in which the direction of a line on a plane does not contain 3-DOF translation terms. Based on this, a cost function for rotation is constructed, and 3-DOF rotation parameters are estimated. With this rotation, the remaining 3-DOF translation parameters are calculated in a manner that minimizes the cost function for translation only. In other words, an original 6-DOF problem is decoupled into two 3-DOF estimation problems. Given these cost functions, degenerate cases are mathematically analyzed for known cases (incomplete), and the robustness is numerically tested for all possible cases (complete). The performance of the method is validated by extensive simulations and experimentations, and the estimated parameters from the proposed method demonstrate better accuracy than previous methods.

2. 

   Accurate Continuous Sweeping Framework in Indoor Spaces With Backpack Sensor System for Applications to 3-D Mapping

   Keonyong Lee, Soo-Hyun Ryu, Suyong Yeon, HyunGi Cho, ChangHyun Jun, Jaehyeon Kang, Hyunga Choi, Janghun Hyeon, Insik Baek, Woonhyung Jung, Hanul Kim, and Nakju Doh

   IEEE Robotics and Automation Letters (RA-L), 2016

   Abs DOI

   In indoor environments, there exists a few distinctive indoor spaces’ features (ISFs). However, up to our knowledge, there is no algorithm that fully utilizes ISF for accurate 3-D SLAM. In this letter, we suggest a sensor system that efficiently captures ISF and propose an algorithm framework that accurately estimates sensor’s 3-D poses by utilizing ISF. Experiments conducted in six representative indoor spaces show that the accuracy of the proposed method is better than the previous method. Furthermore, the proposed method shows robust performances in a sense that a set of adjusted parameters of the related algorithms does not need to be recalibrated as target environment changes. We also demonstrate that the proposed method not only generates 3-D depth maps but also builds a dense 3-D RGB-D map.

2014

1. 

   Analysis of the reference coordinate system used in the EKF-based SLAM

   Keonyong Lee, Soo-hyun Ryu, Jaehyeon Kang, Hyunga Choi, ChangHyun Jun, and Nakju Doh

   In International Conference on Ubiquitous Robots and Ambient Intelligence (URAI), 2014

   Abs DOI

   In this paper we analyze the differences in the EKF-SLAM frameworks that are constructed using different coordinate system. First, we derive the EKF equations in the world coordinate system through modification of the EKF equations in the robot coordinate system. When the reference coordinate system is changed, the equations change due to its nonlinear behavior. We identify four EKF-SLAM frameworks, and then we demonstrate that any framework can be used in EKF-SLAM by simulation and real-life experiments.

2. 

   Robust-PCA-based hierarchical plane extraction for application to geometric 3D indoor mapping

   Suyong Yeon, ChangHyun Jun, Hyunga Choi, Jaehyeon Kang, Youngmok Yun, and Nakju Doh

   Industrial Robot: An International Journal, 2014

   Abs DOI

   Purpose - The authors aim to propose a novel plane extraction algorithm for geometric 3D indoor mapping with range scan data. Design/methodology/approach - The proposed method utilizes a divide-and-conquer step to efficiently handle huge amounts of point clouds not in a whole group, but in forms of separate sub-groups with similar plane parameters. This method adopts robust principal component analysis to enhance estimation accuracy. Findings - Experimental results verify that the method not only shows enhanced performance in the plane extraction, but also broadens the domain of interest of the plane registration to an information-poor environment (such as simple indoor corridors), while the previous method only adequately works in an information-rich environment (such as a space with many features). Originality/value - The proposed algorithm has three advantages over the current state-of-the-art method in that it is fast, utilizes more inlier sensor data that does not become contaminated by severe sensor noise and extracts more accurate plane parameters.

3. 

   Spline-based RRT path planner for non-holonomic robots

   Kwangjin Yang, Sangwoo Moon, Seunghoon Yoo, Jaehyeon Kang, Nakju Doh, Hong Bong Kim, and Sanghyun Joo

   Journal of Intelligent & Robotic Systems, 2014

   Abs DOI

   Planning in a cluttered environment under differential constraints is a difficult problem because the planner must satisfy the external constraints that arise from obstacles in the environment and the internal constraints due to the kinematic/dynamic limitations of the robot. This paper proposes a novel Spline-based Rapidly-exploring Random Tree (SRRT) algorithm which treats both the external and internal constraints simultaneously and efficiently. The computationally expensive numerical integration of the system dynamics is replaced by an efficient spline curve parameterization. In addition, the SRRT guarantees continuity of curvature along the path satisfying any upper-bounded curvature constraints. This paper presents the underlying theory to the SRRT algorithm and presents simulation and experiment results of a mobile robot efficiently navigating through cluttered environments.