Initialising ...
Initialising ...
Initialising ...
Initialising ...
Initialising ...
Initialising ...
Initialising ...
今渕 貴志; 羽成 敏秀; 川端 邦明
Proceedings of 2025 IEEE/SICE International Symposium on System Integration (SII2025), p.1416 - 1421, 2025/01
This paper describes a 3D reconstruction based on grouping similar structures for the aim of generating 3D information for understanding the workspace from the images acquired inside the Primary Containment Vessel (PCV) of the Fukushima Daiichi Nuclear Power Station. In the decommissioning works, preliminary surveys are carried out in the PCV, and the workers need to understand the workspace from a large amount of camera images, which requires a great deal of effort. We are currently working on 3D reconstruction from camera images of the PCV, however, one of the challenges is to improve the visibility of reconstructed model containing noise and artifact. In this study, we propose a method of grouping similar structures on the image and utilizing predicted group labels for 3D reconstruction process to highlight structures shapes and to refine 3D modeling. Our key idea is to perform unsupervised segmentation for grouping similar structures that are suitable for images acquired in the PCV because they are difficult to assign correct semantics for unclear structures and the few learning resources. We show on the reasonable performance of our method by validating it by video images of a typical plant environment and survey videos of the PCV taken under adverse conditions such as radiation noise.
羽成 敏秀; 中村 啓太*; 今渕 貴志; 川端 邦明
Journal of Robotics and Mechatronics, 36(6), p.1537 - 1549, 2024/12
本稿では、時系列画像から3次元モデルを効率的に生成するための画像選択法を導入した立体復元処理について述べる。効率的な立体復元に適した画像を得るために、時系列画像中の冗長画像を除去する画像選択法の適用を試みた。提案手法では、オプティカルフローと固定しきい値に基づいて、時系列画像から適切な画像を選択する。その結果、提案手法により、カメラで取得した時系列画像に基づく立体復元処理の計算量を削減することができた。その結果、立体復元精度を一定に保ったまま、立体復元処理の計算量を削減できることを確認した。
谷藤 祐太; 羽成 敏秀; 川端 邦明
Proceedings of International Topical Workshop on Fukushima Decommissioning Research 2024 (FDR2024) (Internet), 3 Pages, 2024/10
In this paper, we describe the results of a feasibility study of a noise reduction method from images using deep learning technology for decommissioning work. Currently, remotely operated robots have been used for the decommissioning work at the Fukushima Daiichi Nuclear Power Station (FDNPS) due to the high radiation environment. We have been conducting research and development for providing clear images during operations by removing only noise from images containing noise to contribute to safe and secure decommissioning work. Since we do a feasibility study of the noise reduction method using deep learning, the main target is not the video, but rather images, which are components of the video. We adopted the approach of building a learning model that can cope with various types of noise by training many noisy images in the deep learning process.
中村 啓太*; 馬場 啓多*; 渡部 有隆*; 羽成 敏秀; 松本 拓*; 今渕 貴志; 川端 邦明
Artificial Life and Robotics, 29(4), p.546 - 556, 2024/09
This paper describes a method for integrating multiple dense point clouds using a shared landmark to generate a single real-scale integrated result for photogrammetry. It is difficult to integrate high-density point clouds reconstructed by photogrammetry because the scale differs with each photogrammetry. To solve this problem, this study places a QR code of known sizes, which is a shared landmark, in the reconstruction target environment and divides the reconstruction target environment based on the position of the QR code that is placed. Then, photogrammetry is performed for each divided environment to obtain each high-density point cloud. Finally, we propose a method of scaling each high-density point cloud based on the size of the QR code and aligning each high-density point cloud as a single high-point cloud by partial-to-partial registration. To verify the effectiveness of the method, this paper compares the results obtained by applying all images to photogrammetry with those obtained by the proposed method in terms of accuracy and computation time. In this verification, ideal images generated by simulation and images obtained in real environments are applied to photogrammetry. We clarify the relationship between the number of divided environments, the accuracy of the reconstruction result, and the computation time required for the reconstruction.
中村 啓太*; 羽成 敏秀; 今渕 貴志; 川端 邦明
Proceedings of 2024 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM 2024), p.7 - 8, 2024/07
写真測量法は、対象物を撮影した複数の画像から3次元的に復元する技術である。実際の撮影では、対象物を撮影するカメラの画角に振動が発生し、写真測量に適した画像を撮影できず、対象物を復元できない場合がある。そこで、この振動をシミュレーション上で乱数を用いて実装し、振動の大きさが写真測量法で得られる復元結果に与える影響を検証した。検証の結果、振動の大きさと3次元復元の成功率との関係が示された。
松本 拓; 羽成 敏秀; 川端 邦明; 中村 啓太*; 八代 大*
Artificial Life and Robotics, 29(2), p.358 - 371, 2024/05
This paper describes a three-dimensional (3D) modeling method for sequentially and spatially understanding situations in unknown environments from an image sequence acquired from a camera. The proposed method chronologically divides the image sequence into sub-image sequences by the number of images, generates local 3D models from the sub-image sequences by the Structure from Motion and Multi-View Stereo (SfM-MVS), and integrates the models. Images in each sub-image sequence partially overlap with previous and subsequent sub-image sequences. The local 3D models are integrated into a 3D model using transformation parameters computed from a camera trajectory estimated by the SfM-MVS. In our experiment, we quantitatively compared the quality of integrated models with a 3D model generated from all images in a batch and the computational time to obtain these models using three real data sets acquired from a camera. Consequently, the proposed method can generate a quality integrated model that is compared with a 3D model using all images in a batch by the SfM-MVS and reduce the computational time.
横村 亮太*; 後藤 雅貴*; 吉田 健人*; 割澤 伸一*; 羽成 敏秀; 川端 邦明; 福井 類*
IEEE Robotics and Automation Letters (Internet), 9(4), p.3275 - 3282, 2024/04
被引用回数：0 パーセンタイル：0.00(Robotics)廃炉作業におけるロボットの遠隔操作のエラーを低減するため、作業環境を常時観察できるRail DRAGONを開発した。Rail DRAGONは、原子炉格納容器(PCV)内に長尺の軌道構造体(レールモジュール)を組み立てて押し込み、そのレール上に複数台のモニタリングロボットを繰り返し配置することで構築され、高放射線環境下での常時監視を可能にしたものである。特に、Rail DRAGONの構成要素である屈曲可能なレールモジュール、直線状のレールモジュール、基部ユニット、モニタリングロボットを開発した。具体的には、可搬性・作業性に優れた超長尺多関節構造物の実現手法を提案・実証している。また、処分を考慮しつつ、容易に展開・交換が可能な観測機器の展開手法を提案し、その実現可能性を検証する。
中村 啓太; 羽成 敏秀; 松本 拓; 川端 邦明; 八代 大*
Journal of Robotics and Mechatronics, 36(1), p.115 - 124, 2024/02
During the decommissioning activities, a movie was shot inside the reactor building during the investigation of the primary containment vessel by applying photogrammetry, which is one of the methods for three-dimensional (3D) reconstruction from images, to the images from this movie, it is feasible to perform 3D reconstruction of the environment around the primary containment vessel. However, the images from this movie may not be suitable for 3D reconstruction because they were shot remotely by robots owing to limited illumination, high-dose environments, etc. Moreover, photogrammetry has the disadvantage of easily changing 3D reconstruction results by simply changing the shooting conditions. Therefore, this study investigated the accuracy of the 3D reconstruction results obtained by photogrammetry with changes in the camera angle of view under shooting conditions. In particular, we adopted 3D computer graphics software to simulate shooting target objects for 3D reconstruction in a dark environment while illuminating them with light for application in decommissioning activities. The experimental results obtained by applying artificial images generated by simulation to the photogrammetry method showed that more accurate 3D reconstruction results can be obtained when the camera angle of view is neither too wide nor too narrow when the target objects are shot and surrounded. However, the results showed that the accuracy of the obtained results is low during linear trajectory shooting when the camera angle of view is wide.
馬場 啓多*; 渡部 有隆*; 中村 啓太*; 松本 拓; 羽成 敏秀; 川端 邦明
Proceedings of 29th International Symposium on Artificial Life and Robotics (AROB 29th 2024) (Internet), p.751 - 756, 2024/01
This study proposes a partial-to-partial point cloud registration method based on estimated parameters in photogrammetry and QR code. Some research and development on Generating a 3D map of the workspace by photogrammetric methods have been proposed for the decommissioning work at the Fukushima Daiichi Nuclear Power Plant. Photogrammetry is a method for 3D reconstruction of the location and shape of target objects from many images, and the processing time depends on the number of images. Considering the reconstruction of a large area, the number of images increases, and processing time also increases significantly. To reduce such computational time, this study considers applying SfM-MVS (Structure from Motion and Multi-View Stereo), which is one of the photogrammetry methods, to each segmented image group, aligning each obtained result, integrating them, and creating a model of the entire space. This alignment is called partial-to-partial registration and it is difficult to find the correspondence points for registration. Therefore, we place markers such as QR codes in the target reconstruction space to make it easy to find the correspondence points. We adopt the QR code as a 2D code because it is easy to reconstruct by photogrammetry. In this paper, we discuss the validity of this approach by comparing it with the integrated model using all images applying SfM-MVS. We verify the validation of the proposed method by simulation due to the large number of images and the ease of modifying the environment. The experiment about varying the number of image divisions shows that the reconstruction result from all images is more accurate than the integrated result. However, all of these models have high reconstruction accuracy. Moreover, the accuracy of the integrated model does not depend on the number of divisions.
松本 拓; 羽成 敏秀; 川端 邦明; 八代 大*; 中村 啓太*
Proceedings of 2023 IEEE International Conference on Robotics and Biomimetics (IEEE ROBIO 2023) (Internet), 7 Pages, 2023/12
This paper describes an automatic system that performs three-dimensional (3D) modeling from image sequences acquired from a camera during a survey to understand the state of the environment sequentially. The developed system sequentially generates local 3D models from image sequences acquired from a camera by the Structure from Motion and Multi-View Stereo (SfM-MVS) and integrates the models. This system, consisting of a camera, a computer, and storage, handles streaming, image acquisition, 3D modeling, and integration processes. The system is designed to perform each process in parallel while performing image acquisition. Local 3D models are integrated based on the similarity of camera trajectories, which correspond to overlapped images in each sequence, estimated from SfM. Our experiments verified the operation of the developed system from the image acquisition to the integration of local 3D models in two cases. Consequently, we confirmed that an integrated model could be generated by automatically executing each process in the system through experiments.
松本 拓; 羽成 敏秀; 川端 邦明; 八代 大*; 中村 啓太*
Proceedings of 22nd World Congress of the International Federation of Automatic Control (IFAC 2023) (Internet), p.12107 - 12112, 2023/07
This paper describes a method that integrates Three-Dimensional (3D) environment models reconstructed from image sequences to reduce the computation time of 3D environment modeling that estimates camera poses and simultaneously reconstructs a 3D environment model from images based on photogrammetry. However, 3D environment modeling is time-consuming when using many images because it finds correspondence points between them by feature matching. Therefore, we assume that the computation time is reduced by reconstructing 3D environment models from image sequences divided from an image sequence because the number of images used in 3D environment modeling becomes less. However, it is difficult to integrate the 3D environment models because the scale between them may not be the same, and overlapping regions between 3D environment models are small for integrating the models. In this paper, we propose a method that integrates 3D environment models based on camera trajectories corresponding to overlapped images between image sequences used in 3D environment modeling. To integrate them, transformation parameters are calculated from poses of camera trajectories between 3D environment models. After that, the transformed camera trajectory is aligned using coarse and fine registration. Consequently, compared with 3D environment modeling that processes an image sequence in batch, the proposed method could reduce the computation time and reconstruct a comparable integrated model.
羽成 敏秀; 川端 邦明; 中村 啓太*
Proceedings of 22nd World Congress of the International Federation of Automatic Control (IFAC 2023) (Internet), p.11597 - 11602, 2023/07
本論文では、時系列画像に基づく立体復元の計算効率を向上させるための多峰性検出による画像選択手法について述べる。立体復元にかかる計算時間を低減するためには、時系列画像から適切な画像を選択する必要がある。そのため、画像選択過程において統計的な検定による多峰性検出を導入し、カメラから取得した画像間のオプティカルフローに基づく変位の健全性評価に適用した。その結果、立体復元の計算時間を低減するために、時系列画像から適した画像を抽出可能であることが示唆された。このことから、提案手法によって選択された画像群は、立体復元計算の効率的な実行に寄与できることを確認した。
中村 啓太; 羽成 敏秀; 川端 邦明; 馬場 啓多*
Artificial Life and Robotics, 28(2), p.352 - 360, 2023/02
In photogrammetry, when each image captures the features of the target object for 3D reconstruction, a more highly accurate 3D reconstruction can be obtained in a shorter time from a small number of images. For this reason, effective images for 3D reconstruction must be selected. We generate test images by constructing a virtual environment and changing the shooting conditions based on the simulation for this image selection. In particular, we focus on linear trajectory shooting, in which the camera moves on a straight rail and obtains images. We verify the 3D reconstruction considering the calculation time reduction based on effective image selection. The experimental result shows that the camera pose estimation can be improved by using images obtained at multiple shooting angles in the case of linear trajectory shooting. The additional experimental result reveals that the calculation time for the reconstruction result is reduced by applying the images selected at regular intervals.
松本 拓; 羽成 敏秀; 川端 邦明; 八代 大*; 中村 啓太*
Proceedings of 28th International Symposium on Artificial Life and Robotics (AROB 28th 2023) (Internet), p.768 - 773, 2023/01
This paper reports on an automatic process for 3D environmental modeling from acquired image sequences. We have studied methods of 3D environment modeling to understand unknown environments. However, 3D environment modeling using acquired image sequences is a time-consuming task. When 3D environment modeling is performed using image sequences acquired in an unknown environment, the generated models do not always produce the expected result. Therefore, we consider that the quality of the generated 3D environment models could be quickly verified by realizing the sequential presentation of the generated models. To verify the effectiveness of the automatic process, we implemented it under different execution environments. Consequently, the automatic process could generate a 3D environment model from acquired image sequences.
中村 啓太; 馬場 啓多*; 羽成 敏秀; 川端 邦明
Proceedings of 2022 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM 2022), p.1497 - 1498, 2022/07
Although the three-dimensional (3D) reconstruction result obtained by structure from motion and multi-view stereo from an image can visually confirm whether 3D reconstruction is achieved, it is not possible to scale the obtained 3D reconstruction result. Therefore, we verify whether the 3D reconstruction can be scaled based on the size of the two-dimensional (2D) code by simultaneously performing a 3D reconstruction of its target and its code whose size is known. Experimental results show that the scaled 3D reconstruction is highly accurate according to the size of the reconstructed 2D code.
羽成 敏秀; 川端 邦明; 中村 啓太
Proceedings of 2022 IEEE/SICE International Symposium on System Integration (SII 2022) (Internet), p.1041 - 1045, 2022/01
被引用回数：9 パーセンタイル：97.14(Computer Science, Interdisciplinary Applications)本稿では、時系列画像からの立体復元計算を効率に行うための画像選択法について述べる。立体復元計算の効率を向上させるためには、時系列画像から適切な画像を選択する必要がある。そこで本研究では、遠隔操作ロボットに搭載されたカメラから得られた画像間の変位に基づくしきい値を検討した。その結果、提案手法は時系列画像からオプティカルフローに基づくしきい値を設定することで、効率的な立体復元のために適切な画像を選択できることが確認された。これにより、重複画像や類似度の高い画像を除外して効率的な立体復元を行うことで、計算コストを削減できることが示唆された。
中村 啓太; 羽成 敏秀; 川端 邦明
Proceedings of 27th International Symposium on Artificial Life and Robotics (AROB 27th 2022) (Internet), p.771 - 775, 2022/01
画像からの三次元復元において、画像枚数が少なくても、各画像が三次元復元の対象物の特徴を捉えていれば、短時間で高精度な三次元復元を実現できることが知られている。そこで、三次元復元に有効な画像を選択するために、シミュレーションによる仮想環境の構築と撮影条件の変更により、画像選択のためのテストデータを生成することが考えられる。本研究では、直線レール上をカメラが移動して画像を撮影する場合に着目し、この場合における、三次元復元にかかる計算時間短縮を考慮した有効な画像選択条件をシミュレーションにより調査・検証する。
中村 啓太; 馬場 啓多*; 吉川 拓真*; 羽成 敏秀; 川端 邦明; 松本 拓
New Trends in Intelligent Software Methodologies, Tools and Techniques; Frontiers in Artificial Intelligence and Applications, Vol.355, p.427 - 438, 2022/00
It is generally difficult to measure complex shapes such as stairs with high accuracy for indoor environment scanning by the robot. Therefore, we consider the 3D (three-dimensional) reconstruction of stairs using SfM (Structure from Motion) and MVS (Multi-View Stereo) which perform 3D reconstruction from images acquired by the robot vision. In this study, we verify whether it is possible to acquire a 3D reconstruction result of stairs by using images shot while ascending and descending the stairs as input to the reconstruction method. To calculate the accuracy of the reconstruction result, we use 3D computer graphics software to generate artificial image data to be applied to the 3D reconstruction. Experimental results show that 3D reconstruction results of the stairs are more accurate by applying both images shot when ascending and descending stairs to the 3D reconstruction methods.
羽成 敏秀; 川端 邦明; 中村 啓太*; 成瀬 継太郎*
Proceedings of International Workshop on Nonlinear Circuits, Communications and Signal Processing (NCSP 2020) (Internet), p.242 - 245, 2020/02
本発表では、遠隔操作ロボットによって収集された時系列画像から効率的な立体復元を実現するための画像選定手法を提案する。提案手法は、画像間で求められるオプティカルフローに基づいて立体復元の計算に適した画像を選択する。この提案手法を用いて、移動と停止を繰り返す遠隔操作ロボットによって記録された時系列画像から立体復元に適切な画像を選択できることを確認した。その結果、提案手法で選択された比較的少数の画像で立体復元を効率的に実行できることを実証した。
Wright, T.*; 羽成 敏秀; 川端 邦明; Lennox, B.*
Proceedings of 17th International Conference on Ubiquitous Robots (UR 2020) (Internet), p.315 - 321, 2020/00
In exploratory robotics for nuclear decommissioning, environmental understanding is key. Sites such as Fukushima Daiichi Power Station and Sellafield often use manually controlled or semi-autonomous vehicles for exploration and monitoring of assets. In many cases, robots have limited sensing capabilities such as a single camera to provide video to the operators. These limitations can cause issues, where a lack of data about the environment and limited understanding of depth within the image can lead to a mis-understanding of asset state or potential damage being caused to the robot or environment. This work aims to aid operators by using the limited sensors provided i.e. a single monocular camera, to allow estimates of the robot's surrounding environments to be generated in situ without having to off load large amounts of data for processing. This information can then be displayed as a mesh and manipulated in 3D to improve the operator awareness. Due to the target environment for operation being radioactive, speed is prioritised over accuracy, due to the damaging effects radiation can cause to electronics. In well lit environments images can be overlaid onto the meshes to improve the operators understanding and add detail to the mesh. From the results it has been found that 3D meshes of an environment/object can be generated in an acceptable time frame, less than 5 minutes. This differs from many current methods which require offline processing due to heavy computational requirement of Photogrammetry, or are far less informative giving data as raw point clouds, which can be hard to interpret. The proposed technique allows for lower resolution meshes good enough for avoiding collisions within an environment to be generated during a mission due to it's speed of generation, however there are still several issues which need to be solved before such a technique is ready for deployment.
