Vision Sensor Devices¶

Overview ¶

Vision sensors such as cameras and range sensors can be mounted on Choreonoid body models as devices. This page describes the list of available vision sensor devices and the main parameters of each device.

Vision sensor devices are described as device nodes under the elements of the link on which they are mounted in body-format model files. The outputs of the described devices can be simulated with the method explained in Vision Sensor Simulation.

The main parameters common to all vision sensor devices are as follows.

Parameter	Default value	Meaning
`name`	—	The name of the device. It is used for obtaining the device from controllers, etc.
`id`	—	The ID of the device.
`translation` / `rotation`	—	The position and orientation of the sensor in the parent link coordinate system.
`on`	true	The initial state (on/off) of the sensor.
`frame_rate`	30	Specifies how many measurement outputs the sensor produces per second.
`optical_frame`	gl	Specifies the optical coordinate system of the sensor from “gl”, “cv”, and “robotics”. See Camera Node for details.

Camera ¶

A camera that outputs color images. In model files, it is described as a device node of type: Camera, with COLOR (the default) specified for format.

The main parameters are as follows.

Parameter	Default value	Meaning
`format`	COLOR	The type of information to be output. With COLOR, color images are output. For values other than COLOR, refer to the range camera described below.
`lens_type`	NORMAL	Specifies the lens type from “NORMAL”, “FISHEYE”, and “DUAL_FISHEYE”.
`width` / `height`	640 / 480	The resolution (width and height) of the output images.
`field_of_view`	45°	The field of view of the camera.
`near_clip_distance` / `far_clip_distance`	0.04 / 200	The distances to the near and far clip planes that define the imaging range [m].

For the details of the description in model files, see Camera Node.

Range Camera (RangeCamera)¶

A camera that outputs depth images and 3D point clouds, which can be used to simulate RGB-D cameras and depth cameras. In model files, it is not an independent node type; a camera is internally treated as a range camera by specifying one of the following for the format of a Camera node:

DEPTH : Outputs depth information
COLOR_DEPTH : Outputs color and depth information
POINT_CLOUD : Outputs a 3D point cloud
COLOR_POINT_CLOUD : Outputs color information and a 3D point cloud

In addition to the camera parameters, the following parameters are available.

Parameter	Default value	Meaning
`min_distance` / `max_distance`	0.1 / 10	The minimum and maximum measurable distances [m].
`detection_rate`	1.0	The probability that a measurement value is obtained at each measurement point. By setting a value less than 1, missing measurements can be simulated.
`error_deviation`	0.0	The standard deviation of the Gaussian noise added to the measured distances [m]. Measurement errors can be simulated.

Range Sensor (RangeSensor)¶

A scanning range sensor like a laser scanner (LiDAR). In model files, it is described as a device node of type: RangeSensor. Both 2D LiDARs that scan only in the horizontal direction and 3D LiDARs that also scan in the vertical direction can be represented.

The main parameters are as follows.

Parameter	Default value	Meaning
`yaw_range`	120°	The angle of the scan range in the horizontal direction. The scan is performed symmetrically around 0 degrees. Set 0 when no horizontal scan is performed.
`yaw_step`	1°	The angle step of the horizontal scan. The number of samples can also be specified with `yaw_samples` instead.
`pitch_range`	0°	The angle of the scan range in the vertical direction. Set 0 when no vertical scan is performed.
`pitch_step`	0°	The angle step of the vertical scan. The number of samples can also be specified with `pitch_samples` instead.
`scan_rate`	30	The number of scans per second [Hz]. (Treated as the same value as frame_rate.)
`min_distance` / `max_distance`	0.1 / 10	The minimum and maximum measurable distances [m].
`detection_rate`	1.0	The probability that a measurement value is obtained at each measurement point. By setting a value less than 1, missing measurements can be simulated.
`error_deviation`	0.0	The standard deviation of the Gaussian noise added to the measured distances [m]. Measurement errors can be simulated.

For the details of the description in model files, see RangeSensor Node.

LivoxMid360 ¶

A device corresponding to the MID-360, a LiDAR by Livox. It becomes available by introducing LivoxMid360 Plugin. In model files, it is described as a device node of type: LivoxMid360.

The MID-360 is a LiDAR with a non-repetitive scanning method; it scans with a different pattern each time, which covers the field of view densely over time. This device outputs point clouds simulating this scanning pattern.

It is a device that extends RangeSensor, and the following parameters are available in addition to the RangeSensor parameters.

Parameter	Default value	Meaning
`samples`	24000	The number of measurement points per scan.
`angular_precision`	0.5°	The angular precision in the scanning directions.
`angle_seq_file`	—	A CSV file of the angle sequence that defines the scanning pattern. The scanning pattern data “${SHARE}/livox/scan_mode/mid360.csv” is bundled.

A model of the MID-360 is provided as share/model/misc/mid360.body, and a sample project of its simulation is provided as sample/VisionSimulator/Mid360.cnoid. Refer to them for description and usage examples.

Use in Simulation ¶

The outputs of these vision sensor devices can be simulated using the GL vision simulator. See Vision Sensor Simulation for the concrete procedure. If you want to apply effects such as noise and distortion to camera images, see Camera Effect Simulation.

Vision Sensor Devices¶

Overview¶

Camera¶