Intel RealSense 3D Camera for SLAM

As technologies advance and real-time mapping and imaging become more important across the board, the importance of SLAM in the technological market will only increase. SLAM or Simultaneous Localization And Mapping refer to the methodology used to map an environment at the same time it is being traversed. In other words, SLAM offers a way for humans and machines to better understand their environment and react to it in real-time.

However, SLAM like all forms of mapping and imagining is tied to the efficacy of its sensors, particularly its cameras. This is why to get the best results and keep advancing on the field it is important to test and compare new cameras and the results they offer, which is why today we’ll take a look at Intel’s RealSense 3D Camera technology and what it offers to SLAM.

What is the technology behind Intel RealSense?

Intel RealSense is a lineup of stereo depth cameras designed to offer solutions to depth perceptions in a wide variety of fields including but not limited solely to SLAM. However, to truly grasp what RealSense offers and how it is different from the alternatives we need to review everything that is encompassed in the RealSense brand.

Intel RealSense is comprised of both its hardware which supports 3D imaging through the use of multiple cameras in a stereo array to interpret depth in real-time through RGBD cameras. This information is then delivered to an onboard processor that performs all the depth calculations through the use of its specialized software to provide images that capture true depth without any need for external calibration or additional configurations.

Intel RealSense ultimately aims to provide a ready-to-use stereo camera that is approachable yet competitive in a market where depth perception is becoming more important and commonplace by the day.

System Overview

While Intel RealSense cameras are designed to operate with minimal configuration right out of the box this doesn’t mean they are a complete SLAM system on their own, as such proper installation, configuration, and hardware sync will be needed to make the most out of these cameras for imaging and mapping.

The main elements needed to set up a SLAM system are your Intel RealSense camera, a companion computer, two USB drives, and an external computer for software installation and execution. The camera must be mounted looking forwards on the vehicle or rig in a way that minimizes vibration, then it must be connected to the companion computer through its USB ports.

Once the camera and computer software has been properly configured the system is ready to tackle imaging and mapping tests and is in practice a fully functioning SLAM system. Do keep in mind however that compared to a Dioram system Intel RealSense proved to have issues with relocalization and the Y-axis in low visibility settings, so there will be a learning curve to make the most of this hardware.

RealSense Cameras:

Currently, the Intel RealSense range of cameras includes a multitude of models for various uses and budgets. The D and L series are currently the ones being most heavily advertised by the company with the D415, in particular, being sold at an average MSRP of $259.

While all of the cameras in the RealSense lineup have stereo functions and are capable of depth perception not all of them are necessarily optimized for SLAM or able to function both indoors and outdoors. So, to provide an accurate comparison between these cameras and other sensors or infrared alternatives in the market we will be focusing on 3 specific examples of the line and offering a general overview of their functions and viability for SLAM.

Intel 3d Camera SDK

To be accurate the Intel 3D Camera SDK is not a model in the RealSense lineup, but rather Intel’s very own SDK (Software Development Kit) for use alongside their range of stereo cameras. Nonetheless explaining its features and functions is important to understand the possible roles and applications of RealSense technology on SLAM systems.

This SDK 2.0. library allows users to view images, record them, change settings or update the firmware of its accompanying camera. On top of this, the software includes various filters that allow the user to fill potential gaps in images and recordings to provide a more accurate representation of reality even if there are hardware limitations in place. The RealSense SDK is open source and works on Mac, Windows, Linux, and Android so the platform is versatile and can be potentially used with other brands of cameras.

Depth Camera d400

The D400 series of depth cameras rely on stereo vision to obtain depth information, which means they don’t need to rely on lasers or other emitters like LIDAR and ToF systems. Stereoscopic vision largely operates on a similar logic as human vision, that is to say, that two cameras are used simultaneously and through triangulation, the camera can reconstruct an image that amounts for depth.

the D400 series in specific uses two infrared cameras for its stereoscopic features alongside an RGB camera, meaning that it can provide up to 4 different data products from a single sense module. The D400 series can provide users with an RGB image, depth image, left infrared image, and a right infrared image. Due to the way stereoscopic vision works the D400 cameras tend to have more error variance the farther an item is from them, however, to combat this issue they come with an infrared projector that can help obtain better features on distant objects.

RealSense d415 Camera

The RealSense D415 can be seen as something of an upgrade to the D400 series, and at first glance, a lot of the features might appear equivalent. The D415 is once again a stereoscopic camera with infrared and RGB capabilities designed to acquire depth from objects through triangulation and with additional infrared support to enhance the resulting image or recording.

However, the key difference with the D400 appears when we take a look at its field of view. The D415 has a very focused field of view which means it can see a reduced range compared to its earlier counterpart but offers a much more accurate and precise depth map. The D415 offers the highest quality per degree ratio in the entire RealSense family and this makes it ideal as a secondary camera for detailed 3D reconstruction.

Intel RealSense ROS

The Intel RealSense ROS is a series of packages designed to enable the R200, F200, SR300, and D400 RealSense cameras to work directly with ROS as a main or auxiliary source of visual input. There are four packages included in the RealSense ROS which mainly amount to communication drivers and camera nodes for librealsense.

The main advantage of these included packages is that they provide an efficient and straightforward way to integrate RealSense technology into existing projects and libraries, and help promote the RealSense suite of products as ready for use straight from the box. Since ROS is additionally open source this means that it’s easy to obtain and distribute updated versions of the libraries in the case of bugs and there is always a constant source of external support for the range of RealSense products.

Software

While RealSense cameras count on a dedicated software suite for their ease of use and installation, this doesn’t mean the above-mentioned programs are enough to construct a fully functioning SLAM system. The RealSense SDK is a reliable tool to update firmware and for image correction, and the ROS packages are useful to integrate the cameras into existing projects but interested users will still need to consider APSync and ArduPilot to get their system off the ground.

APSync is an open-source software manager for companion computers and is a necessary installation to make sure the SLAM system has on-board processing and analysis capabilities. ArduPilot on the other hand is an autopilot system that allows for the programming of routes and tasks for a wide variety of automated vehicles including flying ones. While ArduPilot isn’t the only option available any SLAM system that isn’t reliant on human movement will require autopilot software and ArduPilot is one of the most widely supported ones in the field.

Key Application Areas

in general terms, the main applications for a SLAM system supported by RealSense cameras continue to be mapping and robotics navigation. The main advantage that RealSense devices offer compared to other sensor technologies is their efficiency and relative simplicity. Stereoscopic vision is easy to process and offers almost immediate results in in-depth perception which can provide autonomous robots with the kind of visual input required to traverse unmapped environments and react to obstacles.

It is important to take into account the limitations and peculiarities of infrared, however, as RealSense cameras are very sensitive to visual obstructions and have a limited range of view compared to LiDAR technologies.

Summary

Intel’s lineup of RealSense stereoscopic cameras proves to be an interesting proposal in the current market. Fully functioning stereoscopic cameras designed to seamlessly integrate with the existing software suites used in SLAM and autonomous robotics is an enticing prospect and one that is backed by a wide variety of models and releases based on the need of each user.

However, at the same time, potential buyers must keep in mind that a camera does not make a complete SLAM system and that due to the inherent limitations of infrared RealSense cameras might be better suited as auxiliary sensors to obtain additional depth information on a smaller range.