Skip to main content

Running ROS 2 on multiple machines

Chapter description​

ROS allows you to run nodes on a single robot as well as on dozens of robots. In this tutorial you will learn how to configure ROS to work on multiple computers. Two situations will be considered: the robots are in a local network, the robots are connected to the Internet. You should have a ROSbot and a computer handy to continue. Alternatively, you can use two computers for the simulation version of Gazebo.

This tutorial is dedicated to the ROSbot XL. However, most of the information and commands are compatible with all ROSbots, and most of the changes are simple and come down to launching the appropriate container, depending on your hardware configuration.

Available ROSbot platforms:

You can find the repository containing the final result after completing all the ROS 2 Tutorials here.

Connecting ROS 2 in LAN network​

ROS 2's default settings provide the necessary connection for all devices connected to the same network.

1. Connect device to the LAN

The first thing you need to do is connect them to the same LAN first. You can check the address of your device with the command.

ROSbot
hostname -I

Then you can verify the connections. On second device type:

Laptop
ping <ROSBOT_IP>

2. Run example

Now we can run any node or topic. So we will post a custom String message in the /msg subject.

ROSbot
ros2 topic pub -r 1 /msg std_msgs/msg/String data:\ 'Hello, ROSbot here'
Laptop
ros2 topic echo /msg

You should see a greeting. It's amazing how quickly the result was achieved 😃. But wait a minute, what if another robot is connected to the network with which we do not want to share data? To solve this problem, let's find out how communication is ensured in ROS 2.

DDS & ROS_DOMAIN_ID​

In ROS 2, the communication middleware used is DDS, and it utilizes the concept of Domain ID to facilitate communication between logical networks sharing the same physical network. By default, all ROS 2 nodes operate within Domain ID 0, allowing them to discover and exchange messages with each other seamlessly. However, nodes belonging to different Domain IDs cannot communicate.

Let's change ROS_DOMAIN_ID and see if our laptop can still receive message.

ROSbot
export ROS_DOMAIN_ID=1
ros2 topic pub -r 1 /msg std_msgs/msg/String data:\ 'Hello, ROSbot here'

First you can check if /msg topic is still visible on second device. Then also configure ROS_DOMAIN_ID and set the same value on both devices.

Laptop
export ROS_DOMAIN_ID=1
ros2 topic echo /msg
caution
  1. It is safe to use ROS_DOMAIN_ID value between 0 and 101, inclusive (Read more here).
  2. Configure ROS_DOMAIN_ID is specific only for current session. If you open new terminal, you need to setup this variable again.

Connecting ROS 2 via Internet​

As mentioned in the previous section you can run ROS nodes even of dozens of devices as long as they are in the same LAN network. This means that you can connect the ROS between the laptop and the robot when they are connected to the same Wi-Fi router. However, the above example will not work if the robot is connected to the Internet, e.g. via the LTE module.

You can solve the issue by using a VPN service. Standard VPNs, however, are designed for a different purpose than mobile robotics and introduce some disadvantages to your system, such as:

  • Maintenance of a VPN server
  • Dealing with a high latency (all traffic needs to go thought a VPN server)
  • Long reconfiguration time (a.k.a. switching a Wi-Fi hotspot by your robot)
  • Lack of "ROS awareness"

Fortunately there is a VPN service designed with ROS 2 in mind - Husarnet VPN Client. Husarnet is an open source peer-to-peer VPN - that means you can connect your robots directly over the Internet, with no central VPN server for data exchange. All traffic is encrypted using ephemeral keys, and it works with ROS out of the box. Just install Husarnet Client on your laptop and robot, connect them to the same Husarnet network, by using an online dashboard and enjoy low-latency connection between your devices. :)

Using Husarnet is easy:

Step 1 / 5

Create a free account at https://app.husarnet.com

Husarnet Register

At this point you should have your robot and laptop connected to the same Husarnet network with the following Husarnet Hostnames.

  • robot: my-robot
  • laptop: my-laptop

Test connection

Post a message from the terminal using the familiar ros2 topic command. For example, on one computer you can type:

ROSbot
ros2 topic pub -r 1 /msg std_msgs/msg/String data:\ 'Husarnet is awesome! :D'

and use ros2 topic echo /msg to listen for incoming topic.

More about Husarnet VPN​

Husarnet is open source and free to use up to 5 devices. Here you can find a nice article showing how to integrate it with your ROS 2 system:

Nice! At this point you should have your laptop and robot connected via LAN or Internet. Let's run some additional demo nodes to see how the connection works in practice.

Real-time video stream​

In the first demo, the camera on one of the platforms will be launched. Thanks to the network connection, we can access any topic between devices. In this example, we will show you how to display the image from the camera on the second device.

ROSbot

  1. Configure ROS_DOMAIN_ID for LAN network, or use Husarnet VPN.

  2. Launch ROSbot:

docker compose up -d microros <rosbot_service> <camera_service>

Depending on your ROSbot model and specific configuration you should use an appropriate name for your <rosbot_service> and <camera_service>. To check available docker services use command docker compose config --services, or find suitable Docker images prepared by Husarion.

Laptop

  1. Configure ROS_DOMAIN_ID for LAN network, or use Husarnet VPN.

  2. Check if connection works with:

ros2 topic list
  1. Run rqt_image_view to see a real-time video from the robot (you need to select topic name after execute below command).
rqt_image_view

Outsourcing robot computing power​

In this section, we will program a robot and a computer to outsource the robot's computing power. The node that we are going to assign to the robot is responsible for image processing. Basically, the robot publishes the stream from the camera, and object recognition is done on the laptop (using the find_object_2d node). This approach allows to increase computing power, especially when it is necessary to use computer graphics.

Please note that data transfer also takes time. For this reason, in some situations it may not be possible to speed up the algorithm, because the time needed for data transfer will be greater than the time saved in the calculation time. It is also possible to compress data, which will reduce the size of data and thus the time of information transfer. However, compression and decompression needs to be fast and spreadable. An example of such compression for video is the H264 coding standard. Below are instructions from the previous exercise showing step by step how to transfer calculations to another device.

  1. Configure the network.
  1. Run ROSbot.
ROSbot
docker compose up -d microros <rosbot_service> <camera_service>
  1. Launch image_recognition.launch.py file on your Laptop device. This file will run:
  • find_2d_object node known from the previous chapter, which will perform calculations on our computerThanks to this, the most time-consuming calculations should be accelerated,
  • track_obj node calculating the appropriate control.
Laptop
ros2 launch tutorial_pkg image_recognition.launch.py

You should now be able to reproduce the behavior from the previous chapter.

Summary​

That's all, in case of LAN connection, the configuration boils down to setting the appropriate domain using ROS_DOMAIN_ID. If you are connected via the Internet, you can use Husarnet. Note that DDS can be set in several ways. For more information, please visit: Quality of Service and Husarnet DDS configuration. After completing this tutorial, you should be able to configure your devices to communicate with multiple devices. You should also know how to transmit and read information remotely, which allows you to control the robot.

by Rafał Górecki, Husarion

Do you need any support with completing this tutorial or have any difficulties with software or hardware? Feel free to describe your thoughts on our community forum: https://community.husarion.com/ or to contact our support: support@husarion.com