Skip to main content

Running ROS on multiple machines

Chapter description​

ROS allows you to run nodes on a single robot and on dozens robots as well, as long, as your devices are in the same network. You will learn here how to configure ROS to work on multiple computers. You will use this configuration to set up system consisting of robot and computer or two computers for Gazebo simulation version. In this tutorial you will need a computer and a robot with the same equipment as in the previous tutorials. In case you are working on Gazebo simulator, it is possible to setup system to work on multiple computers. To do so, you need to have two computers with ROS.

You can run this tutorial on:

Repository containing the final effect after doing all the ROS Tutorials you can find here

Connecting ROS in LAN network​

To run ROS on multiple machines you need to connect them to the same LAN network at first. Only one devices can run ROS Master. For example if you are connecting robot and laptop, run roscore only on one of those two devices.

Network configuration​

Let's assume we are connecting robot and laptop devices, with the following IP addresses:

  • robot: 192.168.1.1 (running roscore - ROS Master)
  • laptop: 192.168.1.2

The steps are as follows:

1st device​

On the robot device open the .bashrc file, and then find lines:

export ROS_MASTER_URI=http://192.168.1.1:11311
export ROS_IP=192.168.1.1

If there are ROS_MASTER_URI and ROS_IPV6 variables in the file, please comment them out and leave only the two above lines:

#export ROS_MASTER_URI=http://master:11311
#export ROS_IPV6=on

export ROS_MASTER_URI=http://192.168.1.1:11311
export ROS_IP=192.168.1.1

2nd device​

On the laptop device open the .bashrc file, then add folowing lines:

export ROS_MASTER_URI=http://192.168.1.1:11311
export ROS_IP=192.168.1.2

If there are ROS_MASTER_URI and ROS_IPV6 variables in the file, please comment them out and leave only the two above lines:

#export ROS_MASTER_URI=http://master:11311
#export ROS_IPV6=on

export ROS_MASTER_URI=http://192.168.1.1:11311
export ROS_IP=192.168.1.2

Task 1

Run roscore on the appropriate machine (for above configuration robot is set as ROS Master) and check on your other machine if the devices are sending information to each other. Use the rosnode command to see if any nodes are visible.

info

If you make changes in .bashrc file always source it with . ~/.bashrc to see changes or open new terminal.

info

Remember that roscore must be running on the device indicated as ROS master.

info

If there is an error when starting roscore on RObot, it is most likely caused by running ROS Master in the docker. To do this, you should stop running containers with the command:

docker kill $(docker ps -q)

Connecting ROS via Internet​

As mentioned in the previous section you can run ROS nodes even of dozens of devices as long as they are in the same LAN network. This means that you can connect the ROS between the laptop and the robot when they are connected to the same Wi-Fi router. However, the above example will not work if the robot is connected to the Internet, e.g. via the LTE module.

You can solve the issue by using a VPN service. Standard VPNs, however, are designed for a different purpose than mobile robotics and introduce some disadvantages to your system, such as:

  • Maintainance of a VPN server
  • Dealing with a high latency (all traffic needs to go throught a VPN server)
  • Long reconfiguration time (a.k.a. switching a Wi-Fi hotspot by your robot)
  • Lack of "ROS awareness"

Fortunately there is a VPN service designed with ROS in mind - Husarnet VPN Client. Husarnet is an open source peer-to-peer VPN - that means you can connect your robots directly over the Internet, with no central VPN server for data exchange. All traffic is encrypted using ephemeral keys, and it works with ROS out of the box. Just install Husarnet Client on your laptop and robot, connect them to the same Husarnet network, by using an online dashboard and enjoy low-latency connection between your devices. :)

Using Husarnet is easy:

Step 1 / 5

Create a free account at https://app.husarnet.com

Husarnet Register

At this point you should have your robot and laptop connected to the same Husarnet network with the following Husarnet Hostnames (that can be used instead of Husarnet IPv6 address)

  • robot: my-robot (running roscore - ROS Master)
  • laptop: my-laptop

The steps to configure ROS are as follows:

.bashrc configuration​

On both the robot and laptop devices open the .bashrc file, and make the following configuration:

export ROS_MASTER_URI=http://master:11311
export ROS_IPV6=on

In online Husarnet dashboard make sure to select ROS master checkbox in setting of my-robot device. It will allow you to reach my-robot device by using both my-robot and master hostname (the second one is used in our configuration)

ROS master option in Husarnet online dashboard

As you can see it's even easier than in previous configuration (LAN network), becasue you don't need to know IP addresses of devices, but only an easy to rembember hostname master.

Task 2

Again run roscore on the appropriate machine that was selected as a ROS master in Husarnet dashboard and check on your other machine if the devices are sending information to each other. Use the rosnode command to see if any nodes are visible.

tip

You can even quickly post a message from the terminal using the familiar rostopic command. For example, on one computer you can type:

rostopic pub -r 1 /msg std_msgs/String "data: 'Husarnet is awesome! :D'"

and use rostopic echo to listen for incoming topic.

More about Husarnet VPN​

Husarnet is open source and free to use up to 5 devices. It works not only with ROS, but also with ROS 2, and here you can find a nice article showing how to integrate it with your ROS 2 system:

OK, at this point you should have your laptop and robot connected via LAN or Internet. Let's run some additional demo nodes to see how the connection works in practice.

Real-time video stream​

In the first demo, the camera on one of the platforms will be launched. Thanks to the network connection, we can access any topic between devices. In this example, we will show you how to display the image from the camera on the second device.

robot part​

  1. Configure .bashrc for ROS master for LAN network, or use Husarnet VPN.

  2. Launch astra.launch

husarion@husarion:~$
roslaunch astra_camera astra.launch

laptop part​

  1. Configure .bashrc for normal device for LAN network, or use Husarnet VPN.

  2. Check if connection works with:

user@laptop:~$
rostopic list
  1. Run rqt_image_view to see a real-time video from the robot (you need to select topic name after execute below command).
user@laptop:~$
rqt_image_view

Outsourcing robot computing power​

In this section, we will program a robot and a computer to outsource the robot's computing power. The node that we are going to assign to the robot is responsible for image processing. Basically, the robot publishes the stream from the camera, and object recognition is done on the laptop (using the find_object_2d node). This approach allows to increase computing power, especially when it is necessary to use computer graphics.

In this section we will also use the image transport node. The image transport node provides the ability to compress and decompress the image. This approach will allow you to speed up data transfer between devices through data reduction. Below are step-by-step instructions for the previous exercise performing calculations on two connected devices.

  1. Configure the network.

  2. Create new image_recognition_rosbot.launch file on ROSbot device. This file will run camera with image_transport which compress the image.

~/ros_ws/src/tutorial_pkg/launch/image_recognition_rosbot.launch
<launch>

    <arg name="use_gazebo" default="false" />
    <arg name="teach" default="false"/>

    <!-- Gazebo -->
    <group if="$(arg use_gazebo)">
        <param name="use_sim_time" value="true" />
        <include file="$(find rosbot_bringup)/launch/rosbot_tutorial.launch">
            <arg if="$(arg teach)" name="world" value="image_teaching"/>
            <arg unless="$(arg teach)" name="world" value="image_recognition"/>
        </include>
    </group>

    <!-- ROSbot -->
    <group unless="$(arg use_gazebo)">
        <include file="$(find astra_camera)/launch/astra.launch"/>
    </group>

    <!-- Compress images -->
    <node pkg="image_transport" type="republish" name="rgb_compress" args=" raw in:=/camera/color/image_raw compressed out:=/rgb_compressed"/>

</launch>
  1. Create secound launch image_recognition_laptop.launch file on your computer. This file will run:
  • image_transport on your host, which will be used to decompress the image,
  • find_2d_object node known from the previous chapter, which will perform calculations on our computerThanks to this, the most time-consuming calculations should be accelerated.
~/ros_ws/src/tutorial_pkg/launch/image_recognition_laptop.launch
<launch>

    <arg name="use_gazebo" default="false" />
    <arg name="teach" default="false"/>

    <!-- Decompress images -->
    <node pkg="image_transport" type="republish" name="rgb_decompress" args=" compressed in:=/rgb_compressed raw out:=/rgb_decompressed" >
        <param name="compressed/mode" value="color"/>
    </node>

    <!-- Teleoperation - keyboard control -->
    <node name="teleop_twist_keyboard" pkg="teleop_twist_keyboard" type="teleop_twist_keyboard.py" output="screen"/>

    <!-- Object detection -->
    <node pkg="find_object_2d" type="find_object_2d" name="find_object_2d">
        <remap from="image" to="/rgb_decompressed"/>
        <param name="gui" value="$(arg teach)"/>
        <param unless="$(arg teach)" name="objects_path" value="$(find tutorial_pkg)/img_data/"/>
    </node>

</launch>
  1. Start launch file on your ROSbot.
husarion@husarion:~$
docker compose up -d rosbot ros-master
roslaunch tutorial_pkg image_recognition_rosbot.launch
  1. Start launch file on your laptop.
husarion@husarion:~$
roslaunch tutorial_pkg image_recognition_laptop.launch

Execute both launch files, open rqt_graph to see how the system is working now.

Task 3

Try to run img_follow code and check experimentally whether the smoothness of the robot's operation has been improved.

Hint
  1. Create workspace.
  2. Copy source file and edit CMakeList.txt. Remember to change `#define OBJECT_TO_FOLLOW` according to your object id.
  3. Build with catkin_make command.
  4. Run with rosrun tutorial_pkg follow_img teach:=false or add this line to image_recognition_laptop.launch.
    <node pkg="tutorial_pkg" type="img_follow" name="img_follow" output="screen">
      <arg name="teach" value="false"/> 
    </node>

Question 1 To connect multiple devices communicating with each other in the ROS application ROS Master must be:
a) running on every device
b) running only on so-called 'master' device
c) ROS master will automatically boot on the correct device provided you have a properly configured .bashrc file
d) it is not possible to connect multiple devices together in ROS

Answer
b)

Summary​

That's all, you don't have to worry about nothing more than properly configuration of .bashrc rest stays the same. Everytime when you have 2 or more devices the pattern is the same. ROS takes care of all data exchange. That is really simple, isn't it? After completing this tutorial, you should be able to configure your devices to communicate with multiple device. You should also know how to send and read information remotely, allowing you to control a robot. You've also seen the image_transport node for image compression.

by Łukasz Mitka, Rafał Górecki, Adam Krawczyk & Dominik Nowak, Husarion

Do you need any support with completing this tutorial or have any difficulties with software or hardware? Feel free to describe your thoughts on our community forum: https://community.husarion.com/ or to contact with our support: support@husarion.com