Running ROS on multiple machines
You can run this tutorial on:
Introduction
ROS allows you to run nodes on a single robot and on dozens robots as well, as long, as your devices are in the same network.
You will learn here how to configure ROS to work on multiple computers. You will use this configuration to set up system consisting of robot and computer or two computers (gazebo version), which perform task of searching an object.
In this tutorial you will need a computer and a robot with the same equipment as in the previous tutorials.
In case you are working on Gazebo simulator, it is possible to setup system to work on multiple computers. To do so, you need to have two computers with ROS.
Connecting ROS powered robots in LAN
To run ROS on multiple machines you need to connect them to the same LAN network at first.
Remember that only one devices can run ROS Master. For example if you are connecting robot and laptop, run roscore
only on one of those two devices.
While working on multiple machines, you need only one roscore
running. Let's assume we are connecting robot and laptop devices, with the following IP addresses:
- robot:
192.168.1.1
(runningroscore
- ROS Master) - laptop:
192.168.1.2
The steps are as follows:
1st device
On the robot device open the .bashrc
file, then find lines:
export ROS_MASTER_URI=http://master:11311
export ROS_IPV6=on
and replace them with:
export ROS_MASTER_URI=http://192.168.1.1:11311
export ROS_IP=192.168.1.1
#export ROS_MASTER_URI=http://master:11311
#export ROS_IPV6=on
2nd device
On the laptop device open the .bashrc
file, then find lines:
export ROS_MASTER_URI=http://master:11311
export ROS_IPV6=on
and replace them with:
export ROS_MASTER_URI=http://192.168.1.1:11311
export ROS_IP=192.168.1.2
#export ROS_MASTER_URI=http://master:11311
#export ROS_IPV6=on
tip
If you do changes in .bashrc
file always source it with . ~/.bashrc
or reopen terminal
tip
Remember that roscore
must be running on the device indicated as ROS master!!!
Connecting ROS powered robot over the Internet
As mentioned in the previous section you can run ROS nodes even of dozens of devices as long as they are in the same LAN network. That means you can run a single ROS system containing your laptop and robot connected to the same Wi-Fi router, but you can't if robot is connected over LTE.
You can solve the issue by using a VPN service. Standard VPNs, however, are designed for other purposes than mobile robotics, and are introducting some drawbacks to your system, like:
- Maintainance of a VPN server
- Dealing with a high latency (all traffic needs to go throught a VPN server)
- Long reconfiguration time (a.k.a. switching a Wi-Fi hotspot by your robot)
- Lack of "ROS awareness"
Fortunately there is a VPN service designed with ROS in mind - Husarnet VPN Client. Husarnet is an open source peer-to-peer VPN - that means you can connect your robots directly over the Internet, with no central VPN server for data exchange. All traffic is encrypted using ephemeral keys, and it works with ROS out of the box. Just install Husarnet Client on your laptop and robot, connect them to the same Husarnet network, by using an online dashboard and enjoy low-latency connection between your devices :)
Using Husarnet is easy:
- 1. Create Account
- 2. Confirm e-mail
- 3. Create a Network
- 4. Copy a Join Code
- 5. Connect Devices
Step 1 / 5
Create a free account at https://app.husarnet.com
Step 2 / 5
After you click "Register" you will be redirected to your account where your will see:
Please click the confirmation link in the e-mail you should have just received:
Step 3 / 5
After you click the confirmation e-mail you will see your account with no networks:
Click a [Create network] button and name your first network, eg. my_network
:
Step 4 / 5
After you create a new network you will see:
Click [Add element] button and you will see a window with your Join Code:
Step 5 / 5
Finally, install the Husarnet Client on your devices and add them to your Husarnet network, by simply executing in the Linux terminal 3 commands:
## 1. Install Husarnet
curl https://install.husarnet.com/install.sh | sudo bash
## 2. Reload systemd starting Husarnet daemon
sudo systemctl restart husarnet
## 3. Join the VPN network (in place of ${JOINCODE} place what you got in step 4/5)
sudo husarnet join ${JOINCODE} my-robot
To the same on you laptop (sudo husarnet join ${JOINCODE} my-laptop
) and that's all. Now we can reach all the devices like they were connected to the same Wi-Fi hotspot, even if they sit in different continents.
At this point you should have your robot and laptop connected to the same Husarnet network with the following Husarnet Hostnames (that can be used instead of Husarnet IPv6 address)
- robot:
my-robot
(runningroscore
- ROS Master) - laptop:
my-laptop
The steps to configure ROS are as follows:
.bashrc
configuration
On the robot and laptop devices open the .bashrc
file, and make the following configuration:
export ROS_MASTER_URI=http://master:11311
export ROS_IPV6=on
In online dashboard make sure to select ROS master
checkbox in setting of my-robot
device. It will allow you to reach my-robot
device by using both my-robot
and master
hostname (the second one is used in our configuration)
As you can see it's even easier than in previous configuration (only LAN network), becasue you don't need to know IP addresses of devices, but only an easy to rembember hostname master
.
More about Husarnet VPN
Husarnet is open source and free to use up to 5 devices. It works not only with ROS, but also with ROS 2, and here you can find a nice article showing how to integrate it with your ROS 2 system:
OK, at this stage you should have your laptop and robot connected over LAN or over the Internet. Let's launch some demo nodes to see how the connection works in practise.
Demo 1: see a real-time video stream from RGB-D camera
tip
A cheatsheet how to run the following ROS nodes / launch files:
- astra.launch:
roslaunch astra_launch astra.launch
- rqt_graph:
rosrun rqt_graph rqt_graph
- image_view:
rosrun image_view image_view image:=/camera/rgb/image_raw
Running on a Physical Robot & a Laptop
robot part
Setup
.bashrc
with configuration for ROS masterLaunch
roscore
Launch
astra.launch
laptop part
Setup
.bashrc
with configuration for normal deviceCheck if connection works with:
rostopic list
run
rqt_graph
to verify if you can see all nodes from two devices.Run
image_view
to see a real-time video from the robot
Running on two laptops (one with Gazebo simulation)
1st laptop (gazebo)
Setup
.bashrc
with configuration for ROS masterlaunch gazebo simulation:
roslaunch tutorial_pkg tutorial_4.launch use_gazebo:=true teach:=true recognize:=false
- Launch
astra.launch
andimage_view
nodes
2nd laptop part
Setup
.bashrc
with configuration for normal deviceCheck if connection works with:
rostopic list
run
rqt_graph
to verify if you can see all nodes from two devices.Run
image_view
to see a real-time video from the robot
Demo 2: Outsourcing Robot Computing Power to Laptop
In this section we will program robot and computer to outsource robot computing power. The node we are going to outsource from a robot is responsible for image processing. So basically a robot publishes camera stream and object recognition is on laptop (using find_object_2d
node).
Such a use case could be a common task - especially if we need to use computer's graphics - probably much more powerful than that in a robot.
About demo
We will follow the pattern form a previous tutorial but this time we do it using two machines (ROSbot and laptop). Anything could be an object to recognize, but remember, that the more edges and contrast colours it has, the easier it will be recognized. A piece of paper with something drawn on it would be enough for this tutorial.
Start
roscore
, open terminal on your laptop and typeroscore
On ROSbot you should run
astra.launch
image transport
andbridge to CORE2
.For connection to CORE2 we will use package
rosbot_ekf
To launch rosserial communication and Kalman filter for mbed firmware run:
roslaunch rosbot_ekf all.launch
For PRO version add parameter:
roslaunch rosbot_ekf all.launch rosbot_pro:=true
On a laptop we will launch also
image transport
andfind_object_2d
node.
Image transport node provides compressed image delivery from one device to another, on sending machine you have to run this node for compression and on receiving machine you have to decompress images. Before we used image view
without any compression which is enough just for see if it's working, but if only you want to implement some real time image processing it's necessary to compress an image.
Running a demo on ROSbot (physical) & laptop
Create launch file under tutorial_pkg/launch
and name it as tutorial_5_rosbot.launch
:
<launch>
<arg name="rosbot_pro" default="false" />
<arg name="teach" default="true"/>
<arg name="recognize" default="false"/>
<include file="$(find rosbot_ekf)/launch/all.launch">
<arg name="rosbot_pro" value="$(arg rosbot_pro)" />
</include>
<include file="$(find astra_launch)/launch/astra.launch"/>
<node pkg="tf" type="static_transform_publisher" name="camera_publisher" args="-0.03 0 0.18 0 0 0 base_link camera_link 100" />
<node pkg="image_transport" type="republish" name="rgb_compress" args=" raw in:=/camera/rgb/image_raw compressed out:=/rgb_republish"/>
</launch>
And launch
file for PC computer, name it `tutorial_5_pc.launch:
<launch>
<arg name="teach" default="true"/>
<arg name="recognize" default="false"/>
<node name="teleop_twist_keyboard" pkg="teleop_twist_keyboard" type="teleop_twist_keyboard.py" output="screen"/>
<node pkg="image_transport" type="republish" name="rgb_decompress" args=" compressed in:=/rgb_republish raw out:=/rgb_raw" >
<param name="compressed/mode" value="color"/>
</node>
<node pkg="find_object_2d" type="find_object_2d" name="find_object_2d">
<remap from="image" to="/rgb_raw"/>
<param name="gui" value="$(arg teach)"/>
<param if="$(arg recognize)" name="objects_path" value="$(find tutorial_pkg)/image_rec/"/>
</node>
</launch>
Use rqt_graph
to see how the system is working now.
That's all you don't have to worry about nothing more than properly configuration of .bashrc
rest stays the same. If you have more than 2 devices the pattern is the same. ROS takes care of all data exchange. That is really simple, isn't it?
Running a demo on ROSbot (Gazebo) & laptop
Gazebo node will be running on one computer (let's name it "gazebo computer"). Second machine - name it laptop - will be responsible for image processing.
At "gazebo computer" use the following launch file and save it as tutorial_5_gazebo_1.launch
:
<launch>
<arg name="teach" default="false"/>
<arg name="recognize" default="true"/>
<arg if="$(arg teach)" name="chosen_world" value="rosbot_world_teaching"/>
<arg if="$(arg recognize)" name="chosen_world" value="rosbot_world_recognition"/>
<include file="$(find rosbot_gazebo)/launch/$(arg chosen_world).launch"/>
<include file="$(find rosbot_description)/launch/rosbot_gazebo.launch"/>
<node pkg="image_transport" type="republish" name="rgb_compress" args=" raw in:=/camera/rgb/image_raw compressed out:=/rgb_republish"/>
</launch>
On laptop use this launch file and save it as tutorial_5_gazebo_2.launch
:
<launch>
<arg name="teach" default="true"/>
<arg name="recognize" default="false"/>
<node name="teleop_twist_keyboard" pkg="teleop_twist_keyboard" type="teleop_twist_keyboard.py" output="screen"/>
<node pkg="image_transport" type="republish" name="rgb_decompress" args=" compressed in:=/rgb_republish raw out:=/rgb_raw" >
<param name="compressed/mode" value="color"/>
</node>
<node pkg="find_object_2d" type="find_object_2d" name="find_object_2d">
<remap from="image" to="/rgb_raw"/>
<param name="gui" value="$(arg teach)"/>
<param if="$(arg recognize)" name="objects_path" value="$(find tutorial_pkg)/image_rec/"/>
</node>
</launch>
Execute both launch files, open rqt_graph
to see how the system is working now.
Summary
After completing this tutorial you should be able to configure your CORE2 devices to work together and exchange data with each other. You should also know how to program robots for performing tasks in cooperation.
by Łukasz Mitka, Adam Krawczyk and Dominik Nowak, Husarion
Do you need any support with completing this tutorial or have any difficulties with software or hardware? Feel free to describe your thoughts on our community forum: https://community.husarion.com/ or to contact with our support: support@husarion.com