Beyond Docker: our journey to software deployment with snaps
Like many other robotics companies, we use Docker to package and deploy ROS software in our robots. While Docker served us well, its complexities and constraints began impacting our ability to scale and secure our software deployments efficiently. We needed a more streamlined solution.
This blog presents the journey we took exploring snaps as a deployment infrastructure. We wrote this blog to help others facing the same challenges with Docker, and to show how you can take the first steps towards an improved deployment infrastructure and a more security-compliant approach.
The Challenge of Deploying Robotic Software
Our robots are used globally, which means that software deployment and updates represent a crucial component of our operations. Initial user setup, releasing critical updates, building bootable images, and managing a range of customizations are some of the challenges we all face in the robotics sector – and how we solve them can determine success in our industry.
At Husarion, our deployment strategy depends on Docker, which offers us the flexibility and scalability we need to manage our robotics applications effectively. Docker enables us to encapsulate our applications in containers, ensuring they could run reliably across different computing environments. This approach also facilitates the reuse of certain Docker images across various robots, streamlining the development process and fostering efficiency.
However, as our deployment became more specialized and safety critical, the limitations of Docker began to surface. We experienced friction with Docker's complexity, particularly in orchestrating and managing updates. The framework we created for handling Docker compositions and remotely upgrading Docker images is predominantly custom-developed. Maintaining distinct Docker images for various high-level robotics functionalities introduce a layer of complexity that is manageable but far from optimal.
To further complicate the deployment process, Docker’s lack of specialized interfaces for accessing host resources force developers to grant manual, fine-grained privileges to containers. This often leads to the risky practice of running containers with the --privileged flag, which undermines security by removing the container's sandboxing.
This setup, while robust, necessitated a reevaluation of our deployment strategy to ensure our solutions remained at the cutting edge of technology while bringing the security that is needed in the current regulatory landscape
Our First ROS 2 Snap
Initially, we chose a simple ROS application that is used to teleoperate the ROSbot XL so that we could learn about the snap creation process and test the results using a standalone node.
We closely followed the tutorials provided by the Ubuntu Robotics team on ROS deployment with snaps. This is the two part guide we used: Part 1: Distributing ROS apps with snaps and Part 2: Distributing ROS apps with snaps.
By following the tutorials, and with a bit of back and forth, we were able to create our first snap. You can find it here https://github.com/husarion/rosbot-xl-teleop-snap.
It’s worth mentioning that we incorporated essential steps from Prevent connectivity issues with LXD and Docker guide to ensure a smooth initial setup, preventing potential connectivity issues with Snapcraft and Docker on the same host system that we used for the snap development. A simple, temporary way to avoid issues would have been to stop the docker services while we were developing the snaps with Snapcraft and LXD, like this:
sudo systemctl stop docker.socket
sudo systemctl stop docker.service
This step is not necessary for running the snaps on our robot so we could test them with no additional setup steps.
Once done, we updated our Docker compose setup (see the default version here) removing the teleop ROS node from inside as it now resided in a snap outside of the Docker ecosystem. We were ready to test it!
We started our docker with the gazebo simulation with the full ROSbot XL robot and instead of entering inside the existing docker and running the teleop executable, we started our teleop snap directly and moved the robot around
- using Docker
- using Snap
docker exec -it rviz bash
ros2 run teleop_twist_keyboard teleop_twist_keyboard
rosbot-xl-teleop.key
ROSbot XL running in Gazebo and RViz controlled over rosbot-xl-teleop.key snap
The robot moved as expected. This encourages the team to keep exploring snaps.
Growing in Complexity
At this point we were confident in our usage of Snapcraft and the Snapcraft ROS plugins and decided to tackle the mapping docker image. This container has many packages installed from the official ROS PPAs, along with custom launch files and parameters from Husarion. Snaps have full support for installing deb packages along with their dependencies, so we took the list from our navigation Dockerfile and moved them in the stage section of our snapcraft.yaml.
parts:
rosbot-xl-nav:
plugin: nil
stage-packages: [ros-humble-nav2-bringup]
As most of the dependencies were shared, we decided to add navigation and localization in the same snap. Next we had to focus on the management of parameters.
In our Docker Compose approach we mount yaml files from the outside:
volumes:
- ./config/nav2_params.yaml:/nav2_params.yaml
For this snap we decided to put them directly in the final artifact. The configurations examples in Part 2: Distributing ROS apps with snaps were very helpful for showing how we could manage parameters.
At this point we knew that the user would no longer be able to modify those parameters because snaps are immutable. Since multiple solutions exist for managing snaps configurations, we decided to improve this aspect after we finished our migration and we confirmed that all the snaps worked on the real ROSbot.
Is it possible to attach config files during snap startup (similar to bind-mount volumes in Docker)?
Here are some resources covering that topic:
Snaps allow some additional features that can be used to spawn system services and react to: changes of configuration, start and stop of those services. We followed this explanation and reached the final version of the new navigation snap.
The Hardware Abstraction Layer Snap: Drivers and Controller
After doing the same process for the rest of our Dockerfiles, we had multiple snaps operating on top of the hardware-abstraction-layer ROS node, the last component housed within Docker. To test the snapping process of our HAL we started running it on a real robot instead of using our simulation.
We were happy to see that all the snaps were working on the real robot with no changes, meaning that our hardware abstraction layer is properly separating the real hardware from the Gazebo simulated robot.
One thing to notice is the requirement to set sim_time to True or False depending on where you are running the snaps. All the snap launcher scripts support this variable, for example see here.
The snapcraft.yaml we wrote for our HAL was in the end pretty standard. We needed to enable only the usual plugs: [network, network-bind] since our hardware devices all communicate with USB and network-based channels.
Launching the ROS 2 driver itself is also much simpler and easier to remember than a Docker version. For example, this is how you run the ROSbot XL ROS 2 driver with Docker vs Snap:
- using Docker
- using Snap
docker run \
--network=host \
--ipc=host \
--device=/dev/ttyUSBDB \
--device=/dev/bus/usb \
-e USER \
-d husarion/rosbot-xl:humble-0.8.12-20240115 \
ros2 launch rosbot_xl_bringup bringup.launch.py
rosbot-xl
Both Snap and Docker use containerization approaches for software deployment, but Snap is almost indistinguishable from running just a native app on the host OS users are used to. Many users are not familiar with the complexity provided by Docker, so the barrier of entry is much lower in Snaps.
Our Ouster LIDAR drivers were put in a separate snap as not all our robots use that LIDAR. In fact as described in Developer guide - Part 4: Building ROS snaps with content sharing, we learned that it’s better to separate entities in different snaps if those entities do not necessarily require each other.
| Real world | RViz interface |
|---|---|
 |  |
Real ROSbot XL and RViz visualisation together with a telop interface running in snap
The Overview of the Snap Journey
Let us now recap the main differences and good points we found in snaps during our journey.
Startup Time
Initially we did not expect the startup time to be similar between Docker and Snaps but we were pleased to find out that there are no measurable differences in the overhead of starting up ROS nodes delivered as a snap compared to inside a docker.
Over-the-Air Updates
Our approach with versioning and releasing updates to our Docker images relies on an appropriate tagging strategy where, for example, husarion/rosbot-xl:humble is our latest stable image and husarion/rosbot-xl:humble-0.8.2-20230712 is a specific version of it.
After that we can use a watchtower Docker container added to our compose.yaml file that updates the Docker image if a newer version is available in Docker Hub.
With snaps, instead we simply tag our new snap and promote it to stable as explained in Using the Snap Store to distribute ROS applications . At that point the snaps are automatically updated in the robots by the snapd daemon. This allows for OTA updates without hacks or using 3rd party software.
User Experience Like a Native App
After using snaps for a while, we are convinced that one very strong advantage of snaps is the user experience when installing and starting them. In fact, with some script and launcher configuration inside the snap itself, what the user sees is an application to be started.
With Docker, you look at your container as a separate computer running next to your PC and need to care about the docker network configuration and the startup options to be added in the container setup.
With snaps it's much simpler and easier to comprehend. We can hide details related to the proper Snap config inside the Snap itself without sharing too many capabilities from the host OS to the container.
Comparing Docker and Snap for ROS Stack Deployment
One of our key strategies involved adopting a content-sharing approach for the snap process. Given that all Husarion applications rely on the same version of ROS basic libraries, sharing them not only saves space but also eliminates a redundancy. Although achievable in Docker, this approach proved to be more challenging to implement.
The snap ecosystem from Canonical provides a nice The ROS 2 Humble extension crafted specifically for ROS packages. Thanks to the information in our well-structured package.xml, the snap creation process was streamlined, taking only a few hours. The plugin installs the ROS dependencies, prepares a workspace and appends all the needed pre-launcher scripts so that when running the snap no manual sourcing is necessary.
Looking ahead, our focus in the coming weeks will revolve around extending the configuration capabilities of the snaps. This effort aims to create a versatile solution applicable across all our robotic platforms.