This post shows how to add automated tests to ROS. If you want to get straight to the data, scroll to the bottom of the article for a cheat sheet that shows which testing framework to use depending on your code under test and your build system.

Why Write About Testing?

Between outdated tools, unclear documentation, and no good examples, learning how to write automated tests in ROS is hard.

If you search "ROS unit testing", there is a very nice ROS Wiki Page on why to test, but the how is still unclear to a lot of people. The second search result begins with a disclaimer that some of the documentation is out of date because catkin changed things. Well, catkin was released 6 years ago, in ROS Fuerte, but I guess the documentation still hasn't been updated. In fact, catkin itself is old, and we now have ament_cmake which was written for ROS2.

This post is a starting point for people who want to learn how to automate their ROS tests. It's not a complete guide, but I hope I can help clear up some of the confusion around this topic.

Know Your Testing Frameworks

Part of the reason it's so hard to get started on this topic is that there are lot of testing frameworks out there. Some are designed for C++, some for Python, and some for ROS.

Just like handy ROS tools, it's tough to find out about all of these frameworks on your own. It's not clear which ones are even compatible with your build system, much less which ones are the best to use.

I put a "cheat sheet" version of this data at the end of the article for easy lookup, but if you want details on each system, read on!

ROS Tests

First, some automated tests require a running ROS system. Maybe you want to check that a service call returns correctly, or your node depends on other nodes while it's being tested.

For catkin, rostest is your friend here. rostest uses .test files, which are like roslaunch files, to set up an entire ROS system. This means a roscore, a parameter server, and as many nodes as you want. Then it conducts your tests using another testing framework. So if you use rostest, you ALSO have to use one of the options below. Be sure to add your rostest to CMakeLists.

For ament, there are no good options if you want an already-working solution. You will have to use the new and poorly-documented system_tests to run tests against multiple ROS2 nodes. Or try using launch_testing to test that your launch file works as expected.

C++ Tests

If you want to write your tests in C++, use gtest. Simple!

Python Tests

If you need to write your tests in Python, there are at least 6 decent testing frameworks. Only some are properly integrated with catkin and ament_cmake.

For catkin, you have three options:

  1. unittest: Old but still very functional. There is a good guide on how to write a unittest test and add it to your CMakeLists over on the ROS Wiki.

  2. nose: Nose is a newer framework that some people prefer to unittest. Note: nose is now in "maintenance mode" because there is a nose2, but nose2 is not integrated into catkin. Write your nose tests as usual, then add the tests to your CMakeLists.

  3. pytest: pytest is yet another Python testing framework. Here's a blog post by Alex Rössler on how to add these tests to your project.

For ament/ament_cmake, you can use nose (remember: not in active development) or pytest. But the ROS2 documentation is not very good so you may find that you need to browse the source code to learn how to add the proper commands to your CMakeLists. Here's the ament_cmake source for nose and pytest.

Determine Your Code Under Test

Determine your code under test next.

Are you trying to test a Python library that does matrix and vector operations?

Or are you trying to run an integration test to make sure that a launch file sets up your C++ object detector to publish ROS messages to a /detected_objects topic?

Think carefully about what part of the system is critical to test. You might find that you need two or even three types of test: testing the basic library, testing a single nodes' ROS interface, and testing many nodes talking to each other.

Write the Tests

Now that you know your code under test and your options for testing libraries, you are ready to write your tests and add them to the build system!

Be sure to build and run your tests (here instructions for catkin_make and catkin tools) once you've finished writing them.

I hope this served as a handy reference for anyone looking to add automated tests to their ROS code. If you know of other test frameworks or tutorials I can link to to improve this tutorial, please leave a comment or contact me.

Cheat Sheet

Here's a couple of tables with the information above, for quick lookup.

Testing using catkin

Code Under Test Testing Options
Python class or function, with no ROS unittest, nose, pytest
C++ class or function, with no ROS gtest
Python class or function with ROS1 rostest file + one of: gtest, unittest, or nose
C++ class or function with ROS1 rostest file + one of: gtest, unittest, or nose
roslaunch file (with ROS1 nodes written in C++, Python, or both) rostest file + one of: gtest, unittest, or nose

Testing using ament_cmake

Code Under Test Testing Options
Python class or function pytest, nose
C++ class or function gtest
Multiple ROS2 nodes system_tests
ROS2 Launch File launch_testing

I hope you enjoyed this article. If you’d like to receive emails when I post new content, please sign up for my mailing list below.