This post shows how to add automated tests to ROS. If you want to get straight to the data, scroll to the bottom of the article for a cheat sheet that shows which testing framework to use depending on your code under test and your build system.

Why Write About Testing?

Between outdated tools, unclear documentation, and no good examples, learning how to write automated tests in ROS is hard.

If you search "ROS unit testing", there is a very nice ROS Wiki Page on why to test, but the how is still unclear to a lot of people. The second search result begins with a disclaimer that some of the documentation is out of date because catkin changed things. Well, catkin was released 6 years ago, in ROS Fuerte, but I guess the documentation still hasn't been updated. In fact, catkin itself is old, and we now have ament_cmake which was written for ROS2.

This post is a starting point for people who want to learn how to automate their ROS tests. It's not a complete guide, but I hope I can help clear up some of the confusion around this topic.

Know Your Testing Frameworks

Part of the reason it's so hard to get started on this topic is that there are lot of testing frameworks out there. Some are designed for C++, some for Python, and some for ROS.

Just like handy ROS tools, it's tough to find out about all of these frameworks on your own. It's not clear which ones are even compatible with your build system, much less which ones are the best to use.

I put a "cheat sheet" version of this data at the end of the article for easy lookup, but if you want details on each system, read on!

ROS Tests

First, some automated tests require a running ROS system. Maybe you want to check that a service call returns correctly, or your node depends on other nodes while it's being tested.

For catkin, rostest is your friend here. rostest uses .test files, which are like roslaunch files, to set up an entire ROS system. This means a roscore, a parameter server, and as many nodes as you want. Then it conducts your tests using another testing framework. So if you use rostest, you ALSO have to use one of the options below. Be sure to add your rostest to CMakeLists.

For ament, there are no good options if you want an already-working solution. You will have to use the new and poorly-documented system_tests to run tests against multiple ROS2 nodes. Or try using launch_testing to test that your launch file works as expected.

C++ Tests

If you want to write your tests in C++, use gtest. Simple!

Python Tests

If you need to write your tests in Python, there are at least 6 decent testing frameworks. Only some are properly integrated with catkin and ament_cmake.

For catkin, you have three options:

1. unittest: Old but still very functional. There is a good guide on how to write a unittest test and add it to your CMakeLists over on the ROS Wiki.

2. nose: Nose is a newer framework that some people prefer to unittest. Note: nose is now in "maintenance mode" because there is a nose2, but nose2 is not integrated into catkin. Write your nose tests as usual, then add the tests to your CMakeLists.

3. pytest: pytest is yet another Python testing framework. Here's a blog post by Alex Rössler on how to add these tests to your project.

For ament/ament_cmake, you can use nose (remember: not in active development) or pytest. But the ROS2 documentation is not very good so you may find that you need to browse the source code to learn how to add the proper commands to your CMakeLists. Here's the ament_cmake source for nose and pytest.

Determine Your Code Under Test

Determine your code under test next.

Are you trying to test a Python library that does matrix and vector operations?

Or are you trying to run an integration test to make sure that a launch file sets up your C++ object detector to publish ROS messages to a /detected_objects topic?

Think carefully about what part of the system is critical to test. You might find that you need two or even three types of test: testing the basic library, testing a single nodes' ROS interface, and testing many nodes talking to each other.

Write the Tests

Now that you know your code under test and your options for testing libraries, you are ready to write your tests and add them to the build system!

Be sure to build and run your tests (here instructions for catkin_make and catkin tools) once you've finished writing them.

I hope this served as a handy reference for anyone looking to add automated tests to their ROS code. If you know of other test frameworks or tutorials I can link to to improve this tutorial, please leave a comment or contact me.

Cheat Sheet

Here's a couple of tables with the information above, for quick lookup.

Testing using catkin

Testing using ament_cmake

ROS has a ton of different terminal commands which can be difficult to remember, especially for beginners. We can use bash aliases to ease some of the pain. In this post I share the bash aliases that I use to speed up my ROS development.

Some commands are too long or complex to type quickly, and others have syntax that doesn\'t make sense. Other commands interact in unexpected ways, such as sourcing multiple workspaces at once.

Here\'s an example. Have you ever tried to run catkin_make while inside a package directory, like catkin_ws/src/mypackage? It doesn\'t work! Even though catkin_make should be able to figure out what workspace you\'re in, it gives up. Then you have to go up a couple of directories, build, and go back down, or open another terminal just for building.

After facing this particular issue a few dozen times, I finally decided to fix it once and for all. So, I spent a bit of time creating aliases and quality of life shortcuts. Then, I put these into a bash script that I source in my .bashrc file. That way I always have access to these helpful aliases and shortcuts.

Below, I\'ll go through the commands one by one and explain what they do and why they\'re necessary. I find that after using these commands for a few days, it\'s really tedious to go back to writing ROS code without them.

If you want the entire file, scroll to the bottom where I\'ve put a GitHub link. Otherwise, read on for details.

My Aliases and Functions

Sourcing

alias sourceros=\'source /opt/ros/melodic/setup.bash\'

sourceing ROS is the classic step that everyone forgets when first learning to use ROS. Many beginners place the source command straight into their .bashrc. This is fine if you only use your computer for ROS development.

I use my Linux machine for multiple different projects and languages, and I don\'t want to source ROS every time I open a new terminal - it can take a bit of time and it adds or modifies a lot of environment variables. This alias makes it so when I do want to work with ROS, I can quickly set up my terminal.

alias sourcethis=\'unset CMAKE_PREFIX_PATH;source ./devel/setup.bash\'

I use this command to quickly source whatever workspace I am currently in. This is very handy when you work with multiple catkin workspaces. Often, if I want to do a quick proof of concept, or if I want to quickly set up a test environment for vision_msgs, I just make another workspace and cleanly switch into it using this command.

unsourceros() {
unset CMAKE_PREFIX_PATH
unset ROS_ROOT
unset ROS_PACKAGE_PATH
unset ROS_MASTER_URI
unset ROS_DISTRO
unset ROS_ETC_DIR
unset ROSLISP_PACKAGE_DIRECTORIES

unset ROSCONSOLE_FORMAT

# clear ros paths from other env vars that we can't totally remove
export PATH=$(echo $PATH | perl -ne 's/\:?[^\:]*ros[^\:]*(?=\:)?//g; print;')
export PYTHONPATH=$(echo $PYTHONPATH | perl -ne 's/\:?[^\:]*ros[^\:]*(?=\:)?//g; print;')
export LD_LIBRARY_PATH=$(echo $LD_LIBRARY_PATH | perl -ne 's/\:?[^\:]*ros[^\:]*(?=\:)?//g; print;')
export PKG_CONFIG_PATH=$(echo $PKG_CONFIG_PATH | perl -ne 's/\:?[^\:]*ros[^\:]*(?=\:)?//g; print;')
export CMAKE_PREFIX_PATH=$(echo $CMAKE_PREFIX_PATH | perl -ne 's/\:?[^\:]*ros[^\:]*(?=\:)?//g; print;')
export MANPATH=$(echo $MANPATH | perl -ne 's/\:?[^\:]*ros[^\:]*(?=\:)?//g; print;')
}

This annoyingly-long function undoes most of the changes that sourcing setup.bash makes.

Building

As I mentioned at the start of the post, catkin_make is quite bad at figuring out how to build unless you are in the root of your catkin workspace. catkin build from catkin tools has a much nicer interface that does work from sub-directories, but that doesn\'t solve the problem of building from any directory.

During ROS development, you probably only have one catkin workspace sourced at a time (unless you\'re an advanced user who has multiple overlay workspaces). So I thought it would be simple to write an alias that could build this workspace immediately, no matter what else you are doing?

alias roshome='cd $(echo $CMAKE_PREFIX_PATH | cut -d ':' -f 1);cd ..'

Automatic return to your last-sourced catkin root directory, which is handy as part of the following commands.

cb() {
  (roshome && exec catkin build --summarize "$@")
}

This executes catkin build in your last-sourced catkin workspace root, and it uses a subshell so you don't have to worry about it changing your working directory.

cm() {
  (roshome && exec catkin_make "$@")
}

Does the same thing as above, but for catkin_make.

cmi() {
  (roshome && exec catkin_make_isolated --install "$@")
}

Same as above, but for catkin_make_isolated, which is preferred over catkin_make. For more details on the differences between these two commands, read this ROS Answers post.

alias cmclean='(roshome && rm -rf build devel install && cm)'
alias cbclean='(roshome && rm -rf build devel install && cm)'
alias cmiclean='(roshome && rm -rf build_isolated devel_isolated install_isolated && cmi)'

Finally, here are some versions that will clean your catkin workspace (without a prompt) and immediately start a build. This is much faster than cding to your catkin workspace, typing catkin_make clean, confirming the clean, and then typing catkin_make.

Other

export ROSCONSOLE_FORMAT=\'[${severity}] ${message}\'

By default, ROS log messages sent to stdout and shown in the terminal start with a timestamp. This timestamp is not in human-readable format though. Instead, it is a Unix Timestamp, which is usually not very helpful for someone just trying to write some code or do a bit of minor debugging. It\'s also quite a long string and so it makes it harder to read the actual log message, often making it wrap to the next line.

Luckily, ROS formats its output using the ROSCONSOLE_FORMAT environment variable, so I customize this to get rid of the giant timestamp. There are other useful variables you can put into this format string. Some of the ones you can use are severity, message, time, thread, logger, file, line, and function.

Using the Script

I put all these aliases into a script I called kinetic.bash along with helpful comments for each command. It\'s available on GitHub.

You could copy the contents of this file directly into your .bashrc, but I recommend downloading the entire file, putting it somewhere where you won\'t accidentally delete it, and then add a source command in your .bashrc. This will keep your .bashrc organized and still give you all the aliases.

Going Further

If you want to apply this sort of personal tooling technique to other parts of your developer experience, you can create a repository on GitHub with your own personal tooling and bash scripts. In this repository, you can store scripts like the one I shared in this blog post, along with any other configuration files for programs like Vim or Tmux. These repositories are usually called dotfiles.

Most of the details about dotfiles are outside of the scope of this post. If you want to learn more, there is a large community of GitHub users who share their dotfiles publicly. Here\'s an unofficial guide to dotfiles on GitHub, and here is my own dotfiles repo.

I\'m always looking for more useful ROS aliases or helper scripts. If you have any that you like, share them in a comment or send me a message, or consider starting on your very own .dotfiles journey.

While working in my robotics lab, a common situation arises: I want to run a script or train a neural network that may take anywhere from a few minutes to a few hours. I don't know how long it will take the program to complete, and I don't want to have to babysit the terminal it's running in. I've found two good ways to alert myself when these scripts complete.

When I first started working through the ROS tutorials, one of the first annoyances I faced was the sheer number of terminals I was opening. Continue reading

In my years working with ROS in a research capacity, I've used dozens of development machines, and I've learned a few guidelines when picking out laptops for working with Linux, ROS, and robots. I've updated the guide for newer 2019 laptop models.

You can use easily speech recognition to emit ROS messages and control your robots with your voice. In this post, we'll learn how to install some popular speech recognition libraries on a ROS machine. This post describes how to set up the Sphinx libraries and custom code developed by human-robot interaction researchers at UT Austin and elsewhere.

Continue reading

In our robotics lab at UT Austin, we often use augmented reality (AR) tags to determine the position and orientation of an object. One good ROS package we use for tracking markers is ar_track_alvar, which can track ArUco-style markers (shown below) and calculate their 6D pose. The package makes detecting AR tags as easy as running a roslaunch file (with some slight configuration tweaks).

Continue reading