Using rosbag_pandas to Analyze rosbag Files with Python 1


ROS offers great tools to record execution traces using the rosbag command line tool. However, after the data is recorded, quickly extracting the data to perform analysis and produce graphics may be an issue to those unfamiliar with the process.  ROS provides a Python and C++ API to read and write data from the bags (http://wiki.ros.org/rosbag/Code%20API), but the data returned by the examples in the API are not in a format that is easy to quickly analyze.

Often when working in the Nimbus lab, we want to quickly view a graph of data from a bag.  We also run more complex analysis on files that require timestamped information from multiple topics.  This analysis is done in Python, Matlab, and a myriad of other languages and tools.  This makes getting the data from the bag file extremely important.   Pandas (http://pandas.pydata.org/) is  a wonderful Python library that provides data structures and tools for data,  processing, preparation, and analysis. Also since pandas has methods  to plot and export its data structures, getting the bag information into pandas will make plotting and exporting trivial.  Rosbag_pandas (https://github.com/aktaylor08/RosbagPandas) is  a python package that allows users to import data recorded from a rosbag into a pandas Dataframe that is indexed by the time the message was recorded.

Installation

To use rosbag_pandas you must first install numpy, pandas, and of course ROS.  After the dependencies are installed you can install rosbag_pandas on your system using your favorite method(e.g. pip install rosbag_pandas, easy_install, or running the setup.py script).  After installing the package, three tools will be available: rosbag_pandas, bag_to_csv.py, and bag_graph.py.

 

bag2csv.py

 

Bag to csv is a python script that can be used to extract data from a bag file and place it in a .csv file.   The script has a number of parameters that can be used to change what is included in the file.    The include and exclude arguments can be used to only store data from certain topics.  By default all topics are included and no topics are excluded.  The program will select the topics by first identifying all of the include topics and then removing the excluded topics from the set.  These arguments can be regular expressions or a list of topic name strings.  The script can also fill missing values using the –fill flag.  This will forward and backward fill data points so that there is no missing values in the any row in the csv file.   Finally, by default the script ignores the header fields in a message. This behavior can be turned off by passing the  –include-header flag to the script.

bag_graph.py

 

bag_graph.py is a tool that allows the quick graphing of fields on a ros topic.  It has two required arguments “-b –bag” and “-s –series.”  The bag is the bag file to use and the series argument is the data fields in the bag to graph.   As a quick example if we want a graph of the height of a UAV we can issue the command: `bag_graph.py -b bag.bag -s /a/subject_pose/translation/z` and get the following result.

graph1

Additionally, the script can handle graphing more than one topic and field at a time.  The image below was creating by running the script with the arguments:  `bag_graph.py -b bag.bag -s /a/subject_pose/translation/z /a/subject_pose/translation.x`

double
The graphs of multiple series can also be placed on a single axis by including the `-c` flag:  `bag_graph.py -b bag.bag -s /a/subject_pose/translation/z  /a/subject_pose/translation.x -c`

combined

 

Getting a Dataframe

The final usage of the package using it to get a pandas Dataframe object populated with data from the  bag file.  This is done by importing rosbag_pandas and calling the method bag_to_dataframe.  This method takes as arguments the file and topics to include and exclude in the returned Dataframe. In the frame, there is a column for each field in the messages on the topics parsed from the bag. Each time a message is received in the bag file,  a row is added to the dataframe that contains values for that topic, but NaN values for the rest of the columns.  This opens the data in a bag file up to all of the tools that are available in pandas and other python libraries. For example rosbag_pandas lets us print the min, and max values on each of the topic’s fields with 4 lines of code:

import rosbag_pandas

df = rosbag_pandas.bag_to_dataframe(‘bag.bag’)

for c in df.columns:

print c, df[c].min(),  df[c].max()

This barely scratches the surface of what can be done with a Dataframe full of data from ROS. Hopefully it gives you an idea of what can be done and how to install and use this simple package.

 


Leave a Comment

One thought on “Using rosbag_pandas to Analyze rosbag Files with Python

  • Najeeb Najeeb

    This is a great tool, amazing work Adam, a real time saver and highly appreciated.

    You may face problems when attempting to use this with some old versions, I had a problem using it under Ubuntu Hydro, let me share my experience. It was failing wish some strange errors.

    check version of pandas:

    python
    print(pandas.version.version)

    if stuck at 0.7.0 then things may not work out so well, better to upgrade

    sudo pip install --upgrade pandas

    if this fails with somethign like (Command python setup.py egg_info failed with error code 1) then download ez_setup.py from (https://bitbucket.org/pypa/setuptools/raw/bootstrap/ez_setup.py) then run it.

    python ez_setup.py

    Now pip can properly upgrade your pandas, so now try

    sudo pip install --upgrade pandas

    Check your version of pandas again

    python
    print(pandas.version.version)

    I now have 0.15.2 and it works fine.