Exporting ROS Bag Files to Matlab


This tool has been deprecated in favor of a new method which directly reads bag data into Matlab. See this post for details.

 

Original post:

Recently, Adam introduced a tool to analyze ROS bag files using the Python pandas package, and plot the data using matplotlib. You can read his post here. While this is a great tool, I have a a couple of requirements not addressed by the tool. First, I usually analyze my data in Matlab, and getting the bag data to a CSV file only gets me half way to my goal. Second, the pandas script does not handle some of the messages I use in my code, specifically, arrays of geometry_msgs/Vector3 in a message. Finally, I want to be able to analyze bags that I do not have the message file definitions for. Since I often modify my message files in the field in response to debugging problems I encounter during an experiment, I sometimes do not have the message files used to create a bag.

Once in Matlab, I can use the wide range of Matlab data analysis tools to analyze my information, or send the data to our partner researchers who are more familiar with Matlab than they are with ROS or Python. To satisfy my needs, I have created a toolchain of Python and Matlab scripts that allow me to import bagged data into Matlab.

All of the following programs can be found at the following GitHub repository, along with more examples of importing CSV files to Matlab.

https://github.com/unl-nimbus-lab/bag2csv

Getting the data into a CSV

One of the first tasks I did when I joined the lab was modify a Python script my advisor, Carrick Detweiler, wrote for extracting data from bag files to CSV files. The bag_reader.py program in this repo represents a further refinement of this script that addresses shortcomings in the first script that made it incapable of analyzing bag files that contained messages with fields of arrays.

This script is capable of converting all of the messages for a specified topic in a bag file to a CSV representation. bag_reader outputs a CSV file where the columns are the fields within the messages published on a specified topic. There is a header line at the beginning with the field names within the specified topic. If bag_reader encounters a field that is an array type, for example, ranges from a Hokuyo laser scanner, it joins the array contents with underscores and puts them into one column in the CSV file.

The two most common tasks I do with bag_reader is to list all the messages in a bag, and extract messages from a certain topic. To list the contents of a bag, simply run the following command:

bag_reader -b test.bag -i

The -b option specifies the bag file I want to work on, and -i tells the script to list all of the topics and the fields within the messages.

The second common task I perform is to convert all messages published on a certain topic to a CSV file. The following code does this task. -t is the flag to specify the topic to read, and -o specifies the output CSV file.

bag_reader -b test.bag -t /temperature -o temperature.csv

The script will also print out help text with the -h option.

bag_reader -b test.bag -h

The script has one major shortcoming. One of the first things the script does is search for the first message of a specified topic in the bag file, and uses that as a template for formatting the remaining messages. If the first message has a different format from the remaining messages in the file, the script will fail. One example of how this could happen is if the message should contain a vector of geometry_msgs/Vector3 types, and the first message has a blank array. The ROS bag format will not place the x, y, and z elements in the blank array, so my script does not learn that this message should contain those elements, and will fail when it encounters messages with values in the array.

Matlab

Once I have the data in CSV files, it becomes trivial to get the data into Matlab. I use the builtin Matlab tools and create scripts that import each message type into Matlab. For example, the temperature.csv file used in the prior example contains messages from the standard ROS sensor_msgs/temperature message. To read all of the data into a Matlab struct, I can use the following script.

function [bag_data] = sensor_msgs_temperature_extract(filename)
fid = fopen(filename);
C = textscan(fid, ...
'%s%n%n%n%n%n', ...
'delimiter',',', ...
'headerLines', 1);
fclose(fid);
bag_data = struct(...
'seq', C{2}, ...
'nsecs', C{3}, ...
'secs', C{4}, ...
'times', C{4} + (C{3} / 1e9), ...
'temperature', C{5}, ...
'variance', C{6});
end

The textscan() command uses the data format to read all of the lines in the file into a cell array called ‘C’. I use ‘C’ to fill out a struct with the information.

Conclusion

I hope other people find this tool useful, and can use it in their research. If you find any bugs or have a good feature to add, let me know, as I plan to continuously improve this tool.

Leave a Comment