GPS
Porting a Ragel GPS NMEA Parser to AVR ATMega128
04/06/2010 19:43
Synopsis: This
article shows the result of porting the GPS NMEA
parser introduced in
“A Ragel Parser for NMEA Sentences” to
an AVR ATMega128
with a Falcom FSA03 GPS
for source data. Furthermore, this port to
the AVR has refined the NMEA parser in several
ways including making more it robust and testing
it under non-GPS locked conditions. The code
reads GPS NMEA data on one ATMega128 TTL serial
port and pumps a JSON structure containing GPS
location data out the other other port, an XBee
data radio.
One of my goals with Ragel was to build powerful parsers that could live on small microcontrollers. Early on, when I started Ragel Recipies, I stated I wanted to see how far down the technology stack I could push Ragel including building parsers for microcontrollers. This is the successful culmination of that proof point. Ragel is indeed well suited for building parsers for embedded systems.
Ragel AVR Target
The target board is the RAD-GPS board I’ve designed which includes an ET-AVR ATMega128 stamp, an FSA03 GPS and an XBee data radio. The board has an onboard power supply that can take 6-30V input and a JTAG port for debugging which came in very handy to debug the port. Here’s an image of the target board. My plan is to use this board for the autonomous vehicle we’ll enter into the Sparkfun autonomous vehicle competition in mid-April.
The target C compiler is CodeVision AVR.
The previous Ragel NMEA parser I presented targeted GCC, so I needed to start with a basic skeleton for the AVR. With the CodeVision AVR wizard, I generated a basic application that included receive interrupt handlers for communication on both ATMega128 serial ports.
Refactoring CodeVision AVR Wizard Code
I then refactored the entire wizard output. One thing I’ve always disliked about CodeVision AVR wizard output is that it generates one large monolithic code file. That makes it cluttered to work with and in my case, I would have had to include all that code in the Ragel .rl definition. In any case, I was going to need to break it up.
So, I ripped out all the async comms and put those routines into their own .c file and added .h externs. Expecting to include a gps.c file into the project, I added a gps.h skeleton file and included it in the main .c file which I called autoveh.c (because I’m gunning to use this for my autonomous vehicle.)
At this point, it was a lot like I had torn up my kitchen...from a code perspective I had pipes sticking out, wires splayed everywhere.
Porting the Ragel .rl file to AVR
I copied the gps.rl file I developed in the previous Ragel Recipe: “A Ragel Parser for NMEA GPS Sentences” to my AVR project folder and began modifying it for the AVR target. The main thing I added was include files to the top of the gps.rl to target the AVR:
I then generated the parser from the OS X host where I had Ragel installed and shared source directory with my Windows VM where CodeVision AVR compiler lives. I added the gps.c file to the AVR project and started trial compiles with CodeVision AVR.
After working out various syntax errors, my next task was to integrate the GPS NMEA stream coming from the AVR serial port into the parser itself. The CodeVision AVR wizard generates an interrupt handler that fills a circular buffer with serial data. I needed to pull that data and feed it to the parser. There are multiple ways to do this, but the most straightforward way that I went for was to read an entire sentence out of the buffer an then pass that sentence into the parser. Ragel can parse fragments and maintain state which is nice, but in my case, that didn’t really buy me much. If I wanted to save memory or make it somewhat faster, I could have spent the time to feed the circular buffer directly to Ragel, but because that structure is modified at interrupt time, it would have taken more work to make that safe.
So, the main control loop of the firmware is simply reading GPS NMEA sentences from the ATMega128 serial port, making a single sentence and passing it into the Ragel parser. The Ragel parser tries to make sense out of the data and populates a data structure with location data. That structure, as explained in the previous article is cumulative. As various NMEA sentences are encountered and parsed, various portions of the structure get filled in and fields overwritten.
That way, the main control loop of the firmware can refer to that structure at any time the most current location data. NMEA sentences which contain redundant information, say latitude and longitude, get parsed and the structure updated, so lat/lon may get updated very frequently and other types of attributes, say altitude or speed, less frequently.
The main loop is constantly pumping out a JSON structure over the XBee serial interface with some of the critical data in it - mainly using this for debug right now, but ultimately, the JSON can be directly stuffed into a CouchDB or used directly by some other AJAX type application if needed.
When I’m testing, I have an XBee Regulated breakout board from Sparkfun connected to my OS X machine so I can monitor the output of the RAD-GPS board. I get a continuous stream of JSON output like this:
Most of these attributes are self-explanatory and map directly to GPS concepts, but there are a few extra attributes I added for diagnostic and management purposes. First, ‘vid’ is vehicle ID. My thinking is that if there are multiple devices to be tracked and all are transmitting on the same PAN ID of XBee, then it would be useful to sort them out. Haven’t worked out all the multiple XBee network things, but just making a data format that anticipates not being the only device out there. Secondly, “nmeaerrors” and “sentences” are counters. NMEA errors are sentences that for, whatever reason, the Ragel parser couldn’t parse. The “sentences” attribute is the number of sentences we’ve parsed...it helps indicate whether NMEA data is flowing at all.
This JSON structure is then streaming out the RAD-GPS XBee port to my OS X box and I can see what’s going on with the parser and GPS performance.
The Ragel .rl definition file for the NMEA parser on an AVR microcontroller then boiled down to this listing. Notice, if you compare this .rl with the .rl in the previous article, the Ragel machines that are defined are much more robust for handling non-locked GPS conditions where there are many fields that have no data in them “,,,,,” sort of output from the GPS. It also includes the NMEA error counts and will dump the syntax problems it encounters along the way....usually an indication my Ragel definition didn’t account for some sentence structure I wasn’t aware of a priori.
The main autoveh.c firmware loop which initializes the AVR micro and reads GPS data and passes it to the “parse_nmea()” function looks like this:
Lessons Learned
The main lessons I learned in the Ragle parser port to the AVR are:
Ragel can definitely be used to generate substantially complex parsers which are small enough to easily fit in a modern flash-based microcontroller like the AVR ATMega128. This opens up huge possibilities for writing network protocols in Ragel that can be shared between the embedded device and the servers and systems that embedded devices often must communicate with.
The micro must be fast enough to service serial data interrupts from the GPS and process and parse the sentences. The Ragel generated parser has also proven itself to be fast enough and have AVR cycles to do real tasks. From my experiments so far, the ATMega128 running at 16MHz is plenty strong enough to run Ragel parsers, specifically, a real-world, complex parser such as the NMEA parser. The parser I threw at the AVR is not a toy.
I’m encouraged to continue using this as a tracking platform and platform for autonomous vehicle control.
Landon Cox
www.ESawdust.com
Also see these other articles in the Ragel Recipes parser series:
“A Skeleton Command Line Intepreter in Ragel for Ruby”
“A Ragel Parser for NMEA GPS Setnences”
“What is Ragel? Why Ragel?”
One of my goals with Ragel was to build powerful parsers that could live on small microcontrollers. Early on, when I started Ragel Recipies, I stated I wanted to see how far down the technology stack I could push Ragel including building parsers for microcontrollers. This is the successful culmination of that proof point. Ragel is indeed well suited for building parsers for embedded systems.
Ragel AVR Target
The target board is the RAD-GPS board I’ve designed which includes an ET-AVR ATMega128 stamp, an FSA03 GPS and an XBee data radio. The board has an onboard power supply that can take 6-30V input and a JTAG port for debugging which came in very handy to debug the port. Here’s an image of the target board. My plan is to use this board for the autonomous vehicle we’ll enter into the Sparkfun autonomous vehicle competition in mid-April.
The target C compiler is CodeVision AVR.
The previous Ragel NMEA parser I presented targeted GCC, so I needed to start with a basic skeleton for the AVR. With the CodeVision AVR wizard, I generated a basic application that included receive interrupt handlers for communication on both ATMega128 serial ports.
Refactoring CodeVision AVR Wizard Code
I then refactored the entire wizard output. One thing I’ve always disliked about CodeVision AVR wizard output is that it generates one large monolithic code file. That makes it cluttered to work with and in my case, I would have had to include all that code in the Ragel .rl definition. In any case, I was going to need to break it up.
So, I ripped out all the async comms and put those routines into their own .c file and added .h externs. Expecting to include a gps.c file into the project, I added a gps.h skeleton file and included it in the main .c file which I called autoveh.c (because I’m gunning to use this for my autonomous vehicle.)
At this point, it was a lot like I had torn up my kitchen...from a code perspective I had pipes sticking out, wires splayed everywhere.
Porting the Ragel .rl file to AVR
I copied the gps.rl file I developed in the previous Ragel Recipe: “A Ragel Parser for NMEA GPS Sentences” to my AVR project folder and began modifying it for the AVR target. The main thing I added was include files to the top of the gps.rl to target the AVR:
I then generated the parser from the OS X host where I had Ragel installed and shared source directory with my Windows VM where CodeVision AVR compiler lives. I added the gps.c file to the AVR project and started trial compiles with CodeVision AVR.
After working out various syntax errors, my next task was to integrate the GPS NMEA stream coming from the AVR serial port into the parser itself. The CodeVision AVR wizard generates an interrupt handler that fills a circular buffer with serial data. I needed to pull that data and feed it to the parser. There are multiple ways to do this, but the most straightforward way that I went for was to read an entire sentence out of the buffer an then pass that sentence into the parser. Ragel can parse fragments and maintain state which is nice, but in my case, that didn’t really buy me much. If I wanted to save memory or make it somewhat faster, I could have spent the time to feed the circular buffer directly to Ragel, but because that structure is modified at interrupt time, it would have taken more work to make that safe.
So, the main control loop of the firmware is simply reading GPS NMEA sentences from the ATMega128 serial port, making a single sentence and passing it into the Ragel parser. The Ragel parser tries to make sense out of the data and populates a data structure with location data. That structure, as explained in the previous article is cumulative. As various NMEA sentences are encountered and parsed, various portions of the structure get filled in and fields overwritten.
That way, the main control loop of the firmware can refer to that structure at any time the most current location data. NMEA sentences which contain redundant information, say latitude and longitude, get parsed and the structure updated, so lat/lon may get updated very frequently and other types of attributes, say altitude or speed, less frequently.
The main loop is constantly pumping out a JSON structure over the XBee serial interface with some of the critical data in it - mainly using this for debug right now, but ultimately, the JSON can be directly stuffed into a CouchDB or used directly by some other AJAX type application if needed.
When I’m testing, I have an XBee Regulated breakout board from Sparkfun connected to my OS X machine so I can monitor the output of the RAD-GPS board. I get a continuous stream of JSON output like this:
Most of these attributes are self-explanatory and map directly to GPS concepts, but there are a few extra attributes I added for diagnostic and management purposes. First, ‘vid’ is vehicle ID. My thinking is that if there are multiple devices to be tracked and all are transmitting on the same PAN ID of XBee, then it would be useful to sort them out. Haven’t worked out all the multiple XBee network things, but just making a data format that anticipates not being the only device out there. Secondly, “nmeaerrors” and “sentences” are counters. NMEA errors are sentences that for, whatever reason, the Ragel parser couldn’t parse. The “sentences” attribute is the number of sentences we’ve parsed...it helps indicate whether NMEA data is flowing at all.
This JSON structure is then streaming out the RAD-GPS XBee port to my OS X box and I can see what’s going on with the parser and GPS performance.
The Ragel .rl definition file for the NMEA parser on an AVR microcontroller then boiled down to this listing. Notice, if you compare this .rl with the .rl in the previous article, the Ragel machines that are defined are much more robust for handling non-locked GPS conditions where there are many fields that have no data in them “,,,,,” sort of output from the GPS. It also includes the NMEA error counts and will dump the syntax problems it encounters along the way....usually an indication my Ragel definition didn’t account for some sentence structure I wasn’t aware of a priori.
The main autoveh.c firmware loop which initializes the AVR micro and reads GPS data and passes it to the “parse_nmea()” function looks like this:
Lessons Learned
The main lessons I learned in the Ragle parser port to the AVR are:
Ragel can definitely be used to generate substantially complex parsers which are small enough to easily fit in a modern flash-based microcontroller like the AVR ATMega128. This opens up huge possibilities for writing network protocols in Ragel that can be shared between the embedded device and the servers and systems that embedded devices often must communicate with.
The micro must be fast enough to service serial data interrupts from the GPS and process and parse the sentences. The Ragel generated parser has also proven itself to be fast enough and have AVR cycles to do real tasks. From my experiments so far, the ATMega128 running at 16MHz is plenty strong enough to run Ragel parsers, specifically, a real-world, complex parser such as the NMEA parser. The parser I threw at the AVR is not a toy.
I’m encouraged to continue using this as a tracking platform and platform for autonomous vehicle control.
Landon Cox
www.ESawdust.com
Also see these other articles in the Ragel Recipes parser series:
“A Skeleton Command Line Intepreter in Ragel for Ruby”
“A Ragel Parser for NMEA GPS Setnences”
“What is Ragel? Why Ragel?”
A Ragel Parser For NMEA GPS Sentences
03/19/2010 03:34
Synopsis: This is
a Ragel definition
file, gps.rl, which implements a parser for NMEA
GPS sentences $GPGGA, $GPRMC, $GPGSA, GPGLL,
GPGSV, GPVTG. The target parser is generated in
C and the main() is a set of test functions that
send the parser through some paces based on
actual GPS strings I gathered from a custom GPS
PCB with a Falcom FSA03
GPS that I built.
As promised in my introduction to Ragel Recipes, some recipes would be full-blown Ragel parsers and some would just be Ragel snippets. This is the former category - a complete Ragel NMEA parser.
The target parser was built using GCC on an OS X machine, but is completely portable to any Linux platform. I haven’t spent any time on Windows with Ragel to add anything wise about using it with Windows. Ragel 6.6 (Dec 2009) was used to compile the .rl file.
You can take this parser, replace main(), read NMEA sentences from your systems serial port where you have a GPS connected, and feed each line to the parser. The way I implemented this code is there is a location data structure that accumulates the current location data as it is reading the sentences, so your app, at any one time only has to examine the data structure currentPosition to get everything that has been learned by the GPS - the structure contains things like latitude, longitude, speed, heading and a lot more.
Instructions for building this are simple (assuming you have installed the latest Ragel and GCC):
To run the code, from a shell, just execute ‘gps’:
More validation code would need to be added to main() to make it a completely automated test for more rigorous use in a production setting.
Landon Cox
www.ESawdust.com
Full list of Ragel Recipes
As promised in my introduction to Ragel Recipes, some recipes would be full-blown Ragel parsers and some would just be Ragel snippets. This is the former category - a complete Ragel NMEA parser.
The target parser was built using GCC on an OS X machine, but is completely portable to any Linux platform. I haven’t spent any time on Windows with Ragel to add anything wise about using it with Windows. Ragel 6.6 (Dec 2009) was used to compile the .rl file.
You can take this parser, replace main(), read NMEA sentences from your systems serial port where you have a GPS connected, and feed each line to the parser. The way I implemented this code is there is a location data structure that accumulates the current location data as it is reading the sentences, so your app, at any one time only has to examine the data structure currentPosition to get everything that has been learned by the GPS - the structure contains things like latitude, longitude, speed, heading and a lot more.
Instructions for building this are simple (assuming you have installed the latest Ragel and GCC):
To run the code, from a shell, just execute ‘gps’:
More validation code would need to be added to main() to make it a completely automated test for more rigorous use in a production setting.
Landon Cox
www.ESawdust.com
Full list of Ragel Recipes
asdfasdf
