Yesterday a friend of mine pointed me to the following thread:
Giving.github is a project that tries to bring people together. There are a lot of programmers looking for fun projects in the world, and so are scientists/charities and other people with problems!
Be sure to check it out, maybe you can help somebody as well.
Mara-Naboisho Lion Project
One of the problems posted to giving.github has something to do with Lion collars. The Mara-Naboisho Lion Project has a couple of collars and some files, but no idea what the data inside the files mean. Their initial description was:
The Mara-Naboisho Lion Project has asked for our help decoding and plotting the positions of lions fitted with GPS collars.
They receive SMSes from the collars:
These are presumably SMS PDUs, although simple online decoders don’t seem to make much sense of them. There’s the ASCII text “IRIDIUM” in them, presumably being the SMSC or sender or something string, as IRIDIUM is a satellite telephone system and presumably the bearer when the collars send up their positions.
What format is it, and how do we decode it?
Not SMS PDU
This is where I started analyzing, and I quickly ruled out that the hex-code is pure PDU (which can easily be translated). It seems to be some dialect/own protocol.
The first major breakthrough was realizing that there is one big header that both files have in common, and the real message starts with 0A. And the two messages after that have a repeating pattern:
This sure adds some structure to the files. I also guessed the first file has actually 6 entries and the second file has only 5 entries.
First field deciphered
The second breakthrough came through Twitter. I posted this problem in a tweet and after a couple of hours I got a reply that there was progress on the stackoverflow question.
This was DStibbe’s reply:
Perhaps the ‘13’ is not part of the headers but the message instead?
The messages would then be:
It does seem like a log , with the message for Aug 06 (guessing) has 6 entries and the second from Aug 07 having 7.
What would a collar record? I’m guessing longitude,latitude and timestamp?
Chances are high that the first eight digits represent the timestamp. They are incremental from bottom to top.
Eg. 13ef348a = 334443658.
334443658 seconds = 10 years, 8 months, 6 days, 23:00:58
This matches the date in the title of the first sms: Collar07854_100806 210058.SMS
It even matches the timestamp minus 2 hrs in the title!
At this point the progress stopped a bit, we couldn’t get the other bytes translated. There seemed to be certain integer values: 0039 d0ed for example. This probably isn’t floating point (which we thought it would be, latitude/longitude/altitude) because floating point numbers have the exponent in the first 11 bits. When a number starts with 00’s it doesn’t have the exponent part filled.
So we asked the project for some more information, for example about the recorded data (where/when) and the manufacturer of the collars.
They gave us this KML file, and this company name: Vectronic-Aerospace.
Lat/lon, I’ve got you!
The biggest part we wanted to decipher was the lan/lon location (that is what the researchers want to have). I came to this breakthrough after reading a PDF file from Vectronic-Aerospace’s website. It has some screenshots from their original software (why aren’t they using that free software?!). In these screenshots I noticed integer coordinates very close to the integers I’ve managed to get from the hex-data! It was called ECEF data, this is X,Y,Z data! Until now my guess was that the SMS files had lat/lon coded in binary, but it has XYZ information.
After a quick google I managed to find some code to translate from ECEF coordinates to lat/lon and it gave me the following location:
When plotted in Google Maps it gave me a location in Germany, at first I thought it was another dead end, but then I looked here. It turns out it was the EXACT location of Vectronic Aerospace, the creators of the collar! Problem solved.
There are still some mysteries, for example… what are the last bytes? Looking at the screenshots in the PDF I suspect it would be main battery voltage, backup voltage and temperature. The project also probably wants to integrate the collar files with the website. And there are of course a lot more giving.github.com problems to solve!