Yesterday a friend of mine pointed me to the following thread:
https://github.com/giving/giving.github.com/issues/11

giving.github.com

Giving.github is a project that tries to bring people together. There are a lot of programmers looking for fun projects in the world, and so are scientists/charities and other people with problems!

Be sure to check it out, maybe you can help somebody as well.

Mara-Naboisho Lion Project

One of the problems posted to giving.github has something to do with Lion collars. The Mara-Naboisho Lion Project has a couple of collars and some files, but no idea what the data inside the files mean. Their initial description was:

The Mara-Naboisho Lion Project has asked for our help decoding and plotting the positions of lions fitted with GPS collars.

They receive SMSes from the collars:

File: Collar07854_100806210058.SMS

074952494449554d0000000000000000000000000000000000000000
000000040f3330303033343031323239373839300000000000000000
0000000000000000f10a0806100028008c13ef348a0039d0fe000de8
71004cc92b5ca92d6213ef26640039d108000de86b004cc92d5ca92d
5d13ef18620039d101000de865004cc92c5ca92d5813ef0a930039d0
fc000de864004cc9311c682d5413eefc170039d045000de7d4004cc9
5b7c692c5013eeee280039d0ff000de85f004cc92a7c692d6fffffff
ffffffffffffffffffffffffffffffffffff
    
File: Collar07854_100807060011.SMS

074952494449554d0000000000000000000000000000000000000000
000000040f3330303033343031323239373839300000000000000000
0000000000000000f10a0807020038008c13efb2eb0039d0ed000de8
53004cc92e3cea2d8813efa5060039d0fb000de860004cc9291c6a2d
8413ef96fd0039d0fc000de85e004cc92d5c6a2d7f13ef88e00039d0
f6000de85a004cc92b5c6a2d7b13ef7ad80039d0fa000de85a004cc9
327c6a2d77ffffffffffffffffffffffffffffffffffffffffffffff
ffffffffffffffffffffffffffffffffffff

These are presumably SMS PDUs, although simple online decoders don’t seem to make much sense of them. There’s the ASCII text “IRIDIUM” in them, presumably being the SMSC or sender or something string, as IRIDIUM is a satellite telephone system and presumably the bearer when the collars send up their positions.

What format is it, and how do we decode it?

Not SMS PDU

This is where I started analyzing, and I quickly ruled out that the hex-code is pure PDU (which can easily be translated). It seems to be some dialect/own protocol.

The first major breakthrough was realizing that there is one big header that both files have in common, and the real message starts with 0A. And the two messages after that have a repeating pattern:

Common header?

074952494449554d000000000000000
0000000000000000000000000000000
040f333030303334303132323937383
9300000000000000000000000000000
0000f10a

Sub header?
SMS 1: 0806100028008c13
SMS 2: 0807020038008c13


Message #1:
ef348a0039d0fe000de871004cc92b5ca92d6213
ef26640039d108000de86b004cc92d5ca92d5d13
ef18620039d101000de865004cc92c5ca92d5813
ef0a930039d0fc000de864004cc9311c682d5413
eefc170039d045000de7d4004cc95b7c692c5013
eeee280039d0ff000de85f004cc92a7c692d6fff
ffffffffffffffffffffffffffffffffffffffff

Message #2:
efb2eb0039d0ed000de853004cc92e3cea2d8813
efa5060039d0fb000de860004cc9291c6a2d8413
ef96fd0039d0fc000de85e004cc92d5c6a2d7f13
ef88e00039d0f6000de85a004cc92b5c6a2d7b13
ef7ad80039d0fa000de85a004cc9327c6a2d77ff
ffffffffffffffffffffffffffffffffffffffff
ffffffffffffffffffffffffffffffffffffffff

This sure adds some structure to the files. I also guessed the first file has actually 6 entries and the second file has only 5 entries.

First field deciphered

The second breakthrough came through Twitter. I posted this problem in a tweet and after a couple of hours I got a reply that there was progress on the stackoverflow question.

This was DStibbe’s reply:

-——————————————————————————–
Perhaps the ‘13’ is not part of the headers but the message instead?
The messages would then be:

Message #1
13ef 348a 0039 d0fe 000d e871 004c c92b 5ca9 2d62
13ef 2664 0039 d108 000d e86b 004c c92d 5ca9 2d5d
13ef 1862 0039 d101 000d e865 004c c92c 5ca9 2d58
13ef 0a93 0039 d0fc 000d e864 004c c931 1c68 2d54
13ee fc17 0039 d045 000d e7d4 004c c95b 7c69 2c50
13ee ee28 0039 d0ff 000d e85f 004c c92a 7c69 2d6f
ffff ffff ffff ffff ffff ffff ffff ffff ffff ffff
ff

Message #2
13ef b2eb 0039 d0ed 000d e853 004c c92e 3cea 2d88
13ef a506 0039 d0fb 000d e860 004c c929 1c6a 2d84
13ef 96fd 0039 d0fc 000d e85e 004c c92d 5c6a 2d7f
13ef 88e0 0039 d0f6 000d e85a 004c c92b 5c6a 2d7b
13ef 7ad8 0039 d0fa 000d e85a 004c c932 7c6a 2d77
ffff ffff ffff ffff ffff ffff ffff ffff ffff ffff
ffff ffff ffff ffff ffff ffff ffff ffff ffff ffff
ff

It does seem like a log , with the message for Aug 06 (guessing) has 6 entries and the second from Aug 07 having 7.
What would a collar record? I’m guessing longitude,latitude and timestamp?

Chances are high that the first eight digits represent the timestamp. They are incremental from bottom to top.

Eg. 13ef348a = 334443658.
334443658 seconds = 10 years, 8 months, 6 days, 23:00:58

This matches the date in the title of the first sms: Collar07854_100806 210058.SMS
It even matches the timestamp minus 2 hrs in the title!
-——————————————————————————–

More information

At this point the progress stopped a bit, we couldn’t get the other bytes translated. There seemed to be certain integer values: 0039 d0ed for example. This probably isn’t floating point (which we thought it would be, latitude/longitude/altitude) because floating point numbers have the exponent in the first 11 bits. When a number starts with 00’s it doesn’t have the exponent part filled.

So we asked the project for some more information, for example about the recorded data (where/when) and the manufacturer of the collars.
They gave us this KML file, and this company name: Vectronic-Aerospace.

Lat/lon, I’ve got you!

The biggest part we wanted to decipher was the lan/lon location (that is what the researchers want to have). I came to this breakthrough after reading a PDF file from Vectronic-Aerospace’s website. It has some screenshots from their original software (why aren’t they using that free software?!). In these screenshots I noticed integer coordinates very close to the integers I’ve managed to get from the hex-data! It was called ECEF data, this is X,Y,Z data! Until now my guess was that the SMS files had lat/lon coded in binary, but it has XYZ information.

After a quick google I managed to find some code to translate from ECEF coordinates to lat/lon and it gave me the following location:
Lat: 52.43067054630929
Lon: 13.525755535406619
Height: 88.99500464554876

When plotted in Google Maps it gave me a location in Germany, at first I thought it was another dead end, but then I looked here. It turns out it was the EXACT location of Vectronic Aerospace, the creators of the collar! Problem solved.

What’s next?

There are still some mysteries, for example… what are the last bytes? Looking at the screenshots in the PDF I suspect it would be main battery voltage, backup voltage and temperature. The project also probably wants to integrate the collar files with the website. And there are of course a lot more giving.github.com problems to solve!