Dual Source GPS/DCF77 NTP-Server with AIS and ADS-B receivers
Inhaltsverzeichnis
1 Scope
This project is a continuation of my stream of experiments with the NTP protocol and GPS-disciplined clock models. I need to reference the past projects before we go into details:
- NTP-Server with Raspberry Pi and Trimble Resolution SMT
- NTP-Server with Raspberry Pi and Sure Electronics GPS Eval board
These past projects raised the question, how precise a DCF-77 clock would be. And this project here also needs to be seen in the context of an April 1st joke of Rainer DF2NU, who proposed that DCF-77 can be switched off now because the GPS system is there and it is much more precise. So I started to collect ideas to investigate how less precise a DCF-77 disciplined clock model would be compared to the GPS Pulse Per Second. The reason for the reduced precision is because the 77,5 kHz signals are reflected randomly in the ionosphere and the exact reflection point cannot be predicted, resulting in a varying signal runtime.
In addition to the clock question I was thinking of setting up a pair of receivers for the AIS frames broadcasted around Regensburg by the shipd on the danube and I wanted to receove ADS-B frames. Even though slowly navigating ships allow for some frame processing delay, and in case the delay is a couple of seconds, one would still consider such an application a real time application. A jet plane however travels quite distance in a couple of seconds, so a positioning application would need to add some timestamping logic in order to allow for a realistic position information, particularly if received by multiple receivers.
For this reason it makes perfect sense to add a precise clock to a position frame receiver setup.
2 List of components
- The chassis: http://shop.meconet.de/Gehaeuse/Outdoor/Chassis-Outdoor-CA-OTU-RB-433-RB-435-RB-493-RB-800-3-LAN-4-N-Type-large::111356.html - 72 €
- A Raspberry Pi 3: https://www.amazon.de/Raspberry-Pi-3-Model-B/dp/B01CEFWQFA - 35 €
- SanDisk 32GB MicroSD Card: https://www.amazon.de/SanDisk-Android-microSDHC-Speicherkarte-SD-Adapter/dp/B013UDL5RU/ref=sr_1_1 - 10 €
- GUDE Expert Mouse Clock 0107: https://www.amazon.de/VALUE-Funkuhr-Expert-mouseCLOCK-Kabel/dp/B0049K3ZI6 116 €
- FlightAware Pro Stick USB ADS-B Receiver: https://www.amazon.de/FlightAware-Pro-Stick-ADS-B-Empf%C3%A4nger/dp/B01D1ZAP3C/ref=sr_1_cc_2 - 17 €
- FlightAware ADS-B 1090MHz Band-pass SMA Filter: https://www.amazon.de/ADS-B-1090MHz-Band-pass-SMA-Filter/dp/B010GBQXK8/ref=sr_1_1 20 €
- FlightAware 1090MHz ADS-B Antenne - 66cm / 26in: https://www.amazon.de/1090MHz-ADS-B-Antenne-66cm-26in/dp/B00WZL6WPO/ref=sr_1_cc_1 - 45 €
- Pigtail N - SMA - 3 €
- USB A Extension Male/Female - 3 €
- Passive POE Injector: https://www.amazon.de/PoE-Passiv-Injector-PowerOverEthernet-Spannungsversorgung-WRAP-Boards/dp/B00FXOH4C0/ref=sr_1_24 - 3 €
- DC/DC converter: 3 €
- Ferrite cores: 2 €
- diverse standoff headers, screws: 5 €
- Sparkfun USB2serial 3,3V: https://www.sparkfun.com/products/9873 - 10 €
- Adafruit Ultimate GPS Breakout (66 channel w/10 Hz): http://www.watterott.com/de/Adafruit-Ultimate-GPS-Breakout-66-channel - 45 €
- BMP180 temperature and air pressure sensor: http://www.ebay.de/itm/BMP180-Temperatur-Druck-Modul-/291751659851?hash=item43edc1094b:g:TWoAAOSwc1FXZJQU - 5 €
- MMCX 10cm: 10 €
- FL Pigtail 25cm: 10 €
- AISspotter USB stick: http://busse-yachtshop.de/s/AIS-Empfaenger-im-USB-Stick-Gehaeuse - 99 €
- 2x N/N cable 2m with H155: 10 €
- Marineband radio antenna: 35 €
- diverse PCBs and pin headers: 2 €
~ 560 Euro
The prices are partly a bit over the market, roundet up, so that there are no surprises.
3 Configuration
3.1 Image and Packages
The Raspberry installation is based on a Minibian 2016-03-12 image: https://minibianpi.wordpress.com/
I found that sometimes it is hard to install it into a running state with improper readers / adapters. But if you have a good adapter, it should work properly with any SD card.
The minibian standard image does not have a "pi" user like Raspian, but the login succeeds with "root" and "raspberry". Of yourse, this information is just for those who want to replicate my project. Be sure that I have chosen another password, and please do the same :-)
If you choose to attach a serial console to your Raspberry, you must make sure that the agetty stops attaching to your /dev/ttyAMA0 device. You can do that with the following commands:
systemctl stop serial-getty@ttyAMA0.service systemctl disable serial-getty@ttyAMA0.service systemctl mask serial-getty@ttyAMA0.service
Then reboot and check if it has succeeded using lsof. I think it does not come with the Minibian distro, so it is one of the tools you need to install:
lsof | grep ttyAMA0
3.2 Hardware Connectivity and Kernel
I configured the kernel like this:
cat /boot/config.txt gpu_mem=16 dtparam=i2c1=on dtparam=i2c_arm=on dtoverlay=pps-gpio,gpiopin=18 init_uart_baud=9600 enable_uart=0
Accordingly the kernel modules configuration file /etc/modules looks like this:
snd-bcm2835 i2c-bcm2708 i2c-dev
Since we have a FTDI to connect the GPS and another FTDI that is used in the GUDE DFC-77 clock, we need to keep them apart and ensure that they will always and forever in the future use the correct device name:
Check who is who:
root@multirx:/etc/udev/rules.d# lsusb | grep 403 Bus 001 Device 006: ID 0403:6015 Future Technology Devices International, Ltd Bridge(I2C/SPI/UART/FIFO) Bus 001 Device 005: ID 0403:6001 Future Technology Devices International, Ltd FT232 USB-Serial (UART) IC Bus 001 Device 004: ID 0403:e88a Future Technology Devices International, Ltd Expert mouseCLOCK USB II
Then create this file:
vi /etc/udev/rules.d/70-persistent-dev.rules KERNEL=="ttyUSB*", ATTRS{idProduct}=="e88a", SYMLINK+="refclock-0" KERNEL=="ttyUSB*", ATTRS{idProduct}=="6001", SYMLINK+="gps0" KERNEL=="ttyUSB*", ATTRS{idProduct}=="6015", SYMLINK+="ais0"
These statements will give me the following devices:
/dev/refclock-0 /dev/gps0 /dev/ais0
The names are important because they need to correspond to the expectations of the accoring NTP modules.
We need to ensure that certain kernel modules will NOT be loaded automatically:
root@multirx:~# cat /etc/modprobe.d/rtl-sdr-blacklist.conf # This system has librtlsdr0 installed in order to # use digital video broadcast receivers as generic # software defined radios. blacklist dvb_usb_rtl28xxu blacklist e4000 blacklist rtl2832
3.3 Packages
I found that it makes sense to install the following tools:
apt-get update apt-get install i2c-tools # I2C-Toolkit for commandline apt-get install python-smbus # Python library for I2C apt-get install libi2c-dev # library for C apt-get install git # need this later... apt-get install lsof # determine who is using the serial port apt-get install apache2 # for all the later web stuff apt-get install lsusb # see who is connected via USB apt-get install gnuplot # charting tool
3.4 Fundamental system config
- Configure the timezone using the command dpkg-reconfigure tzdata
3.5 NTP
3.5.1 Fundamental NTPD configuration
Here is my configuration:
root@multirx:~# cat /etc/ntp.conf | grep -v "#" | grep -v "^$" driftfile /var/lib/ntp/ntp.drift statistics loopstats peerstats clockstats filegen loopstats file loopstats type day enable filegen peerstats file peerstats type day enable filegen clockstats file clockstats type day enable filegen sysstats file sysstats type day link enable tos minsane 3 tos orphan 10 tos mindist 0.4 server 0.debian.pool.ntp.org iburst server 1.debian.pool.ntp.org iburst server 2.debian.pool.ntp.org iburst server 3.debian.pool.ntp.org iburst server 127.127.8.0 mode 19 prefer fudge 127.127.8.0 flag2 1 stratum 0 refid GUDE fudge 127.127.20.0 time2 0.435 flag1 0 refid GPS stratum 1 flag3 1 server 127.127.22.0 minpoll 4 prefer fudge 127.127.22.0 refid PPS flag2 0 flag3 1 logfile /var/log/ntp logconfig =all enable calibrate restrict -4 default kod notrap nomodify nopeer noquery restrict -6 default kod notrap nomodify nopeer noquery restrict 127.0.0.1 restrict ::1 restrict 172.16.0.0 mask 255.240.0.0 restrict 192.168.0.0 mask 255.255.0.0 root@multirx:~#
3.5.2 Check that PPS works
The /dev/pps0 should be created upon boot if the Kernel-PPS module is loaded.
root@multirx:~# ppstest /dev/pps0 trying PPS source "/dev/pps0" found PPS source "/dev/pps0" ok, found 1 source(s), now start fetching data... source 0 - assert 1462395271.608254834, sequence: 4022 - clear 0.000000000, sequence: 0 source 0 - assert 1462395272.608254999, sequence: 4023 - clear 0.000000000, sequence: 0 source 0 - assert 1462395273.608254163, sequence: 4024 - clear 0.000000000, sequence: 0 source 0 - assert 1462395274.608255327, sequence: 4025 - clear 0.000000000, sequence: 0 source 0 - assert 1462395275.608373487, sequence: 4026 - clear 0.000000000, sequence: 0 ^C root@multirx:~#
3.5.3 Ensure that you have a NTPd what can handle PPS and DFC-77
Since I decided against using /dev/ttyAMA0 for the NMEA interface for NTP, I was rather free in terms of the serial device. Yet, I connected the PPS pin to GPIO 18.
The package, once again has to be modified and rebuilt:
- Get the build dependencies and the source code of the package:
cd ~ apt-get -y build-dep ntp apt-get -y source ntp
- The source code was stored in my home directory. In the moment and for me it is the version ntp-4.2.6.p5+dfsg. Note that it could be different for you.
cd ntp-4.2.6.p5+dfsg/debian/
- Edit the rules file and add the missing support for PPS/ATOM. Search for configure and append a line at the end of the statement. Don't forget the \ at the end of the previous line.
./configure CFLAGS='$(CFLAGS)' CPPFLAGS='$(CPPFLAGS)' LDFLAGS='$(LDFLAGS)' \ --prefix=/usr \ --enable-all-clocks --enable-parse-clocks --enable-SHM \ --disable-debugging --sysconfdir=/var/lib/ntp \ --with-sntp=no \ --with-lineeditlibs=edit \ --without-ntpsnmpd \ --disable-local-libopts \ --enable-ntp-signd \ --disable-dependency-tracking \ --with-openssl-libdir=/usr/lib/$(DEB_HOST_MULTIARCH) \ --enable-ATOM --enable-JUPITER \ --enable-ONCORE \ --enable-LOCAL-CLOCK \ --enable-RAWDCF \ --enable-HOPF6021 \ --enable-TRIMTSIP \ --enable-NMEA
- Note that I am enabling some more GPS technologies because I have some Navman Jupiters hhere, and I would like to use them in the future...
- Now add the changelog file so that it reflects the change:
ntp (1:4.2.6.p5+dfsg-7+deb8u1) jessie-security; urgency=medium
- into:
ntp (1:4.2.6.p5+dfsg-8+deb8u1) jessie-security; urgency=medium
- Now 'cd' into the package directory and build the package.
cd .. sudo dpkg-buildpackage -b
- It will take a while now. In my case three deb files were created:
-rw-r--r-- 1 root root 351566 Dec 2 00:37 ntp_4.2.6.p5+dfsg-8+deb8u2_armhf.deb -rw-r--r-- 1 root root 69132 Dec 2 00:37 ntpdate_4.2.6.p5+dfsg-8+deb8u2_armhf.deb -rw-r--r-- 1 root root 1031588 Dec 2 00:36 ntp-doc_4.2.6.p5+dfsg-8+deb8u2_all.deb
- Now install them:
dpkg -i ntp_4.2.6.p5+dfsg-8+deb8u2_armhf.deb dpkg -i ntpdate_4.2.6.p5+dfsg-8+deb8u2_armhf.deb dpkg -i ntp-doc_4.2.6.p5+dfsg-8+deb8u2_all.deb
- Now restart the ntp server with the command service ntp restart and after ~ 1 minute it should give you this output using ntpq -p:
root@multirx:~# ntpq -p remote refid st t when poll reach delay offset jitter ============================================================================== +SunSITE.icm.edu 210.100.177.101 2 u 60 64 377 32.793 0.294 0.946 -75-19-182-46.Nb 141.82.25.202 3 u 37 64 377 30.565 -1.566 1.363 +re.uni-paderbor .DCF. 1 u 56 64 377 33.836 4.432 1.691 -cloud.a-schieb. 129.69.1.153 2 u 62 64 377 24.333 0.130 0.670 xGENERIC(0) .GUDE. 0 l 13 64 307 0.000 -115676 622495. *GPS_NMEA(0) .GPS. 1 l 4 16 377 0.000 5.555 62.038 oPPS(0) .PPS. 0 l - 16 377 0.000 -0.007 0.003
The 'o' in front of the PPS record says that the PPS is being used. The '*'in front of the GPS_NMEA record says that it provides the best time of all. Just the GUDE clock has been sorted out for whatever reasons. So let's check what's wrong.
3.5.4 Diagnose DFC-77
There is a little tool as part of the ntpd distribution called testdcf.
/root/ntp-4.2.6.p5+dfsg/parseutil/testdcf
Stop NTPD (free the /dev/refclock-0 device) and call it this way:
root@multirx:~# testdcf /dev/refclock-0 DCF77 monitor 4.10 - Copyright (C) 1993-2005, Frank Kardel RADMLSMin....PHour..PMDay..DayMonthYear....P RADMLS1248124P124812P1248121241248112481248P - ............................................ *** INCOMPLETE \ ............................................ *** INCOMPLETE \ ............................................ *** INCOMPLETE ... | ............................................ *** INCOMPLETE | ............................................ *** INCOMPLETE \ ............................................ *** INCOMPLETE \ ............................................ *** INCOMPLETE \ ............................................ *** INCOMPLETE \ ............................................ *** INCOMPLETE RADMLSMin....PHour..PMDay..DayMonthYear....P RADMLS1248124P124812P1248121241248112481248P
If you get something like this, you receive DFC-77. Congratulations!
The GUDE clock has a blinking LED. If the connection to the Raspberry comes up properly (no reason it should not), it will be constant green. Whenever it receives a pulse, it will blink red. After a while it will blink red each second, and that's a good sign. Meaning you can leave it as it is. If it blinks properly, the above given command will return some output after a couple of minutes.
3.6 ADS-B
ADS-B is something new to me. Since jetplanes do transmit their current position, but as they do not broadcast a timestamp that goes with it, the treatment of these frames in the context of real-time data processing is somewhat problematic. After you have received the frame, you cannot determine when you have received it any more, unless you immediately attach a timestamp. The aircraft will move on and you will not even be able to tell when it was where.
This is the reason why the combination of ADS-B and a high precision time receiver makes perfect sense.
And I was really overwhelmed how easy it was to receive these messages.
The trick is probably the 1090 MHz bandpass filter, which is offered by FlightAware.
All you computer folks listen up! An ADC is a wide band sensor. Every signal somewhere else in the spectrum will overrun the ADC input and make your neat little DVB-T stick deaf. The way out is a passive piece of hardware. Sorry, you cannot do this in software! You really need to filter away what you never want to hear. This is the way how you can increase sensitivity of your DVB-T stick tremendously.
The only thing you need is a tool called dump1090:
https://github.com/antirez/dump1090
Then compile it, store it somewhere under /usr/local/bin/ and use it, first of all interactively.
root@multirx:~# dump1090 --interactive
will give you a constantly updating overview of your local airspace:
Hex Mode Sqwk Flight Alt Spd Hdg Lat Long Sig Msgs Ti/ ------------------------------------------------------------------------------- 400152 S 3051 33425 474 067 47.794 10.937 26 89 2 71C218 S 2311 KAL914 33000 518 060 47.795 11.394 39 575 0 49D2B9 S 1000 TVS6X 39000 457 050 48.116 11.596 166 3831 0
... which you can match with a glance at https://www.flightradar24.com/
Sorry, no big traffic here in the moment. But the list can become rather crowded.
A very cooooool property of this nice little tool is that it includes a webserver and provides a live updated web interface of your airspace:
root@multirx:~/dump1090# ./dump1090 --interactive --net --net-http-port 8080
Make sure to start the tool in its own directory. Otherwise this feature will not work...
Some ideas are still pending:
- http://forum.flightradar24.com/threads/8591-Raspberry-Pi-How-To-Install-Raspian-OS-Dump1090-FR24-Data-Feeder
- http://myscope.net/adb-s-flugradar/
3.7 AIS
My first test with the AISspotter device was very promising. The UDEV rules will provide a symbolic link to the correct device no matter in which sequence they will be instantiated. So we connect to /dev/ais0
As of now I still cannot say anything because there are no ships here in Munich. I will install the system in Regensburg with all the ships on the river Danube. We will see the outcome.
3.8 I2C
Previous experiments have shown that air pressure has some influence on the stability of a local oscillator. Even though I have not forseen any real temperature isolation for this kind of experiment, I am interested in the environmental data of the box.
I have been using Lady Ada's I2C tools for the BMP180 sensor.
root@multirx:/usr/local/bin# cat get_env_data.py | grep -v "#" from Adafruit_BMP085 import BMP085 bmp = BMP085(0x77) temp = bmp.readTemperature() pressure = bmp.readPressure() altitude = bmp.readAltitude() print "%.2f C" % temp, print "%.2f hPa" % (pressure / 100.0), print "%.2f m" % altitude
Besides this little piece of code I have the other BMP085 stuff... Here is what I get when I execute it:
root@multirx:/usr/local/bin# ./get_env_data.py 4.90 C 964.69 hPa 412.20 m
Pretty cool outside, the current air pressure and the derived altitude above sea level, which is a pretty accurate guess for Munich.
I use a shell script to store these values in kind of a minutely overrunning buffer:
root@multirx:/usr/local/bin# cat /usr/local/bin/env_data.sh #!/bin/bash DATAFILE="/var/log/envdata/envdata.log" YESTERDAY="/var/log/envdata/yesterdaydata.log" TODAY="/var/log/envdata/todaydata.log" tail -3000 $DATAFILE \ | grep $(date -d yesterday +"%Y-%m-%d") \ > $YESTERDAY tail -30000 $DATAFILE \ | grep $(date +%F) \ > $TODAY # Ueberlauf verhindern cat $YESTERDAY > $DATAFILE cat $TODAY >> $DATAFILE COMMAND=/usr/local/bin/get_env_data.py MESSZEITPUNKT=$(date '+%F %T') MESSUNG=$($COMMAND) OUTLINE="$MESSZEITPUNKT $MESSUNG" echo $OUTLINE >> $DATAFILE
Here is what I get:
2016-12-10 22:27:01 5.10 C 964.68 hPa 412.37 m 2016-12-10 22:28:02 5.20 C 964.70 hPa 412.03 m 2016-12-10 22:29:01 5.00 C 964.67 hPa 412.29 m 2016-12-10 22:30:01 5.00 C 964.75 hPa 412.72 m 2016-12-10 22:31:01 5.10 C 964.71 hPa 412.72 m 2016-12-10 22:32:01 5.00 C 964.74 hPa 411.94 m 2016-12-10 22:33:01 5.00 C 964.68 hPa 412.55 m 2016-12-10 22:34:01 5.10 C 964.68 hPa 412.29 m 2016-12-10 22:35:01 5.00 C 964.61 hPa 413.15 m 2016-12-10 22:36:01 5.10 C 964.71 hPa 412.72 m 2016-12-10 22:37:01 5.00 C 964.65 hPa 412.72 m 2016-12-10 22:38:01 5.00 C 964.68 hPa 412.89 m 2016-12-10 22:39:01 4.90 C 964.64 hPa 413.41 m
And I chart this with GNUPlot.
root@multirx:/usr/local/bin# cat prep_env.gpl #!/usr/bin/gnuplot set output '/var/www/html/graphs/envstats.png' # Save the plot to a *.png file set term png font arial 8 size 900, 600 # Set size of image: width, height set grid set xdata time set timefmt "%Y-%m-%d %H:%M" set mxtics default set xtics border mirror norotate autofreq set ytics border mirror norotate autofreq set mxtics set xlabel "Tageszeit" set ylabel "Luftdruck [hPa]" set y2label "Temperatur [C]" set format x "%H:%M" set y2tics set y2range [-10:40] #set xrange [:] plot "/var/log/envdata/envdata.log" using 1:5 smooth unique title "Luftdruck", \ "/var/log/envdata/envdata.log" using 1:3 smooth unique title "Temperatur im Gehäuse" axes x1y2 unset multiplot # EOF
4 Results
5 Data Integration
In the future I will write some more things about how I am using the data, but that exceeds the current scope of this little project.