Dual Source GPS/DCF77 NTP-Server with AIS and ADS-B receivers

Aus DL8RDS Wiki
Wechseln zu: Navigation, Suche

1 Scope

This project is a continuation of my stream of experiments with the NTP protocol and GPS-disciplined clock models. I need to reference the past projects before we go into details:

These past projects raised the question, how precise a DCF-77 clock would be. And this project here also needs to be seen in the context of an April 1st joke of Rainer DF2NU, who proposed that DCF-77 can be switched off now because the GPS system is there and it is much more precise. So I started to collect ideas to investigate how less precise a DCF-77 disciplined clock model would be compared to the GPS Pulse Per Second. The reason for the reduced precision is because the 77,5 kHz signals are reflected randomly in the ionosphere and the exact reflection point cannot be predicted, resulting in a varying signal runtime.

In addition to the clock question I was thinking of setting up a pair of receivers for the AIS frames broadcasted around Regensburg by the shipd on the danube and I wanted to receove ADS-B frames. Even though slowly navigating ships allow for some frame processing delay, and in case the delay is a couple of seconds, one would still consider such an application a real time application. A jet plane however travels quite distance in a couple of seconds, so a positioning application would need to add some timestamping logic in order to allow for a realistic position information, particularly if received by multiple receivers.

For this reason it makes perfect sense to add a precise clock to a position frame receiver setup.

2 List of components

~ 560 Euro

The prices are partly a bit over the market, roundet up, so that there are no surprises.

3 Configuration

3.1 Image and Packages

The Raspberry installation is based on a Minibian 2016-03-12 image: https://minibianpi.wordpress.com/

I found that sometimes it is hard to install it into a running state with improper readers / adapters. But if you have a good adapter, it should work properly with any SD card.

The minibian standard image does not have a "pi" user like Raspian, but the login succeeds with "root" and "raspberry". Of yourse, this information is just for those who want to replicate my project. Be sure that I have chosen another password, and please do the same :-)

If you choose to attach a serial console to your Raspberry, you must make sure that the agetty stops attaching to your /dev/ttyAMA0 device. You can do that with the following commands:

systemctl stop serial-getty@ttyAMA0.service
systemctl disable serial-getty@ttyAMA0.service
systemctl mask serial-getty@ttyAMA0.service

Then reboot and check if it has succeeded using lsof. I think it does not come with the Minibian distro, so it is one of the tools you need to install:

lsof | grep ttyAMA0

3.2 Hardware Connectivity and Kernel

I configured the kernel like this:

cat /boot/config.txt
gpu_mem=16
dtparam=i2c1=on
dtparam=i2c_arm=on
dtoverlay=pps-gpio,gpiopin=18
init_uart_baud=9600
enable_uart=0

Accordingly the kernel modules configuration file /etc/modules looks like this:

snd-bcm2835
i2c-bcm2708
i2c-dev

Since we have a FTDI to connect the GPS and another FTDI that is used in the GUDE DFC-77 clock, we need to keep them apart and ensure that they will always and forever in the future use the correct device name:

Check who is who:

root@multirx:/etc/udev/rules.d# lsusb | grep 403
Bus 001 Device 006: ID 0403:6015 Future Technology Devices International, Ltd Bridge(I2C/SPI/UART/FIFO)
Bus 001 Device 005: ID 0403:6001 Future Technology Devices International, Ltd FT232 USB-Serial (UART) IC
Bus 001 Device 004: ID 0403:e88a Future Technology Devices International, Ltd Expert mouseCLOCK USB II

Then create this file:

vi /etc/udev/rules.d/70-persistent-dev.rules
KERNEL=="ttyUSB*", ATTRS{idProduct}=="e88a", SYMLINK+="refclock-0"
KERNEL=="ttyUSB*", ATTRS{idProduct}=="6001", SYMLINK+="gps0"
KERNEL=="ttyUSB*", ATTRS{idProduct}=="6015", SYMLINK+="ais0"

These statements will give me the following devices:

/dev/refclock-0
/dev/gps0
/dev/ais0

The names are important because they need to correspond to the expectations of the accoring NTP modules.

We need to ensure that certain kernel modules will NOT be loaded automatically:

root@multirx:~# cat /etc/modprobe.d/rtl-sdr-blacklist.conf
# This system has librtlsdr0 installed in order to
# use digital video broadcast receivers as generic
# software defined radios.
blacklist dvb_usb_rtl28xxu
blacklist e4000
blacklist rtl2832

3.3 Packages

I found that it makes sense to install the following tools:

apt-get update
apt-get install i2c-tools      # I2C-Toolkit for commandline
apt-get install python-smbus   # Python library for I2C
apt-get install libi2c-dev     # library for C
apt-get install git            # need this later...
apt-get install lsof           # determine who is using the serial port
apt-get install apache2        # for all the later web stuff
apt-get install lsusb          # see who is connected via USB
apt-get install gnuplot        # charting tool

3.4 Fundamental system config

Configure the timezone using the command dpkg-reconfigure tzdata

Since the NTPd configuration is bound to the DHCP client by default, it is essential to break this dependency. Otherwise the NTP configuration provided by the DHCP server will be used and the local configuration will be overwritten:

  • Remove the file /var/lib/ntp/ntp.conf.dhcp
  • Edit the file /etc/dhcp/dhclient.conf and change the request statement so that the ntp servers will not be updated any more:
request subnet-mask, broadcast-address, time-offset, routers,
        domain-name, domain-name-servers, domain-search, host-name,
        dhcp6.name-servers, dhcp6.domain-search,
        netbios-name-servers, netbios-scope, interface-mtu,
        rfc3442-classless-static-routes;
#, ntp-servers;

3.5 NTP

3.5.1 Fundamental NTPD configuration

Here is my configuration:

root@multirx:~# cat /etc/ntp.conf | grep -v "#" | grep -v "^$"
driftfile /var/lib/ntp/ntp.drift
statistics loopstats peerstats clockstats
filegen loopstats file loopstats type day enable
filegen peerstats file peerstats type day enable
filegen clockstats file clockstats type day enable
filegen sysstats  file sysstats  type day link enable 
tos  minsane 3
tos  orphan 10
tos  mindist 0.4 
server 0.debian.pool.ntp.org iburst
server 1.debian.pool.ntp.org iburst
server 2.debian.pool.ntp.org iburst
server 3.debian.pool.ntp.org iburst
server 127.127.8.0 mode 19 prefer
fudge  127.127.8.0 flag2 1 stratum 0 refid GUDE
fudge 127.127.20.0 time2 0.435  flag1 0  refid GPS stratum 1 flag3 1
server 127.127.22.0 minpoll 4 prefer
fudge 127.127.22.0 refid PPS flag2 0 flag3 1
logfile /var/log/ntp
logconfig =all
enable calibrate
restrict -4 default kod notrap nomodify nopeer noquery
restrict -6 default kod notrap nomodify nopeer noquery
restrict 127.0.0.1
restrict ::1
restrict        172.16.0.0  mask 255.240.0.0
restrict        192.168.0.0 mask 255.255.0.0
root@multirx:~# 

3.5.2 Check that PPS works

The /dev/pps0 should be created upon boot if the Kernel-PPS module is loaded.

root@multirx:~# ppstest /dev/pps0
trying PPS source "/dev/pps0"
found PPS source "/dev/pps0"
ok, found 1 source(s), now start fetching data...
source 0 - assert 1462395271.608254834, sequence: 4022 - clear  0.000000000, sequence: 0
source 0 - assert 1462395272.608254999, sequence: 4023 - clear  0.000000000, sequence: 0
source 0 - assert 1462395273.608254163, sequence: 4024 - clear  0.000000000, sequence: 0
source 0 - assert 1462395274.608255327, sequence: 4025 - clear  0.000000000, sequence: 0
source 0 - assert 1462395275.608373487, sequence: 4026 - clear  0.000000000, sequence: 0
^C
root@multirx:~# 


3.5.3 Ensure that you have a NTPd what can handle PPS and DFC-77

Since I decided against using /dev/ttyAMA0 for the NMEA interface for NTP, I was rather free in terms of the serial device. Yet, I connected the PPS pin to GPIO 18.

The package, once again has to be modified and rebuilt:

  • Get the build dependencies and the source code of the package:
cd ~
apt-get -y build-dep ntp
apt-get -y source ntp
  • The source code was stored in my home directory. In the moment and for me it is the version ntp-4.2.6.p5+dfsg. Note that it could be different for you.
cd ntp-4.2.6.p5+dfsg/debian/
  • Edit the rules file and add the missing support for PPS/ATOM. Search for configure and append a line at the end of the statement. Don't forget the \ at the end of the previous line.
./configure CFLAGS='$(CFLAGS)' CPPFLAGS='$(CPPFLAGS)' LDFLAGS='$(LDFLAGS)' \
               --prefix=/usr \
               --enable-all-clocks --enable-parse-clocks --enable-SHM \
               --disable-debugging --sysconfdir=/var/lib/ntp \
               --with-sntp=no \
               --with-lineeditlibs=edit \
               --without-ntpsnmpd \
               --disable-local-libopts \
               --enable-ntp-signd \
               --disable-dependency-tracking \
               --with-openssl-libdir=/usr/lib/$(DEB_HOST_MULTIARCH) \
               --enable-ATOM
               --enable-JUPITER \
               --enable-ONCORE \
               --enable-LOCAL-CLOCK \
               --enable-RAWDCF \
               --enable-HOPF6021 \
               --enable-TRIMTSIP \
               --enable-NMEA
Note that I am enabling some more GPS technologies because I have some Navman Jupiters hhere, and I would like to use them in the future...
  • Now add the changelog file so that it reflects the change:
ntp (1:4.2.6.p5+dfsg-7+deb8u1) jessie-security; urgency=medium
into:
ntp (1:4.2.6.p5+dfsg-8+deb8u1) jessie-security; urgency=medium
  • Now 'cd' into the package directory and build the package.
cd ..
sudo dpkg-buildpackage -b
It will take a while now. In my case three deb files were created:
-rw-r--r-- 1 root root  351566 Dec  2 00:37 ntp_4.2.6.p5+dfsg-8+deb8u2_armhf.deb
-rw-r--r-- 1 root root   69132 Dec  2 00:37 ntpdate_4.2.6.p5+dfsg-8+deb8u2_armhf.deb
-rw-r--r-- 1 root root 1031588 Dec  2 00:36 ntp-doc_4.2.6.p5+dfsg-8+deb8u2_all.deb
  • Now install them:
dpkg -i ntp_4.2.6.p5+dfsg-8+deb8u2_armhf.deb
dpkg -i ntpdate_4.2.6.p5+dfsg-8+deb8u2_armhf.deb
dpkg -i ntp-doc_4.2.6.p5+dfsg-8+deb8u2_all.deb
  • Now restart the ntp server with the command service ntp restart and after ~ 1 minute it should give you this output using ntpq -p:
root@multirx:~# ntpq -p
     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
+SunSITE.icm.edu 210.100.177.101  2 u   60   64  377   32.793    0.294   0.946
-75-19-182-46.Nb 141.82.25.202    3 u   37   64  377   30.565   -1.566   1.363
+re.uni-paderbor .DCF.            1 u   56   64  377   33.836    4.432   1.691
-cloud.a-schieb. 129.69.1.153     2 u   62   64  377   24.333    0.130   0.670
xGENERIC(0)      .GUDE.           0 l   13   64  307    0.000  -115676 622495.
*GPS_NMEA(0)     .GPS.            1 l    4   16  377    0.000    5.555  62.038
oPPS(0)          .PPS.            0 l    -   16  377    0.000   -0.007   0.003

The 'o' in front of the PPS record says that the PPS is being used. The '*'in front of the GPS_NMEA record says that it provides the best time of all. Just the GUDE clock has been sorted out for whatever reasons. So let's check what's wrong.

3.5.4 Diagnose DCF-77

There is a little tool as part of the ntpd distribution called testdcf.

/root/ntp-4.2.6.p5+dfsg/parseutil/testdcf

Stop NTPD (free the /dev/refclock-0 device) and call it this way:

root@multirx:~# testdcf /dev/refclock-0 
  DCF77 monitor 4.10 - Copyright (C) 1993-2005, Frank Kardel

  RADMLSMin....PHour..PMDay..DayMonthYear....P
  RADMLS1248124P124812P1248121241248112481248P
- ............................................ *** INCOMPLETE                
\ ............................................ *** INCOMPLETE                
\ ............................................ *** INCOMPLETE                
...
| ............................................ *** INCOMPLETE                
| ............................................ *** INCOMPLETE                
\ ............................................ *** INCOMPLETE                
\ ............................................ *** INCOMPLETE                
\ ............................................ *** INCOMPLETE                
\ ............................................ *** INCOMPLETE                
  RADMLSMin....PHour..PMDay..DayMonthYear....P
  RADMLS1248124P124812P1248121241248112481248P

If you get something like this, you receive DFC-77. Congratulations!

The GUDE clock has a blinking LED. If the connection to the Raspberry comes up properly (no reason it should not), it will be constant green. Whenever it receives a pulse, it will blink red. After a while it will blink red each second, and that's a good sign. Meaning you can leave it as it is. If it blinks properly, the above given command will return some output after a couple of minutes.

3.6 ADS-B

ADS-B is something new to me. Since jetplanes do transmit their current position, but as they do not broadcast a timestamp that goes with it, the treatment of these frames in the context of real-time data processing is somewhat problematic. After you have received the frame, you cannot determine when you have received it any more, unless you immediately attach a timestamp. The aircraft will move on and you will not even be able to tell when it was where.

This is the reason why the combination of ADS-B and a high precision time receiver makes perfect sense.

And I was really overwhelmed how easy it was to receive these messages.

The trick is probably the 1090 MHz bandpass filter, which is offered by FlightAware.

All you computer folks listen up! An ADC is a wide band sensor. Every signal somewhere else in the spectrum will overrun the ADC input and make your neat little DVB-T stick deaf. The way out is a passive piece of hardware. Sorry, you cannot do this in software! You really need to filter away what you never want to hear. This is the way how you can increase sensitivity of your DVB-T stick tremendously.

The only thing you need is a tool called dump1090:

https://github.com/antirez/dump1090

Then compile it, store it somewhere under /usr/local/bin/ and use it, first of all interactively.

root@multirx:~# dump1090 --interactive

will give you a constantly updating overview of your local airspace:

Hex     Mode  Sqwk  Flight   Alt    Spd  Hdg    Lat      Long   Sig  Msgs   Ti/
-------------------------------------------------------------------------------
400152  S     3051           33425  474  067   47.794   10.937   26    89    2
71C218  S     2311  KAL914   33000  518  060   47.795   11.394   39   575    0
49D2B9  S     1000  TVS6X    39000  457  050   48.116   11.596  166  3831    0

... which you can match with a glance at https://www.flightradar24.com/

Sorry, no big traffic here in the moment. But the list can become rather crowded.

A very cooooool property of this nice little tool is that it includes a webserver and provides a live updated web interface of your airspace:

root@multirx:~/dump1090# ./dump1090 --interactive --net --net-http-port 8080

Make sure to start the tool in its own directory. Otherwise this feature will not work...

Some ideas are still pending:

3.7 AIS

My first test with the AISspotter device was very promising. The UDEV rules will provide a symbolic link to the correct device no matter in which sequence they will be instantiated. So we connect to /dev/ais0

As of now I still cannot say anything because there are no ships here in Munich. I will install the system in Regensburg with all the ships on the river Danube. We will see the outcome.

3.8 I2C

Previous experiments have shown that air pressure has some influence on the stability of a local oscillator. Even though I have not forseen any real temperature isolation for this kind of experiment, I am interested in the environmental data of the box.

I have been using Lady Ada's I2C tools for the BMP180 sensor.

root@multirx:/usr/local/bin# cat get_env_data.py | grep -v "#"
from Adafruit_BMP085 import BMP085
bmp = BMP085(0x77)
temp = bmp.readTemperature()
pressure = bmp.readPressure()
altitude = bmp.readAltitude()
print "%.2f C" % temp,
print "%.2f hPa" % (pressure / 100.0),
print "%.2f m" % altitude

Besides this little piece of code I have the other BMP085 stuff... Here is what I get when I execute it:

root@multirx:/usr/local/bin#  ./get_env_data.py 
4.90 C 964.69 hPa 412.20 m

Pretty cool outside, the current air pressure and the derived altitude above sea level, which is a pretty accurate guess for Munich.

I use a shell script to store these values in kind of a minutely overrunning buffer:

root@multirx:/usr/local/bin# cat /usr/local/bin/env_data.sh
#!/bin/bash

DATAFILE="/var/log/envdata/envdata.log"
YESTERDAY="/var/log/envdata/yesterdaydata.log"
TODAY="/var/log/envdata/todaydata.log"

tail -3000 $DATAFILE \
| grep $(date -d yesterday +"%Y-%m-%d")  \
> $YESTERDAY

tail -30000 $DATAFILE \
| grep $(date +%F) \
> $TODAY

# Ueberlauf verhindern
cat $YESTERDAY > $DATAFILE
cat $TODAY >> $DATAFILE
COMMAND=/usr/local/bin/get_env_data.py
MESSZEITPUNKT=$(date '+%F %T')
MESSUNG=$($COMMAND)
OUTLINE="$MESSZEITPUNKT $MESSUNG"
echo $OUTLINE >> $DATAFILE

Here is what I get:

2016-12-10 22:27:01 5.10 C 964.68 hPa 412.37 m
2016-12-10 22:28:02 5.20 C 964.70 hPa 412.03 m
2016-12-10 22:29:01 5.00 C 964.67 hPa 412.29 m
2016-12-10 22:30:01 5.00 C 964.75 hPa 412.72 m
2016-12-10 22:31:01 5.10 C 964.71 hPa 412.72 m
2016-12-10 22:32:01 5.00 C 964.74 hPa 411.94 m
2016-12-10 22:33:01 5.00 C 964.68 hPa 412.55 m
2016-12-10 22:34:01 5.10 C 964.68 hPa 412.29 m
2016-12-10 22:35:01 5.00 C 964.61 hPa 413.15 m
2016-12-10 22:36:01 5.10 C 964.71 hPa 412.72 m
2016-12-10 22:37:01 5.00 C 964.65 hPa 412.72 m
2016-12-10 22:38:01 5.00 C 964.68 hPa 412.89 m
2016-12-10 22:39:01 4.90 C 964.64 hPa 413.41 m

And I chart this with GNUPlot.

root@multirx:/usr/local/bin# cat prep_env.gpl
#!/usr/bin/gnuplot

set output '/var/www/html/graphs/envstats.png'      # Save the plot to a *.png file
set term png font arial 8 size 900, 600   # Set size of image: width, height

set grid
set xdata time
set timefmt "%Y-%m-%d %H:%M"
set mxtics default
set xtics border mirror norotate autofreq
set ytics border mirror norotate autofreq
set mxtics
set xlabel "Tageszeit"
set ylabel "Luftdruck [hPa]"
set y2label "Temperatur [C]"
set format x "%H:%M"
set y2tics
set y2range [-10:40]
#set xrange [:]

plot "/var/log/envdata/envdata.log" using 1:5 smooth unique title "Luftdruck", \
     "/var/log/envdata/envdata.log" using 1:3 smooth unique title "Temperatur im Gehäuse" axes x1y2

unset multiplot
# EOF

4 Results

5 Data Integration

In the future I will write some more things about how I am using the data, but that exceeds the current scope of this little project.

6 Images

2016-12-10-MultiRX-1.jpg 2016-12-10-MultiRX-6.jpg

2016-12-10-MultiRX-3.jpg 2016-12-10-MultiRX-2.jpg

2016-12-10-MultiRX-4.jpg 2016-12-10-MultiRX-5.jpg

2016-12-10-MultiRX-7.jpg