Europa Oktober 2013

Foto’s van my huidige besoek aan Europa:

Foto’s tydens die reis tussen stede:
https://picasaweb.google.com/103168876809682609244/EuropaOktober2013Onderweg?noredirect=1

Antwerpen:
https://picasaweb.google.com/103168876809682609244/EuropaOktober2013Antwerpen?noredirect=1

Lille:
https://picasaweb.google.com/103168876809682609244/EuropaOktober2013Lille?noredirect=1

Göttingen:
https://picasaweb.google.com/103168876809682609244/EuropaOktober2013Gottingen?noredirect=1

Parys:
https://picasaweb.google.com/103168876809682609244/EuropaOktober2013Parys?noredirect=1

Leuven:
https://picasaweb.google.com/103168876809682609244/EuropaOktober2013Leuven?noredirect=1

Graphing weather data

For the past few years I have been running a couple of weather stations. The data is logged in a MySQL database and the latest values are displayed on a website. Graphing was done by the proprietary software that came with the weather stations.

Recently some of the weather stations started to give me problems, resulting in my graphing being messed up. I decided to create my own graphs from the data in MySQL. Drawing graphs in real-time from MySQL is very intensive, so a better solution with caching is needed.

RRDtool is the de-facto standard when it comes to time-based graphing. It is widely used for weather data, and is lightweight enough to create graphs instantly when a webpage is loaded.

By scheduling a Python script every minute, I pull the latest data from MySQL and add it to separate RRD’s for every sensor. RRD automatically does averaging over certain time periods to make the graphs look good. This works great for sensors like temperature, humidity, barometric pressure and wind speed.

Wind direction on the other hand is a little more difficult. By averaging all wind direction measurements, the resulting graph will always output a line around 180 degrees (South). This is because RRD has no concept of degrees, and a number around 0 is not at all close to a number around 360. A direction just west of north (+-350 degrees), and a direction just east of north (+-10 degrees), will thus provide a direction around south (180 degrees).

To work around this problem is not so easy, but mathematics has a tool we can use. We can break a direction up into two components describing the angle. An east-west, or x component and a north-south, or y component.

x will be the sine of the angle, and y will be the cosine of the angle.
x=sinθ
y=cosθ
This computation is done in Python.

The x and y components are then added to a RRD where they are averaged for certain periods.

When the graph of the wind direction is drawn, the averaged x and averaged y is taken and the angle is computed:
θ=arctan(x/y)
This is done by RRDgraph’s reverse polish notation parser.

If the answer is a negative angle, 360 degrees are added to put it in the correct range for wind direction measurements. The result of this method can be seen here on my weather website.

A strange and unexplained phenomenon is seen on the longer term graphs. The average direction always tends to approach 90 degrees. If this is an error, I do not know. Seeing as the general accepted average direction in and around my city is south-east, it can quite easily be an error.

If anyone has an idea why the average is around 90 degrees, please leave a comment.

Update 2013-10-22

The problem that I describe above was fixed after further investigation of the code I use. Both Python and PHP’s trigonometric functions use radians and not degrees. Being used to measure wind in degrees I totally forgot this, and used degrees in the functions. The results were of course totally wrong. After changing the code to first convert to radians, the resulting graphs look much more believable.

Today someone contacted me to ask if I can tell them what is the prevailing wind direction in Stellenbosch. It gave me a reason to do some analysis in the data and the results are quite interesting.

Firstly I ran a SQL query to count how many times I had a sample for every unique direction. This result set was then exported to a spreadsheet, where the directions were aggregated in the 16 wind sectors. Below you will see two graphs from this aggregated data.

It is interesting to see the prevailing wind being north-east, while the second most common direction is exactly the opposite, south-west.

kragmeter

kragmeter

Comparing the directions of the three datasets I have, one gets the following graph:

kragmeter

DVB-T2 in South Africa – a year later

It is a year since I started to have a look at the digital TV transmissions in Stellenbosch. Initially I used a PCTV Systems DVB-T2 290e to receive the transmissions on my computer under the Linux operating system.

Recently I acquired a DVB-T2 set top box (STB) from eBay. As this STB is made in china the quality is not that good. It is however good enough to receive our DVB transmissions.

After plugging in the STB I was surprised to see that there are more TV channels available than previously. Except for SABC and eTV, it is mostly test and local channels. The list of radio channels are still the same.

The image quality differs a lot between the channels, and none uses HD yet. The STB and PCTV stick are both struggling more to keep a lock on the signal since these extra channels were added. Perhaps due to less FEC or worse SNR. Even a good signal strength and quality (>90% according to the STB) would sometimes just lose lock.

What the audio is concerned, the volume is different between all channels. This is extremely irritating as one now needs to change the volume on your TV every time you change channel. Some channels have nice clear stereo audio, whereas others have mono. A few channels’ audio are over modulated and clips.

In short, a signal strength that would have given good audio and video on analog transmissions now could sometimes give you irritating video break up and bad audio.

Update 2013-06-25

Since I have the STB, only eTV had EPG data. The SABC channels seemed to be off the air yesterday evening. Tonight they are back, including EPG data for the coming week.

Update 2013-06-25

An SABC HD stream was being broadcasted for a while now, but it had a black image without any audio. In the past week I realised they started transmitting video on the channel.

The codec information for the SABC HD channel is as follows:
sabchd

One can compare this to the codec info for the normal SABC2 SD channel:
sabcsd

As can be seen the HD channel is transmitted in full HD. A different audio codec is also used for the HD channel than for the SD channels. Hopefully this codec will give better audio quality than what is being transmitted on the SD channels.

Reggio di Calabria, Italy

Currently I am visiting Reggio di Calabria in the south of Italy for three months. It is for an exchange program the university here has with the University of Stellenbosch.

Some of the photo’s I take will be uploaded to the following two web albums.

Calabria:
In and around the city of Reggio di Calabria:
https://picasaweb.google.com/103168876809682609244/ReggioCalabriaFebApr2013?noredirect=1
Scilla:
https://picasaweb.google.com/103168876809682609244/Scilla17March?noredirect=1

Weekend to Sicily:
Messina, Catania and the train trip: https://picasaweb.google.com/103168876809682609244/Sicily2124022013?noredirect=1
Taormina: https://picasaweb.google.com/103168876809682609244/Taormina2324Feb2013?noredirect=1
Volcano Etna: https://picasaweb.google.com/103168876809682609244/Etna22022013?noredirect=1

Weekend to Malta:
https://picasaweb.google.com/103168876809682609244/Malta14Maart2013?noredirect=1

Weekend to Rome and Surroundings:
Rome City: https://picasaweb.google.com/103168876809682609244/Rome811March?noredirect=1
Vatican City: https://picasaweb.google.com/103168876809682609244/Vatican9March2013?noredirect=1
Napoli and Pompeii: https://picasaweb.google.com/103168876809682609244/NapoliPompei10March?noredirect=1

Weekend to Tuscany
Florence:
https://picasaweb.google.com/103168876809682609244/Florence2225March?noredirect=1
Siena, Pisa and the Tuscan countryside:
https://picasaweb.google.com/103168876809682609244/Toscana23March?noredirect=1

Easter Weekend
Budapest & Vienna with Bus2alps:
https://picasaweb.google.com/103168876809682609244/BudapestVienna28March2April?noredirect=1

More of Sicily
Palermo:
https://picasaweb.google.com/103168876809682609244/Palermo68April?noredirect=1

Rest of my travels:
https://picasaweb.google.com/103168876809682609244/EuropaFebApr2013?noredirect=1
Germany:
https://picasaweb.google.com/103168876809682609244/DuitslandApril2013?noredirect=1
Netherlands:
https://picasaweb.google.com/103168876809682609244/NederlandAprilMei2013?noredirect=1

Hint: Try the “View map” link on the left after you have opened an album.

SDR with Raspberry Pi and DVB-T receivers

My holiday project is to play around with a Raspberry Pi and a RTL-2832U DVB-T USB receiver. This specific receiver has a mode where it dumps raw IQ samples over USB to the computer. The gnuradio experimenters realised this and therefore gnuradio supports this relatively cheap off the shelf USB receiver as a software defined radio.

2012-12-06
The Raspberry Pi has a rather tiny ARMv6 CPU running at 800MHz. Doing signal processing on this CPU (as the GPU is still too closed to do it on that), looked impossible to me.

Doing some tests on my netbook using gnuradio to decode FM radio broadcasts took anything from 30%-120% of CPU. I later discovered rtl_fm which included the entire FM demodulation chain I implemented in gnuradio into one single highly optimised program. This program took about 7% of CPU on my netbook. Things started to look a little less impossible and the next step was to find a Pi and test it on there.

Following the steps here I installed the RTL’s drivers on the Pi, running the Raspbian OS. After that I did the seemingly impossible and ran the following command:
rtl_fm -f 144800000 -s 22050 - | multimon -a AFSK1200 -t raw /dev/stdin

This command uses rtl_fm to receive the Amateur radio frequency of 144.8MHz and demodulate that to 22.05kHz audio. The audio is then piped to multimon, a general purpose digital decoder program for linux. Multimon is set up to try and demodulate AFSK1200, the amateur radio packet mode at 1200baud.

At this point I was really impressed. Just by running that command that used 7% of CPU on my AMD fusion netbook (frequency scaled to 800MHz), did not kill the Pi. It in fact used so little CPU that I thought something was wrong. Having a look with the top system monitor, the rtl_fm program stood at 18%-18.5% CPU usage and multimon stood at 1.5%-2%. Surely something was not working.

A few seconds later I heard an APRS broadcast on my HAM radio set and the decoded packet appeared in the terminal on the Raspberry Pi. Everything was indeed working correctly. I am oficially impressed by the performance of the Raspberry Pi and the osmocom rtl_sdr package/driver.

Well now that I got APRS decoding working, how about DTMF, CTCSS, SSTV and vox recording. Perhaps add a trasmitter and set up a controller for a repeater or a remote weather station.

More updates to follow!

2013-01-20
The past few weeks I built on the previous findings. I got DTMF and CTCSS detected by multimonNG, but only after I realised that there was a problem piping the raw audio data from rtl_fm to multimon. rtl_fm’s audio regularly has a DC offset, with which multimon can’t deal. Therefore I wrote a little C program to do DC blocking according to what I read on DSP guru.

The command as previously used would thus change to:
rtl_fm -f 144800000 -s 22050 - | DCblocker | multimon -a AFSK1200 -t raw /dev/stdin

The previous system was only able to decode AX.25/APRS packets. I however wanted to transmit packets too. Therefore I set up soundmodem on the Raspberry Pi, using a Logitech USB soundcard. I built a VOX to do the PTT on the radio according to G4ILO’s website. This was then connected to an Icom handheld radio.

Now that I got this working, I wanted to do something with it. Because the Raspberry Pi has a few GPIO pins, I thought that one could switch something using APRS messaging. I built a repeater controller. The python source code can be found on Github here.

Muskietejag

Dis Stellenbosch. Dit raak somer. Warm nagte lê voor en het eintlik al reeds begin. Dis nie ‘n slegte gevoel om lekker warm te gaan slaap na die lang wintermaande nie. Maar o wee! Saam met die warmte en slote vol water om ons heen, is daar ‘n pes aan die broei in die dorp. Dis weer muskiettyd.

Gisteraand net voor ek gaan slaap sien ek hierdie teen my kamermuur:
voor

Die eerste muskiet van die seisoen het dit gewaag om in my kamer in te klim!

Ek dink AD Keet som dit baie mooi op in sy gedig Muskiete-jag:

Jou vabond, wag ek sal jou kry
van jou sal net ‘n bloedkol bly
hier op my kamermure.
Deur jou vervlakste gonsery,
deur jou gebyt en plaery
kon ek nie slaap vir ure.

Mag ek my voorstel, eer ons skei,
eer jy die doodslag van my kry –
my naam is Van der Merwe.
Muskiet, wees maar nie treurig nie,
wees ook nie so kieskeurig nie,
jy moet tog eendag sterwe.

Verwekker van malaria,
sing maar jou laaste aria –
nog een minuut van grasie.
Al soebat jy nou nog so lang,
al sê jy ook, ek is nie bang,
nooit sien jy weer jou nasie.

Hoe sedig sit hy, o, die ding!
Sy kinders kan maar kranse bring,
nou gaan die vabond sterwe.
Pardoef ! Dis mis ! Daar gaan hy weer !
Maar dood sal hy, sowaar, ek sweer –
my naam is Van der Merwe !

Die eerste slag was mis. Tweede nie. Die gevolg: ‘n kol teen my kamermuur.
na

So dis weer tyd vir Tabard en Citronellakersies lyk dit vir my…

Spinnekoptyd!
opdatering 2012/11/14

Vanaand het die eerste spinnekop van die seisoen in my woonstel ingeklim. Hier is ‘n foto net nadat ek hom weer vrygelaat het buite.
spinnekop

Power measurement – technical details

I had a few requests for a little more technical detail around this project. For now I will post only a few photos, but will extend it with circuit diagrams and source code in a little while.

The hardware

The light sensor stuck to the front of the prepaid electricity meter covering the meter’s flashing LED:
sensor

The Arduino with ethernet shield counting the flashes and uploading the count every minute to a computer in my house. The Arduino is currently powered by USB from an old netbook, but this will be changed as soon as I acquire the hardware to build a PoE extracter.
arduino

To sense when the LED flashes, I initially wanted to use a light dependant resistor (LDR). Through experimentation it was seen that this would work, but will be very sensitive to other ambient light sources. A friend told me that he thinks a normal LED would work too. Experimenting with a normal red LED did not work at all. However I had some clear plastic LED’s in my toolbox. I tried a random one and miraculously it did work. Even better than the LDR. I think it is a UV LED. The theory behind the use of an LED to sense light is that a photon falling in on the semiconductor substrate causes a small current to flow. This consequently causes a voltage drop at the junction between the LED and the resistor. This is also the point at which I measure with my analog-to-digital converter (ADC). In the schematic below note the polarity of the LED. Through experimentation it was seen that the setup works with the LED connected both ways around, but the voltage drop is much more noticable when it is connected as depicted in the diagram (wrong way around for turning the LED on).

voltage divider

Firmware
The Arduino libraries make everything a breeze. One thing it however does not support is interrupts. So one needs to enable them by hand.

Through tests I saw that if I sample the ADC every 10ms, I will get three samples while the LED is on. As 10ms is the time between ADC samples in the Arduino examples, I also use this value. I set a timer to cause an interrupt every 10ms, check if the LED state changed, and if it did change from off to on, increment a counter.

In the main loop, I check the amount of time that passed since the Arduino started up. If this is equal to 60 seconds, or the difference between the last data upload time and now is 60 second, I upload the flash count.

The code also contains a few error checking conditions.

Communication

As the most network libraries support the download of websites, this is normally the communication method I use. When fetching a website one specifies the url followed by a few parameters.
For example: www.myserver.com/page_name?paramter1=value1&parameter2=value2

So to fetch a website, you actually send data to the web server. Some people might call this a RESTful API. I just call it URL parameters.

The Arduino libraries supports this and handling the parameters and values in PHP on the server’s side is trivial.

So to summarise. The Arduino “visits” a web page on a computer in my house, giving the flash count as an URL parameter. On the computer I extract this parameter/value from the URL, convert the flash count to Watts and then save the data with a time stamp in a MySQL database.

Database and statistics

The database only has one table containing the time stamp and electricity usage at that time in Watt. It is actually an average usage of the minute before that time, but minutely averages are accurate enough for my use.

Using the jpgraph PHP library I select the last 24 hours of data and plot it on a graph. I do the same for 7 days too. These two graphs are then displayed on a webpage that refreshes every 10 minutes.

Also displayed on this page I have the total amount of electricity units I used per day in the past week (summarise the wattage for the day and then divide by 60), and also the average amount of units I used per day.

Future plans

Future plans include computing how much the electricity should have cost me every month. I also want to compare my electricity usage between days I leave my hot water cylinder on for the entire day, versus days I only have it on for 1.5 hours in the morning.

Building a meter I can plug in between my computer and the wall socket will be nice to log my computer’s electricity usage.

Miniaturising and simplifying the hardware I use at the moment will reduce cost and enable wider application. A future version might also be Wifi-enabled.

Power measurement

I live in a flat with a prepaid electricity meter. Buying enough electricity is a task I am faced with every month. One advantage of this system is that you can see how your electricity “units” count down, telling you how much electricity you are using.

Together with an LCD showing the amount of units that are left, the electricity meter also has an LED that flashes faster or slower according to your electricity usage. Below this light it says “1000 flashes per kWh”. Kilowatt-hour is the technical term for what we call a “unit” of electricity.

kragmeter

When I stand in front of the meter I would like to know my current consumption of electricity. I can thus do this by timing the duration between flashes, but this will need very high accuracy of timing. Another way of measuring the consumption is by counting the amount of flashes per minute.

Now a little bit of math:

math
Thus for every flash I count in a minute I use 60 Watts of electricity.

Doing this by hand gets tedious very quickly. So as an electronic engineer I made a plan. I had a Arduino Mega development board with an ethernet shield lying around. The Arduino was programmed to count the flashes per minute and upload the flash count to a computer in my house running a MySQL database.

When I visit a specific website on the computer running the database I now get a graph displaying my electricity usage for the past 24 hours and past 7 days. This graph is drawn by the jpgraph PHP library from the data selected out of the MySQL database.

An example of the graph is shown below. One can clearly see the times I boiled a kettle of water or used the microwave; the hot water cylinder was on; the time at night my computer was turned off; and the on and off cycle of the fridge.

kraggrafiek

A live version of the graphs can be seen here.

A few statistics
One day with the hot water cylinder turned off, me not home and thus only the fridge, mini PC and network running (base load) used 5.187 kWh of electricity.

But how does it work?
Want to know more about how I did it? Go and read the Power measurement – technical details.

IPv6 experimentation

With the depletion of the IPv4 address space, many people are migrating to IPv6. The knowledge of how it works and how to set it up is still limited though.

To experiment and learn how IPv6 works, we set up our own IPv6 internal network in our laboratory. The lab’s network was already running with an IPv4 private IP range routed via a gateway performing network address translation (NAT/masquerading) towards the internet. Our only access to the internet is via a HTTP proxy and therefore we used an openVPN tunnel to a VPS (Virtual Private Server, a VM hosted by an ISP on the internet) to get routed internet access.

Adding IPv6 to this setup meant getting a 6to4 tunnel to the VPS. We used Hurricane Electric’s Tunnelbroker for this. They also provided us with a /64 and /48 IPv6 range we may use.

Our current setup is as follows:
setup

The best thing to see after all this trouble:

jpmeijers@shannon:~$ ping6 ipv6.google.com
PING ipv6.google.com(2c0f:fb50:4001:800::1013) 56 data bytes
64 bytes from 2c0f:fb50:4001:800::1013: icmp_seq=1 ttl=56 time=531 ms
^C
--- ipv6.google.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2000ms
rtt min/avg/max/mdev = 463.276/521.435/569.208/43.873 ms

The latency is a little high, but compared to our normal IPv4 internet it is not that much worse. Being able to access the IPv6 internet weighs much more.

2013-02-20
For some reason I did not get an IPv6 IP today, but I restarted the radvd (router advertising daemon) on the VPS to fix it.

Ad-hoc network workshop

The electronic engineering telecommunications group held a workshop on ad-hoc wireless networks on 30 and 31 August 2012. Presentations were given by both Stellenbosch students as well as a few visiting professors from France.

Our French visitors included Prof Nathalie Mitton and Milan Erdejl from INRIA in Lille, Prof Fabrice Valois from University of Lyon, Dr Antoine Gallais from INRIA in Grenoble, and Dr Pierre Lemonde from the French embassy in South Africa.

The workshop was part of a crossed season collaboration between INRIA and Stellenbosch University.

More details can be found on the workshop’s official webpage, by clicking here.

The last night out in Stellenbosch
Out in STB