Last summer, I hinted that I was about to switch from the monolithic programme with all its threads to a constellation of separate processes.
The main reason was that the monolithic application needs a restart for any change in the configuration and that a crash/exception on one thread breaks everything.
True, having several separate processes impose to find a way to start them all in the first place and something else to look after them. It also uses quite a bit of memory (because of Python overhead, a tiny process is almost as memory hungry as a small one). Last but not least, the interprocess communication can be problematic.
Enters the Raspberry Pi 2
Fortunately, the Raspberry Pi Foundation released the Raspberry Pi 2 which has now a quad-core CPU, 1GB of memory (and even the Raspberry Pi 3, more recently, but I doubt it would make much difference here). At ~ 3MB of memory per process, there is plenty of available RAM! And also, starting a Python process is now almost immediate compared to the 5-10 seconds needed on the 1-B+ model.
Communication : MQTT
Thinking about it, there is not much need of communication between processes. In the majority of cases, it is all bout sending the data to the display interfaces and to a database (for the sensor part).
Anyway, the ubiquitous MQTT can solve all the communication problems... These days, it seems that there isn't a similar hub project around which isn't using MQTT either at the core or at least for plug-ins communication.
I have already detailed the way I format the topic and the payload of the messages.
Every process is now using a bootstrap library which manages the daemonisation, the MQTT communication, and logs. There are 2 types of messages : the data and metadata (starting, heartbeat, ...).
Currently the model used is the following:
All measurements have the MQTT retain option activated to keep the last value available to a reconnecting process
Pushover is a notification system for mobiles (and desktop)
I am currently using 2 storage systems: RRD and a timeseries database (test in progress)
'display_bikes' and 'display_transport' call external webservices and/or doing web scraping of pages. Resulting data is only displayed but never stored.
As mentioned previously, the kit I bought has 2 sockets. But knowing that the remote can control 4 (it has a total of 8 buttons), it would be good to be able to use the remaining unused buttons. And why not, since the state of the socket can't be retrieved, to intercept and store the current state.
Maybe there is a way to squeeze yet another protocol?
It turned out that it was quite easy to do without much interference to the existing two decoding pipelines!
Meet the 433Mhz receiver
Basically, the signal from the remote will appear as a series of High pulses which are shorter (~ 320µS & ~ 960µS) compared to the one from the sensors (~ 500µS/1300µS and 1900µS/3800µS). If a string of these specific pulses is detected, then we switch to the remote decoder.
The signal being sent at least 3 times from the remote, this mean that we can intercept the second or the third transmission. The transmission itself starts with a "very long" pulse (31 times the base pulse). The encoding is based on a tri-state system which basically means, since we can ignore the floating state, that every other bit must always be a 1. The data consists in the address part, the button part and the state (ON/OFF) repeated but inversed.
1010111010 1110101010 1110
can be decoded in (checking then removing all the odd bits):
00100 10000 10
and interpreted as:
address (DIP switches): off-off-on-off-off
Since there is now a little bit more than Lacrosse sensor decoding, I renamed the project on GitHub to 433Mhz receiver...
A few days ago, as I was in a nearby DIY store (looking for something I didn't find in the end), I came by the Electricity aisle and noticed cheap remote sockets. These are a not new thing, I owned some more than 25 years ago — so unstable there were switching on and off randomly — and in France at least, the Belgian company Chacon made a successful product (DI.O) with this kind if items.
One of the major issue with them is that they are without any feedback of the status like you could have on a modern "smart plug". Yet, where a Z-wave socket alone is at least 40€, here a pack of 2 sockets was less than 15€ including the remote and its battery. The "manual" includes a EC Declation of Confirmity so I assume there are safe to use. The power is limited to 1000W but for Christmas lights, this is not an issue!
Obviously, the idea was also the do the reverse engineering on the protocol and if possible try to emulate the remote itself like I did several times in the past. Turns out that this very model of remote is used in dozens of packs and the circuit inside is well known and even officially documented!
Protocol and Home Automation
A quick search show pages and pages on the subject. These one are now called "Smart Home" but were previously sold under the Phenix brand by IDK. Protocol seems virtually identical to the Elro Home Easy plugs.
The next question was: What is the best to integrate these elements in the existing installation? There are two main parts for this: controlling the sockets but also making the most of the remote (especially the two unused ON/OFF buttons).
As described last year, I own a La Crosse Weather Station to which I added several additional (TX2 & TX3) sensors.
Trouble is, as the technology is phased out in favor of a 868Mhz based one, these sensors are harder to come by and are also a bit pricey.
Unexpectedly, I noticed a news about a firmware update of the RFXtrx433E from RFXCom talking about a well known chinese online shop's version of a sensor. Indeed, I was able to become the owner of a "433MHz Wireless Weather Station Digital Thermometer Humidity Sensor" for the very lucky price of US$8.88 all included.
These sensors are apparently compatible with the Alecto WS1700 ones. The protocol was decoded and published on the Pilight wiki and forums.
It looked simple enough so I decided to modify my Arduino/Digispark script in order to decode it as well.
Yet there was a catch! La Crosse TX3 protocol uses a variation of pulses on the high level with timing on the low level being almost identical (and ignored by my decoder's implementation).
For the WS1700... it's the opposite! Hence only a succession of 37 identical 500µS pulses was the only thing seen.
Also, the signal finishes by a "very long" (~9200µS) Low and, as usual with this kind of device, the whole sequence is repeated several times.
If we detect at least 8 continous short high pulses, then we know it's not a TX3 sensor but probably a Alecto compatible one and therefore from now on only low pulses will be read.
The strategy is then to wait for very long pulse, read the full signal, checking the header (~1900µS ~3800µS ~1900µS ~3800µS), and at last validating with the last pulse which should be again a very long one.
The new algorithm is represented in the following flowchart:
The Id, which is randomly generated when inserting new batteries -- same as TX3 -- is ignored... I will assume that nobody else has a compatible sensors around and use the channel number instead.
Channel (there is a physical switch to select it) is coded 00=CH1, 01=CH2, 02=CH3. This will be my sensor ID because it is fixed. But to avoid any clash, I add 1001 so CH1=1001, CH2=1002, CH3=1003.
Battery will be reported as OK or KO.
There is a small button at the back of the sensor to force the emission. Apart for tests, it is not that useful and we can ignore the bit reporting this.
Temperature is in 10th of degrees Celcius so value is divided by 10. Negative values are taken care of as well.
Humidity is straight out the signal received except if not read properly (0%).