MQTT CAN Gateway Issues

I have been developing my code on in a virtual machine on my development box. But the intention was to deploy the apps to the same Linux server MQTT is running on.

To this end I checked all the code into SVN and checked it out on the other machine. After the pain of remembering all the libraries to install again, and discovering the pain of hard coded paths and command line builds of Eclipse C++ projects – I finally got it running – yay!

Issues

After running the app for a little while it would get into a state where I wasn’t publishing any more messages. I added some extra debug logging which indicated that my publishing flag was not getting set back to false. The callback is just a logging a message and setting the variable to false – cant be that. So some changes …

  • I changed the code to make sure I only attempted to publish one message at a time – no idea if you can have more than one outstanding at any given time, but to simplify the issues just have one – nope still problems.
  • I changed the code again to only attempt publishing messages on the main thread – I was publishing the next message using the callback thread, maybe the library didn’t like this tactic? Nope still problems.
  • I changed the code again to use an atomic<bool> thinking inter-thread issues – nope, still problems.

<hair pulling> – Arrg – It can’t be that hard – the code is really simple!

Possible Problem

The problem appears to thread related, and in particular the order in which I set the publishing flag in the main thread, a slight change and all seems to work again.

The code below shows the line I moved.

You can see that, as it was, if the callback function was called before the main thread had a chance to set the publishing flag the sequence would be

main thread(false)->callback(false)->main thread(true)

instead of my intended sequence

main thread(false)->main thread(true)->callback(false).

 

 

MQTT CAN Gateway

Setup

I mentioned in an earlier post that my CAN Input modules now supported analog input mode on the first 8 inputs. I showed how I could use these analog inputs with NTC temperature sensors to take readings on a wired network rather than wireless.

The next step was to write a gateway that would take these analog readings off the CAN bus and send them to MQTT in order for the existing perl script to pick them up and mash them into RRD databases – as it already does for the radio sensor data.

Other earlier posts provided some details and results while writing the SMS MQTT gateway. Its no surprise I mostly just copied the code for that interface and used it as a starting point.

The only major changes were

  • Removing the SMS specific code and replacing it with the CAN device equivalent.
  • Adding publish ability to my MqttDataProvider class
  • Hard coded decoding of CAN messages into MQTT topics/content publish calls inside this interface. The topic used is in this case is “/sensors/can/<bus id>/<id>/temperature”

In theory (perhaps I’ll do this later) this gateway should just publish all incoming messages into a single topic (e.g. “/messages/can/incoming”) that other “decoder” applications can subscribe to, decode any messages they know how to and then publish new messages to new topics (e.g. “/sensors/can/1/1/temperature”). 

Test

Anyway, with those changes done its time to test. Here we see log showing CAN interface messages coming in.

CAN MQTT log

Following each CAN messages we see an MQTT publish success – in this case there have been 21,100+ published messages since start up.

Results

So CAN to MQTT is publishing, but how can we see the data – easy enough. The old perl script just needed a new subscribe string to take into account the data coming from can and radio, so here it is. Notice the “+/+/+” replacing the old “radio/+/+”

I still have to maunally create the RRD databases and add the extra graph generation script entries but its working.

The infrastructure needed to get CAN messages into MQTT is working, ready for something perhaps a bit more useful.

 

 

MQTT SMS Gateway Part 2

I cleaned up the code, mostly re-factoring the code for INI file configuration by removing the boost program_options version and substituting my own. Works out a lot simpler. Here is the relevant INI file sections –

Time for testing – with the SIM card installed this time. Here is the new log, with a few added bits for debugging – name of the serial port, test mode status, etc.

I sent two messages out to different numbers and all is working as expected.

Second Test Run

Next up – I only have one SIM card so I need to figure out a way for the alarm panel interface to publish MQTT messages.

MQTT SMS Gateway

A while back I bought one of these quad band GPRS modems for a job I was working on. The task was simple enough – connect to a data source, wait for outgoing messages, send them out via SMS, report back with success/fail.

The software was installed and “presumably” all is working as expected – as is the usual case you don’t typically hear any feedback if it works, only if it doesn’t work.

Anyway I have this module gathering dust at home and thought I’d write an MQTT to SMS gateway that can run on my Linux server. Once its up and running any other MQTT application could use it to send SMS notifications very simply.

Easy mode would have been to just use the C# code I already had modify to source data from MQTT and run it via Mono on the Linux box. I could have used the Java version in a similar fashion. However the final plan was to run the application on a Raspberry Pi and having the Mono/Java run times really kills you from a memory usage perspective.

I still have plenty of spare time on my hands so I decided to port the application to C++ and make use of the boost libraries. In theory I can compile it to a single executable (+ the shared libraries I guess) and all will be good when running a dozen or so of these little applications.

All GSM Modules are not the same

There are so many of these things around these days. The standard one the Arduino community seems to use is is based on the SIM900 modules. We used one of these in our Alarm Panel device.

It’s really easy to send SMS messages as the library code will do it all for you. The AT commands are simple enough if you want to do it yourself though, as the SIM900 supports “text mode”, so something similar to this.

The module I have contains a HUAWEI MG323B module. Unfortunately it does not support the same “text mode” to send SMS messages, instead you need to use “PDU mode”, something more like this.

Ugly! There are many useful sites on the interwebs that explain PDU mode in great detail. I found a simple library to do this and ported it to C#, Java and now C++ (cant remember the original source or implementation language at the moment – bad).

Boost me up

I haven’t used or looked at the boost libraries for a (very) long time. But a lot of the things you might want to do are pretty much covered so why reinvent. In particular I needed threads, logging, locking schemes, configuration files and serial ports to name a few.

How did it work out? Well mostly fine – once you get going with how each of the libraries work you can get some nice things done quite quickly.

The good – The threads, logging, synchronization schemes and serial port code was reasonably straight forward.

The not so good – configuration files. Here I wanted to reuse the same INI files my windows applications use. A single INI file contains an assortment of settings for system wide settings and for specific applications and components used within these applications.

I might port my configuration file code to C++ just to maintain the simplicity – the boost method just seemed too complex for my liking.

Initial Testing

Here is the log of the application running. Basic sequence is start up, read configuration file, create and start the MQTT interface and F1103 interface, connect everything all up with an event queue. MQTT subscribes to the topic, while F1103 starts sending AT commands to get the GPRS device into the correct mode/state.

NOTE: The AT commands are echoed back so the first RX line is actually what I sent.

Later I publish an MQTT message to the topic via the command line.

F1103 tries to initiate the SMS send but fails – thee error indicates that the SIM card is not present.

Next up – Add SIM card and test again.

Test Run