Using Node-RED to extract Telldus Live data to MQTT

Moderator: Telldus

Post Reply
mountaindude
Posts: 25
Joined: Thu Jan 18, 2018 11:08 am

Using Node-RED to extract Telldus Live data to MQTT

Post by mountaindude » Thu Jan 18, 2018 11:08 am

The subject says it all...

Before the summer I pulled various pieces together and ended up with something rather nice.
Using Node-RED I connect to Telldus Live, from there downloading all device (i.e. switches etc) and sensor data. Worth noting that what you get is not only (for example) the temperature for the sensor, but also battery status, timestamp for last data received from the sensor etc.
Lots of useful information in there.

Once the data has arrived in the Node-RED function (which is just Javascript, so it will work equally well in a standalone Node.js application), MQTT messages are published, with one topic for each device/sensor attribute.
The beauty of this is that MQTT is absolutely AWESOME for machine-to-machine communication.
Other parts of your Node-RED workflow can then pick up the data for the sensors or devices they are interested, and build cool/useful features on top of that data.

Sending a frost alert over Twitter becomes trivial, for example.

Also, there is nothing stopping other apps (non Node-RED) from subscribing to the MQTT topics and act on them once they arrive. You could for example have a small Python app that listens to certain MQTT topics and store whatever data that comes along into a SQL database, for later visualisation.
Very flexible and way easier to maintain than creating a bunch of small apps that each pull out different subsets of the Telldus data.

More info at https://www.ptarmiganlabs.com/2015/08/3 ... a-to-mqtt/

/Göran

mountaindude
Posts: 25
Joined: Thu Jan 18, 2018 11:08 am

Re: Using Node-RED to extract Telldus Live data to MQTT

Post by mountaindude » Thu Jan 18, 2018 11:08 am

Another small piece of code that simplifies your Node-RED and Telldus Live work:

Over and over again I found myself digging through the Telldus Live web interface to find the id of different remote switches/devices.
There has to be an easier way... Node-RED once again to the rescue!

A small function node will get all device names and ids, then output one message for each device. Just connect it to a debug node and voila!

Code and links available at
https://www.ptarmiganlabs.com/2015/09/0 ... -node-red/

Nautilus
Posts: 80
Joined: Thu Jan 18, 2018 11:08 am

Re: Using Node-RED to extract Telldus Live data to MQTT

Post by Nautilus » Thu Jan 18, 2018 11:08 am

Hi,

this is very interesting as I've just acquired RPi2 and I'm now running a Domoticz image (https://www.domoticz.com/wiki/Domoticz_ ... i_SD_Image) that contains NodeJS, Node-RED and MQTT broker (Mosquito). What I had not figured out yet was how to 1) use sensor / device data from Telldus Live and 2) possibily control the devices in Telldus live. Now it seems that at least the first point is achievable with your bridge :D

This being my first experience with RPi, Domoticz, NodeJS, Node-RED and MQTT I must admit I'm a bit overwhelmed with all the new info. So far I've just checked that I can access the Node-RED GUI and installed the Telldus-Live module (https://github.com/TheThingSystem/node-telldus-live). I also have the API keys from api.telldus.com. So I believe I have all prerequisites in place. But how should I go ahead and create some example flow to Node-RED (using the bridge)?

I've checked e.g. http://nodered.org/docs/getting-started/first-flow.html and assume that I should create some inject node (for triggering, e.g. button or use time interval), then use tellduslive-to-mqtt-bridge.js as function node (with added keys and mqtt IP address). But if I would e.g. like to publish a specific sensor value to Domoticz how would I be able to define the output of the function module? I guess I'd need to use something like this (https://www.domoticz.com/wiki/Flows):
Republish simple MQTT message to domoticz/out/[devicename] topic
This flow will republish the JSON-message from Domoticz to mqtt-topic domoticz/out/[devicename]

Code: Select all

[{"id":"86f0c28c.790f4","type":"mqtt-broker","broker":"localhost","port":"1883","clientid":""},{"id":"9b1a71c4.64e59","type":"switch","name":"Republish switch","property":"payload.idx","rules":[{"t":"eq","v":"8"},{"t":"else"}],"checkall":"false","outputs":2,"x":324,"y":283,"z":"8b1c06b3.74e3f8","wires":[["1273e4ad.ed8c1b","716fa260.8e905c"],[]]},{"id":"e53cf059.1ac31","type":"mqtt in","name":"","topic":"domoticz/out","broker":"86f0c28c.790f4","x":75,"y":296,"z":"8b1c06b3.74e3f8","wires":[["9e7ba289.61846"]]},{"id":"1273e4ad.ed8c1b","type":"debug","name":"","active":true,"console":"false","complete":"payload","x":724,"y":273,"z":"8b1c06b3.74e3f8","wires":[]},{"id":"9e7ba289.61846","type":"json","name":"","x":190,"y":338,"z":"8b1c06b3.74e3f8","wires":[["9b1a71c4.64e59"]]},{"id":"bfed3b30.4012c8","type":"mqtt out","name":"","topic":"","qos":"","retain":"","broker":"86f0c28c.790f4","x":708,"y":391,"z":"8b1c06b3.74e3f8","wires":[]},{"id":"716fa260.8e905c","type":"function","name":"Repub","func":"msg.topic = \"domoticz/out/device/\"+msg.payload.name;\n\nif (msg.payload.nvalue == 1) { \n    msg.payload = \"ON\"} \nelse\n{\n    msg.payload = \"OFF\"\n}\n\nreturn msg;","outputs":1,"x":509,"y":326,"z":"8b1c06b3.74e3f8","wires":[["bfed3b30.4012c8","1273e4ad.ed8c1b"]]},{"id":"4c7fbac3.b38044","type":"comment","name":"Instructions","info":"In this example the IDX in the switch node is \nrepublished to a topic named domoticz/out/[devicename]\nthe repub function translate value 1 to ON","x":73,"y":257,"z":"8b1c06b3.74e3f8","wires":[]}]
Well, maybe some trial and error is needed. But does it seem like I'm even going to a right direction...? :) Also any examples from your setup would be more than welcome to make things a bit more concrete for a noob like me :)

Nautilus
Posts: 80
Joined: Thu Jan 18, 2018 11:08 am

Re: Using Node-RED to extract Telldus Live data to MQTT

Post by Nautilus » Thu Jan 18, 2018 11:08 am

Ok, I've made some test and it seems that at least something is happening :)

If I run the device listing I get the devices to the debug window. But then if I try the bridge function it runs but quite soon I get "Error: lost connection to server" in Node-RED GUI. I tried to dig some logs and found /var/logs/node-red.log where last lines were:

Code: Select all

14 Sep 00:06:45 - [red] Uncaught Exception:
14 Sep 00:06:45 - TypeError: Cannot read property 'statusCode' of undefined
    at /usr/lib/node_modules/node-red/node_modules/telldus-live/telldus-live.js:221:47
    at ClientRequest.<anonymous> (/usr/lib/node_modules/node-red/node_modules/oauth/lib/oauth.js:421:9)
    at ClientRequest.emit (events.js:107:17)
    at TLSSocket.socketErrorListener (_http_client.js:272:9)
    at TLSSocket.emit (events.js:129:20)
    at net.js:451:14
    at process._tickCallback (node.js:355:11)
Any idea what this is about? Before that, there were some valid looking data, e.g:

Code: Select all

sensor #5 getSensorInfo: 
{ id: 'XXXXXX',
  clientName: 'XXXX',
  name: 'XXXX',
  lastUpdated: 1442178273,
  ignored: 0,
  editable: 1,
  data: 
   [ { name: 'temp', value: '23.6', scale: '0' },
     { name: 'humidity', value: '0', scale: '0' } ],
  protocol: 'mandolyn',
  sensorId: '12',
  timezoneoffset: 10800,
  battery: '253',
  keepHistory: '0' }
/device/climate/mandolyn/meteo
    uuid=teldus:XXXXXXX
    name: XXXXX
    status: present
    battery: 253
    lastSample: 1442178273000
    lastSample: 2015-09-14 00:04:33
    info:
      temperature: "celcius"
      humidity: "percentage"
    values:
      temperature: 23.6
      humidity: 0

Nautilus
Posts: 80
Joined: Thu Jan 18, 2018 11:08 am

Re: Using Node-RED to extract Telldus Live data to MQTT

Post by Nautilus » Thu Jan 18, 2018 11:08 am

Forgot to mention, I've got something about 100 devices and 10 sensor in my Live account, could it be that this has something to do with the crash. There we're quite a few references to this "Cannot read property 'statusCode' of undefined", e.g. http://stackoverflow.com/questions/2302 ... ashing-app - but I'm not quite sure what to make of it with regards to the telldus-live module...

mountaindude
Posts: 25
Joined: Thu Jan 18, 2018 11:08 am

Re: Using Node-RED to extract Telldus Live data to MQTT

Post by mountaindude » Thu Jan 18, 2018 11:08 am

Don't see how # of devices/sensors would affect things... But there could be corner cases where things time out, I guess.
The code I posted could probably also benefit from better error handling, that should make it more robust and make it throw better/more understandable errors.

I've had way too much to do lately to work active on these things, so those improvements will have to wait a bit, sorry... :/

Nautilus
Posts: 80
Joined: Thu Jan 18, 2018 11:08 am

Re: Using Node-RED to extract Telldus Live data to MQTT

Post by Nautilus » Thu Jan 18, 2018 11:08 am

Ok, and thanks for the reply! :)

Googling the errors gave these references:
http://stackoverflow.com/questions/2302 ... ashing-app
http://stackoverflow.com/questions/2975 ... -undefined

Basically, if response is undefined, due to a timeout, the response.StatusCode cannot be read - which then crashes the node app. Maybe I'll test the proposed "fix" to telldus-live.js line 221
if (!!err) return callback(err, response.statusCode);
I guess that is what causes it in the end and not your code... Have to try to check through the sensors / devices if there are some characters or something that could cause issues...

Nautilus
Posts: 80
Joined: Thu Jan 18, 2018 11:08 am

Re: Using Node-RED to extract Telldus Live data to MQTT

Post by Nautilus » Thu Jan 18, 2018 11:08 am

If someone else is struggling with this I'll just add here that in the end I just removed the lines for error logging to console.log (well, actually all console.log line as there was a huge amount of log coming from this due to the amount of devices and sensors) and it has now been stable for a week or so.

The flow is quite simple. I have a time-interval based inject node to execute the modified function and then an mqtt subscription node per topic with a couple of lines parsing to convert the payload into Domoticz-acceptable format which all end up to mqtt domoticz/in topic (this is a limitation at Domoticz end, it subscribes to only this topic and the payload needs to include device id + values).

Nautilus
Posts: 80
Joined: Thu Jan 18, 2018 11:08 am

Re: Using Node-RED to extract Telldus Live data to MQTT

Post by Nautilus » Thu Jan 18, 2018 11:08 am

I would like to expand the solution into controlling the devices but it proved to be a bit tricky again...:) Any pointers on how to use e.g. the function cloud.onOffDevice(device, onP, function(err, result) ? I was not able to get it work and there was no reference to this in the test.js file at node-telldus-live either.

petzno
Posts: 116
Joined: Thu Jan 18, 2018 11:08 am

Re: Using Node-RED to extract Telldus Live data to MQTT

Post by petzno » Thu Jan 18, 2018 11:08 am

This is a great function for node-red and I got it working. However I struggle to see how I can get this data into domoticz? I have setup the MQTT in domoticz and it is subscribed but I'm not sure where to go from there. I have to say that both MQTT and node-red is new to me, so any advise would be highly appreciated.

Nautilus
Posts: 80
Joined: Thu Jan 18, 2018 11:08 am

Re: Using Node-RED to extract Telldus Live data to MQTT

Post by Nautilus » Thu Jan 18, 2018 11:08 am

petzno wrote:This is a great function for node-red and I got it working. However I struggle to see how I can get this data into domoticz? I have setup the MQTT in domoticz and it is subscribed but I'm not sure where to go from there. I have to say that both MQTT and node-red is new to me, so any advise would be highly appreciated.
Hi, I'm completely new to this so this might not be a wise way to do it, but at least it works... :)

What I've done is that I've subscribed to each topic whose "payload" (= temp or temp + humidity data) I want to publish into Domoticz. E.g. lets say I have sensor name Temp1 -> I would subscribe to (= mqtt input node in node-red):

Code: Select all

common/telldus live/[Name of location]/sensors/Temp1/data/temperature/value
The payload of each received message will hold the value for sensor Temp1. This is then linked to a function node (let's say IDX of Temp1 in Domoticz is 14):

Code: Select all

var temp = msg.payload;
msg.payload = {};
msg.payload.svalue = temp;
msg.payload.idx = 14;
return msg;
And finally this flow is linked to mqtt ouput node domoticz/in, the message being in format:

Code: Select all

{"svalue":"23.6","idx":14}
To include humidity, I needed to add a few lines to the telldus live bridge function to publish one topic for both temp and humidity:

Code: Select all

if (sensor.data[1]) {
     mqttClient.publish('common/telldus live/' + sensor.clientName + '/sensors/' + sensor.name + '/data/temphum/value', sensor.data[0].value + ";" + sensor.data[1].value);
}
This will ouput something like "22.8;42" but for these type of sensors in Domoticz the format needs to be:

Code: Select all

{"svalue":"22.8;42;0","idx":33}
The last value is humidity status (HUM_STAT, 0=Normal, 1=Comfortable, 2=Dry, 3=Wet) which is not supported in Telldus so I've fixed it to 0 with the function node:

Code: Select all

var temp = msg.payload;
msg.payload = {};
msg.payload.svalue = temp + ";0";
msg.payload.idx = 25;
return msg;
Hope this helps... :D

Nautilus
Posts: 80
Joined: Thu Jan 18, 2018 11:08 am

Re: Using Node-RED to extract Telldus Live data to MQTT

Post by Nautilus » Thu Jan 18, 2018 11:08 am

Nautilus wrote:I would like to expand the solution into controlling the devices but it proved to be a bit tricky again...:) Any pointers on how to use e.g. the function cloud.onOffDevice(device, onP, function(err, result) ? I was not able to get it work and there was no reference to this in the test.js file at node-telldus-live either.
Decided to use the python example provided by Telldus to do the actual controlling and just read the switch status with the Mqtt Telldus Bridge.

Andersson8
Posts: 21
Joined: Thu Jan 18, 2018 11:08 am

Re: Using Node-RED to extract Telldus Live data to MQTT

Post by Andersson8 » Thu Jan 18, 2018 11:08 am

Anyone who can make a short description of how to use this? Really new to this kind o programming. Installed node-red, domestics, mqtt, telldus keys... on a Raspberry but then I don't really understand what to do. Is the .js files supposed to be a function block in node red? what would the input be?

Naesstrom
Posts: 4
Joined: Thu Jan 18, 2018 11:08 am

Re: Using Node-RED to extract Telldus Live data to MQTT

Post by Naesstrom » Mon May 14, 2018 12:06 pm

I followed the guide in the Blog and github and have come so far that it can list my switches but no sensors... Tried to make a new function node with the mqtt publish but payload seems to be empty, do I need to enter somewhere in it what module I want to publish or does it publish everything?

Naesstrom
Posts: 4
Joined: Thu Jan 18, 2018 11:08 am

Re: Using Node-RED to extract Telldus Live data to MQTT

Post by Naesstrom » Mon May 14, 2018 12:07 pm

Andersson8 wrote:
Thu Jan 18, 2018 11:08 am
Anyone who can make a short description of how to use this? Really new to this kind o programming. Installed node-red, domestics, mqtt, telldus keys... on a Raspberry but then I don't really understand what to do. Is the .js files supposed to be a function block in node red? what would the input be?
As input you can use an inject node and set the time to 1 minute or anything you want.

Post Reply