TellStick Net sensor update problem

[Urokhtor]

Hello, I am writing a webUI/automation server for a friend of mine which handles logging and displaying sensor data, controlling devices, scheduling tasks etc. I'm interfacing with Telldus API with a Python server using OAuth as told in the documentation. Everything is working well except that we're experiencing problems getting updated data with sensor/info request.

The sensor/info response has fields "lastUpdated" and "data", the latter which hold the sensor's last reading. The problem is that with some sensors, while Telldus' own Live! interface tells the sensor was lastUpdated a minute ago, when the Python server makes a request it receives a response with lastUpdated lagging even over 10 minutes behind the timestamp Telldus' own interface states. Normally this would not be much of a bother, but my friend uses my software and TellStick Net to automate his garden and some controls he's planning to add are very time dependent, so getting data 10 minutes behind the actual time Telldus Live! gives is a big problem for the system's functionality.

Additional info:
The configuration might be a bit peculiar. There are two TellStick Net devices on his property. He bought an another one because sensors in his greenhouse weren't updating properly. The original TellStick Net did recognise them. When he added the second one, both recognised his sensors but he assigned the sensors between those two TellStick Net devices so that one sensor was governed only by one TellStick Net device.

So, when the Python server polls sensor/info for all the sensors that are in the system, the sensors that are governed by the original TellStick Net get the most recent lastUpdated timestamp, which is also seen in Telldus Live! But the sensors governed by the new TellStick Net don't get the most recent lastUpdated timestamp and the timestamp might state lastUpdated was taken 11:20 while Telldus Live! states it was taken 11:28.

So, might the problem be with both TellStick Net devices being in the same area and interfering with each other? Or is the Telldus API server under heavy load and that's why it updates slowly? Any other ideas? I've tried to debug this issue and even manually checked the timestamps for sensor/info responses and they did give old data, so the problem should not be my software not polling the API fast enough. Any ideas on what could be done to resolve this issue, if anything?

And please do ask if I did not provide some important information in this post or it's otherwise incomprehensible.

[micke.prag]

Hi,

You are correct in your observations. Let me explain and answer your questions.

The problem is that with some sensors, while Telldus' own Live! interface tells the sensor was lastUpdated a minute ago, when the Python server makes a request it receives a response with lastUpdated lagging even over 10 minutes behind the timestamp Telldus' own interface states.

The system handles a huge load of sensor readings. We are processing hundreds of million readings each month. This creates a big load on all servers. For each of these sensor reading we get a poll from third party application. To limit this we are experimenting the actual storage of the sensors only each 10 minutes, but in the mean time pushing the current value directly to the interface. This is why the API might get behind in the values.

The configuration might be a bit peculiar. There are two TellStick Net devices on his property. He bought an another one because sensors in his greenhouse weren't updating properly. The original TellStick Net did recognise them. When he added the second one, both recognised his sensors but he assigned the sensors between those two TellStick Net devices so that one sensor was governed only by one TellStick Net device.

Correct. Each sensor is tied to the TellStick Net. It is not possible for one TellStick Net to update the sensor from another TellStick Net.

So, might the problem be with both TellStick Net devices being in the same area and interfering with each other? Or is the Telldus API server under heavy load and that's why it updates slowly?

No and no. They are not interfering with each other. This is intentional behaviour.

Normally this would not be much of a bother, but my friend uses my software and TellStick Net to automate his garden and some controls he's planning to add are very time dependent, so getting data 10 minutes behind the actual time Telldus Live! gives is a big problem for the system's functionality.

I am curious about your system. Could you please explain it in more details? It is crucial for us to know how the system is used IRL so we can build the system as good as possible.

You should also note that how the sensor values are stored is not final. We are experimenting different techniques and you might experience different results depending on which server your TellStick Net is assiged to.
This is also some things we are evaluating:
* A complete different storage engine to allow faster updates but slower reads.
* Pushing new values directly to third party applications
* A paid service to offer higher resolution of sensor readings
* An events system that will allow you to trigger actions on temperature readings.
* APIs for query sensor history

[Urokhtor]

Thank you very much for a fast response. What you say about storing sensors makes sense. However things got pretty interesting. My friend wondered the situation and went and opened the TellStick box and found out the antenna was soldered in pretty badly. But after he opened the device, it started giving out up-to-date sensor data just like its sister box. Sounds weird, but for the time being things have worked fluently.

About the system:
It's actually a modified version of my thesis project, which consists of an automation server running on a computer, controlling potentially multiple Arduino devices that use Ethernet. In this version under discussion the Arduino layer is replaced with your Telldus API.

The system itself stores lists of Telldus devices and sensors in the system. It lets the user control the devices (on/off/dim/bell) through the webUI. Sensors are displayed on the page along with graphs showing their 24 hour and 7 day readings. The sensor data is fetched with sensor/info request from Telldus API and logged down with the minute interval specified by the user. And the graphs are generated from this data.

It also features tasks, which lets the user define which action should control which device at which times, for example "turn on device [id] at 15:00". Moreover it is possible to use sensors to control devices. The user can define for example "if sensor [id] value is over [value], turn on device, if the value is less than [value], turn off device".

This is a vital thing when controlling a big greenhouse (this person I'm coding this for is a major chilehead) because sometimes devices need to respond rapidly to changes in temperature for example. Because of this you might see why getting the latest sensor readings from the Telldus API is so important. But like I stated in the beginning, after opening the box, the API requests started giving out the same data you could see on the Telldus Live! page which makes me wonder because I thought the sensor data was stored in your servers.

Nevertheless, we must monitor the situation. In overall I must say I'm really content with your service. It makes home automation so simple and easy.

I'll keep in mind your push to offer a better logging system. I'll need to adapt the system once those possible changes happen.

Hope this post clarified the functioning of my system a bit.

[micke.prag]

But after he opened the device, it started giving out up-to-date sensor data just like its sister box.

Yes, as I wrote above it depends on the server. Currently its about 50/50 chance which type of server you end up on.

The user can define for example "if sensor [id] value is over [value], turn on device, if the value is less than [value], turn off device".

This will work in our upcoming events system.

This is a vital thing when controlling a big greenhouse (this person I'm coding this for is a major chilehead) because sometimes devices need to respond rapidly to changes in temperature for example.

Most temperatures are so slow that one reading every 10 minutes doesn't change much. This is actually the first request we got for faster updates. I am just thinking. If the temperature changes so much in 10 minutes, isn't something wrong then?

[Urokhtor]

Most temperatures are so slow that one reading every 10 minutes doesn't change much. This is actually the first request we got for faster updates. I am just thinking. If the temperature changes so much in 10 minutes, isn't something wrong then?

I agree that temperature changes are often pretty slow and a longer update interval should be OK. With this particular case there's for example a sensor sniffing when to start a water circulation pump. My friend told it's quite critical to not let the water heat up too much or otherwise the pipes will get stuck. I don't know the exact way this system functions so I can only trust his view on this. Other good example is using a sensor as a thermostat for keeping seed germination temperatures constant.

This will work in our upcoming events system.

Nice to hear you're getting this feature in your system, I myself like building these systems on my own, but for a lot of people this can be a great tool.

[sean]

Glad I found this post, I was wondering why the updates via API (using RemoteStick) were slow. The 10 minute value will explain that!

I currently have Tasker and RemoteStick controlling an air conditioning device. Currently the device is staying on or off too long as the temperature goes up or down, faster updates would be useful in this situation.

Ideally what I'd love to see is tellsticks own events system, with a probably not planned twist.

I'd like to be able to set it to something like:

If temp >24c then turn on Nexa socket for 1 second, then turn off.
If temp <23c then turn on Nexa socket for 1 second, then turn off.

[lioelec]

I'd like to be able to set it to something like:

If temp >24c then turn on Nexa socket for 1 second, then turn off.
If temp <23c then turn on Nexa socket for 1 second, then turn off.

You can do that with an other "server Live", you just loggin and define your teperature events >> http://www.templight.se look here http://www.telldus.com/forum/viewtopic.php?f=11&t=2926

[joemensor]

I'd like to be able to set it to something like:

If temp >24c then turn on Nexa socket for 1 second, then turn off.
If temp <23c then turn on Nexa socket for 1 second, then turn off.

You can do that with an other "server Live", you just loggin and define your teperature events >> http://www.templight.se look here http://www.telldus.com/forum/viewtopic.php?f=11&t=2926

This is great but unfortunately does not allow you to monitor proximity sensors etc..

[frode.forsberg]

Micke: One of my sensors does not send updates of temperature to Tellstick frequently enough. Product is a Capidi RAR801 termometer bought from Clas Ohlson. The device have two sensors, and the readings in the display is accurate, but only one of the sensors show up in Telldus Live with the up to date values. Sensor nr. 1 updates every 10 minutes or so. Sensor nr. 2 has up to 16 HOURS!! delay. Any solutions to this fault? Both sensors are placed inside, one in the kitchen and the other one in my living room. Distance to Tellstick is approx 5 meters from both sensors.
Cheers!