Wireless Energy Meter Plugin - BETA

[KHolm]

Hi,

Edit: Updated to version 0.9 Beta (Version 0.5 is in Automagicallys latest update)

This is the first BETA release of the Wireless Energy Meter Plugin for Automagically.
Copy the file (wem.py) to your plugin directory and turn on your Wireless Energy Meter from http://foogadgets.tictail.com.

When you have turned on your Wireless Energy Meter, you need to look into the signal trace to find the ID of the Primary and Secondary channel.
It will look something like this:

Code: Select all

tellstick,raw:class:sensor;protocol:fineoffset;id:69;model:temperaturehumidity;humidity:6;temp:0.1; 	
tellstick,sensor,protocol:fineoffset,id:69,Temperature,value:0.1 	
tellstick,sensor,protocol:fineoffset,id:69,Humidity,value:6 	
tellstick,raw:class:sensor;protocol:fineoffset;id:68;model:temperaturehumidity;humidity:7;temp:0.1; 	
tellstick,sensor,protocol:fineoffset,id:68,Temperature,value:0.1 	
tellstick,sensor,protocol:fineoffset,id:68,Humidity,value:7 	

The temperaturehumidity 'sensor' with the highest number of humidity will be your primary device, in the above trace id:68.
The temperaturehumidity 'sensor' with the lowest number of humidity will be your secondary device, in the above trace id:69.

The Wireless Energy Meter Plugin must then be configured (/admin/settings/settingsvalue/) with the above identified primary and secondary device.
The Wireless Energy Meter Plugin takes the following configurations:

• Enabled - Set to True to enable the plugin
• ManualMode - Do not touch, for debugging only. Disables the timeout functionality for packet loss detection. To perform a 'manual' timeout, send: wem,action:timeout
• primaryId - The ID of the primary channel
• secondaryId - The ID of the secondary channel
• blinksPerWatt - Number of blinks per kW, typically 1000 or 10000 blinks per kW in major brands.

When the Wireless Energy Meter Plugin has found packages from a Wireless Energy Meter Plugin it will send the following signal.
The plugin will recover from lost packages by having implemented (a sort of) interleaving function provided by the Wireless Energy Meter's protocol.

Code: Select all

wem,data;status:NN;datetime:NN;seq:NN;energy:NN;packetloss:NN;

The above signal contains the following data:

• status = OK or TO_MANY_LOST_PACKAGES
• datetime = Time the signal was received in Automagically
• seq = The received sequence number
• energy = The energy measured since last report
• packetloss = Indication if there was packet-loss

Example:

Code: Select all

wem,data;status:OK;datetime:2014-02-26T21:33:52.274328;seq:18;energy:319;packetloss:0;
wem,data;status:TO_MANY_LOST_PACKAGES

The following Signal Pattern to match will work with the Wireless Energy Meter Plugin:

Code: Select all

wem,data;status:%s;datetime:%s;seq:%d;energy:%d;packetloss:%d;

With this pattern, use the following Parsed variable to store // Variable in Send the following signal(s) on match:

• $1 - status
• $2 - datetime
• $3 - seq
• $4 - energy
• $5 - packetloss

This transform will feed Xively nicely (if you have xively plugin):
Pattern to match: wem,data;status:%s;datetime:%s;seq:%d;energy:%d;packetloss:%d;
Send the following signal(s) on match: xively,report:Energi:$4:W;
For info in Xively plugin, see http://automagically.wikia.com/wiki/Usi ... ely_plugin

The plugin has been tested in live and simulated environment.
In simulated environment, the following tests has been successfully performed:

• Dropping a package on primary channel
• Dropping a package on secondary channel
• Dropping a package on both primary and secondary at the same time
• Dropping two primary packages and one secondary package
• Dropping one primary packages and two secondary package
• Dropping two primary packages and two secondary package
• Wrapping of sequence numbers (99->0)

Improvements to be considered

• The total Energy consumption can be calculated by summing up the incoming data (Etotal = Etotal + temp*10/1000)
• The Power usage can also be calculated (Pmom = temp*10/1000*60*60/(Time_now - Time_previous))

Other improvements
Please suggest...

Simulator
Attached is also as simulator that behaves as a Wireless Energy Meter, if you want to test things without actually having one.
It has been described in this post earlier: http://www.telldus.com/forum/viewtopic. ... 002#p24995

To make it easier to work with the build a remote that uses a few different Send signal devicees.
It can look something like this:

/Marcus

[misterleffe]

This looks nice! But my skills in Linux is on a extremly low level ...

Now I've downloaded the file and then Copy the file (wem.py) to your plugin directory ... Can you please supply me (and others) with a nice command for this? (I'll have to crawl before i can walk...)
How to move it from my Win7 HDD to Pi SD-card?


/misterleffe

[jimmy927]

This looks nice! But my skills in Linux is on a extremly low level ...

Now I've downloaded the file and then Copy the file (wem.py) to your plugin directory ... Can you please supply me (and others) with a nice command for this? (I'll have to crawl before i can walk...)
How to move it from my Win7 HDD to Pi SD-card?

Alternative a) download the file to your desktop and then use winscp or similar to transfer it to your Linux-box ( http://winscp.net/eng/download.php )
Alternative b) just install wget on your Linux-box and run "wget http://my-cool-url" it will then download the file

[KHolm]

Alternative b) just install wget on your Linux-box and run "wget http://my-cool-url" it will then download the file

Code: Select all

$ sudo apt-get install wget
$ sudo apt-get install unzip
$ cd ~/downloads/
$ wget http://www.telldus.com/forum/download/file.php?id=430 -O wem.zip
$ cd ~/source/automagically/site/plugins
$ unzip ~/downloads/wem.zip
$ sudo service automagically stop
$ sudo service automagically start

/Marcus

[misterleffe]

Thanks for the hint about the Vinscp filemanager. No problem when "how to". Maybe I will learn with some practice!

Now installed the plugin for the Wireless Energy Meter and configured Automagically and it seems to count incoming pulses. Also sending it to Xively and the graph is nicely coming up. But this is just the momentary value. How do I implement the TOTAL energy in kWh? It would be nice to have both momentary energy consumption and the total consuption in to different graphs. Should this be done in the Signal Transform as well?

What is your idea Marcus, to implement this feature? Nice if you can make an easy tutorial. By the way, nice work! And very quick solution!! Maybe there will be some suggestions for improvement later on, but so far it seems to work well.



/misterleffe

[KHolm]

How do I implement the TOTAL energy in kWh? It would be nice to have both momentary energy consumption and the total consuption in to different graphs. Should this be done in the Signal Transform as well?

Here's how to achieve what you ask for with the signal handler.

1. Create a new variable of type float called accumulatedEnergy

2. Create a Store global variables signal handler
Pattern to match: accumulatedEnergy:%f
Parsed variable to store: 1
Variable to update: accumulatedEnergy

3. Create a signal transform
Pattern to match: wem,data;status:%s;datetime:%s;seq:%d;energy:%d;packetloss:%d;
Send the following signal(s) on match: $EVAL:"accumulatedEnergy:" + str($accumulatedEnergy$+(float($4)/1000.0))

Now you have accumulated energy in kW in the variable accumulatedEnergy, to be used in remotes or whatever you want.

Next is to figure out when you want to "reset" this. Every hour, on the hour? Every midnight?

4. To do a reset every 60 min, create a Data fetchers (/admin/core/datafetcher/)
Name: Echo
FetchType: Execute
FetchInterval: 60
Executable: /bin/echo 0

5. Report to xively
Pattern to match: datafetcher,4,Echo,fetched:%s
Send the following signal(s) on match: $EVAL:"xively,report:Energy_per_hour:" + str($accumulatedEnergy$) + ":kW;"

6. Then use a Signal Store global variables to reset (acting on the Xively report)
It might be possible to trigger on the "echo", but then you have a race situation.
Pattern to match: xively,report:Energy_per_hour:%s
Variable to update: accumulatedEnergy
Parsed variable to store: 1
Store this constant instead of parsed variable: 0.0

I don't know at this point how to perform a reset at midnight. Should be possible I guess...otherwise a new plugin might be needed.

In one scenario, you might loose a reported value.
- echo runs
- xively report occurs
- a new signal arrives from wem (acc+=energy[Sx])
- reset of acc happens
==> energy[Sx] will not be counted for

/Marcus

[KHolm]

Now installed the plugin for the Wireless Energy Meter and configured Automagically and it seems to count incoming pulses.

So, two systems are live with the ALPHA implementaton, no changes foreseen at this point.
Should we re-classify to BETA and push it into automagicallys source at bitbucket?

Please advice,
/Marcus

[KHolm]

I don't know at this point how to perform a reset at midnight. Should be possible I guess...otherwise a new plugin might be needed.

And that should be rather easy as well.

As per above post, accumulate in a daily variable named accumulatedDayEnergy, possibly based on the xively hourly signal.

N: Use a Send signal device
Name: ResetEvery24
Signal to send at ON/Activate: resetEnergyMeter

N+1: A Scheduled event to send a daily signal.
Device: ResetEvery24
Command: ACTIVATE
RelativeTo: Absolute time
Execution time: 00:00:00
Do the following weekdays: All

N+2. Report to xively
Pattern to match: resetEnergyMeter
Send the following signal(s) on match: $EVAL:"xively,report:Energy_per_day:" + str($accumulatedDayEnergy$) + ":kW;"

N+3. Then use a Signal Store global variables to reset (acting on the Xively report)
It might be possible to trigger on the "echo", but then you have a race situation.
Pattern to match: xively,report:Energy_per_day:%s
Variable to update: accumulatedDayEnergy
Parsed variable to store: 1
Store this constant instead of parsed variable: 0.0

Same possible race as in hourly report:
In one scenario, you might loose a reported value.
- Scheduler executes
- xively report occurs
- a new signal arrives from wem/hourly report (accDay+=energy[Sy])
- reset of acc happens
==> energy[Sy] will not be counted for

/Marcus

[misterleffe]

Now I've implemented the first part of your tutorial. I have it in my remote and I can see it coming up in Xively. But I'll have to wait until I've consumed the first kWh so I can see if it actually works ... But I think it will be presented when the energy has been used! Thanks for your efforts, Marcus!

The point where you said "Next is to figure out when you want to "reset" this. Every hour, on the hour? Every midnight?" What's the meaning of reset the metering? If you reset it for example, every 24h, should the value be stored before reset, so I can get statistics for every day in each month? Then I can have statistics for every month on annual basis? A feature like this would be great, because I want to get statistics for monthly basis, as well as for annual basis. Would this be possible?

Yes, I think the Alpha plugin should be within the Automagically image. To have the features directly in the image is great. If you want to use it's -fine. Otherwise you can just leave it. Same with Xively. Before we leave it to David, the plugin should be tested in all ways. But, what I can explore, it's working like a charm!



/misterleffe

[KHolm]

The point where you said "Next is to figure out when you want to "reset" this. Every hour, on the hour? Every midnight?" What's the meaning of reset the metering? If you reset it for example, every 24h, should the value be stored before reset, so I can get statistics for every day in each month? Then I can have statistics for every month on annual basis? A feature like this would be great, because I want to get statistics for monthly basis, as well as for annual basis. Would this be possible?

Yes, and that what's done in step 4-6. Before the accumulated variable is cleared, it reports to xively to "stored before reset".
Then in step N-N+3 you do the same for your daily consumption. Reports to xively to achive "stored before reset".

So, Xively is where I keep the statistics.

Noted that the followin comment from me is incorrect: accumulate in a daily variable named accumulatedDayEnergy, possibly based on the xively hourly signal. Instead, one must use the wim event to store also daily consumption just like we do with the hourly consumption.

N-3. Create a new variable of type float called accumulatedDayEnergy

N+2. Create a Store global variables signal handler
Pattern to match: accumulatedDayEnergy:%f
Parsed variable to store: 1
Variable to update: accumulatedDayEnergy

N-1. Create a signal transform
Pattern to match: wem,data;status:%s;datetime:%s;seq:%d;energy:%d;packetloss:%d;
Send the following signal(s) on match: $EVAL:"accumulatedDayEnergy:" + str($accumulatedDayEnergy$+(float($4)/1000.0))


To do a monthly report, I still havn't figured out a "perfect" way. What will almost work is a datafetcher that runs every 24 hrs and performs "/bin/date | /usr/bin/tr ' ' ','". Then one look at the 3rd argument (Day of month) and matches 1 to trigger the reset.
Problem is that this will run on a 24 schedule wheni it is started, and we want it to run 00:00:01 on the 1st every monts.
In fact, I'm missing a "Execute linux command device", but I guess one can write a plugin that acts on a signal and performs what one want.

/Marcus

[misterleffe]

Ok, when I read the text more carefully, I noticed what you are telling me. Sometimes I'm to fast and don't read everything. Now I'll have completed the energy measuring in all points. It seems to be counting and the kWh is rising for every minute and it makes me poorer and poorer ... But even lucky to have some control over the situation. (Read "just note that billing from Energy Company is working". Hi Hi !!)

I've noticed one thing about the updating interval. At my remote, the update will come very quickly, but if I have a look at Xively at the same time, there is no update available at that moment. After some minute(s), the update is visible at Xively. Should it be like that? Is it Xively who decide this delay?

"To do a monthly report, I still havn't figured out a "perfect" way. What will almost work is a datafetcher that runs every 24 hrs and performs "/bin/date | /usr/bin/tr ' ' ','". Then one look at the 3rd argument (Day of month) and matches 1 to trigger the reset.
Problem is that this will run on a 24 schedule wheni it is started, and we want it to run 00:00:01 on the 1st every monts.
In fact, I'm missing a "Execute linux command device", but I guess one can write a plugin that acts on a signal and performs what one want."

Marcus, I'm looking forward to this! Will you make a statistic for annual basis as well? If you do that, I think the enery metering is complete!

I will leave a report here when Auotmagically has been up and running for a while with these new settings.



/misterleffe

[KHolm]

I've noticed one thing about the updating interval. At my remote, the update will come very quickly, but if I have a look at Xively at the same time, there is no update available at that moment. After some minute(s), the update is visible at Xively. Should it be like that? Is it Xively who decide this delay?

What you may experience is the 61 sec delay in my algorithm. The result from primary channel is buffered in the selected interleaving algorithm. I assumed that it would be acceptable. This was seen to feasible due to added complexity of the asynchronous nature of Automagically.
Still, there should be very little delays between a Remote Update and Xively - but in that case the delay is in Xively and not in Automagically,

Marcus, I'm looking forward to this! Will you make a statistic for annual basis as well? If you do that, I think the enery metering is complete!

I must admit, this will not be my next priority. Next thing to look at is to feed google spreadsheets from Automagically

/Marcus

[KHolm]

I must admit, this will not be my next priority. Next thing to look at is to feed google spreadsheets from Automagically

/Marcus

I was wrong, a new version of Wireless Energy Meter is attached (and updated in original post).

This version listens to two signals:
wem,action:firstdayofmonth - responds with wem,info;firstdayofmonth:true or wem,info;firstdayofmonth:false
wem,action:firstdayofyear - responds with wem,info;firstdayofyear:true or wem,info;firstdayofyear:false

With this we can now build a daily and yearly reset.

N+4. Create a new variable of type float called accumulatedMonthEnergy

N+5. Create a signal transform
Pattern to match: wem,data;status:%s;datetime:%s;seq:%d;energy:%d;packetloss:%d;
Send the following signal(s) on match: $EVAL:"accumulatedMonthEnergy:" + str($accumulatedMonthEnergy$+(float($4)/1000.0))

N+6: Create a Send signal device that sends a signal
Name: CheckFirstDayOfMonth
Signal to send at ON/Activate: wem,action:firstdayofmonth

N+7: A Scheduled event to send a daily signal.
Device: CheckFirstDayOfMonth
Command: ACTIVATE
RelativeTo: Absolute time
Execution time: 00:00:00
Do the following weekdays: All

N+8. Report to xively in Signal transform
Pattern to match: wem,info;firstdayofmonth:true
Send the following signal(s) on match: $EVAL:"xively,report:Energy_per_month:" + str($accumulatedMonthEnergy$) + ":kW;"

N+9. Then use a Signal Store global variables to reset (acting on the Xively report)
Pattern to match: xively,report:Energy_per_month:%s
Variable to update: accumulatedMonthEnergy
Parsed variable to store: 1
Store this constant instead of parsed variable: 0.0

For yearly accumulation and reset:
Repeat step N+3 to N+9 but use:

• accumulatedYearlyEnergy instead of accumulatedMonthEnergy
• Energy_per_year instead of Energy_per_month
• CheckFirstDayOfYear instead of CheckFirstDayOfMonth
• wem,action:firstdayofyear instead of wem,action:firstdayofmonth
• wem,info;firstdayofyear:true instead of wem,info;firstdayofmonth:true

Mission complete
/Marcus
ps. Unfortunatley the following EVAL did not work, then it had been already working in Automagically.
$EVAL:"whatever" + ("false","true")[time.localtime().tm_mday == 1]

[KHolm]

Version 0.5 now in a pull request to Automagically,
https://bitbucket.org/davka003/automagi ... r-and/diff
/Marcus

[misterleffe]

Lot of work you have done Marcus! Hopefully it will be implemented in Automagically later on.

Now it's a lot of information and just for sort it out ...

Should i delete the previous version of wem in the "plugin directory" and replace it with the attached version? After replacement of the new version(?), can I just continue the settings with the new ones stated N+4 ......N+9?

The settings as it is right now, wouldn't it give "peaks" with resets every hour? I've followed your tutorial point by point. Shouldn't just only the accumulated energy rise all time and just look like "Holmenkollen"? My graph for daily energy looks like just mentioned down hill.

My settings for datafetcher is

-Those you have given in the tutorial +

-Parser type: Line,Character,position,Length
-Parse: Line=0 Char=0 Size=1
-ParsedVariable: 1

Are these settings correct?

Can you attach a clip of your graphs and how it looks in your metering?



/misterleffe