Wireless Energy Meter Plugin - BETA

http://automagucally.weebly.com

Moderators: davka003, Telldus

KHolm
Posts: 163
Joined: Fri Mar 17, 2023 9:45 am

Re: Wireless Energy Meter Plugin - BETA

Post by KHolm »

misterleffe wrote:Lot of work you have done Marcus! Hopefully it will be implemented in Automagically later on. :D
Already in, David merged it in today. Xively plugin also merged.
Will be installed with next update. But, you might need to remove current wem.py* before you perform update.

Remove wem.py and update from command-line

Code: Select all

$ sudo service automagically stop
$ sudo rm ~/source/automagically/site/plugins/wem.py*
$ sudo rm ~/source/automagically/site/plugins/xively.py*
$ sudo service automagically start
$ cd ~/source/automagically/
$ sudo update.sh
€ sudo service apache2 start
$ sudo service automagically start
Your settings may disappear (primaryId, etc) but the rest will be intact.
misterleffe wrote: Should i delete the previous version of wem in the "plugin directory" and replace it with the attached version? After replacement of the new version(?), can I just continue the settings with the new ones stated N+4 ......N+9?
To update to version 0.5 if you don't want to update:

Code: Select all

$ sudo service automagically stop
$ cd ~/downloads
$ rm wem*
$ wget http://www.telldus.com/forum/download/file.php?id=432 -O wem.zip
$ cd ~/source/automagically/site/plugins
$ rm wem*
$ unzip ~/downloads/wem.zip
$ sudo service automagically start
Your settings may disappear (primaryId, etc) but the rest will be intact.
misterleffe wrote: The settings as it is right now, wouldn't it give "peaks" with resets every hour? I've followed your tutorial point by point. Shouldn't just only the accumulated energy rise all time and just look like "Holmenkollen"? My graph for daily energy looks like just mentioned down hill. :?:
That is how it will look in an Automagically remote, yes.
But in Xively: no, it is accumulating and only reports to xively just before the reset. So, xively only get one value per hour/day/month. So, the curve will be flat.
If you don't want "holmenkollen" in Automagically, you need one more variable and use a Store variable signal to capture what you send to Xively in that variable.
misterleffe wrote: -Parser type: Line,Character,position,Length
-Parse: Line=0 Char=0 Size=1
-ParsedVariable: 1
Yes, that should work/is what I have. In reality not even used.
misterleffe wrote: Can you attach a clip of your graphs and how it looks in your metering?
Here:
xively.png
xively.png (102.96 KiB) Viewed 20711 times
/Marcus
misterleffe
Posts: 57
Joined: Fri Mar 17, 2023 9:45 am

Re: Wireless Energy Meter Plugin - BETA

Post by misterleffe »

Ok Marcus, but I don't get the point to have a reset every hour. If I disable or enable the data fetcher, the curve for "accumulatedEnergy" will not change. It's showing the same curve. Maybe I don't understand this correct.

"That is how it will look in an Automagically remote, yes.
But in Xively: no, it is accumulating and only reports to xively just before the reset. So, xively only get one value per hour/day/month. So, the curve will be flat.
If you don't want "holmenkollen" in Automagically, you need one more variable and use a Store variable signal to capture what you send to Xively in that variable"


I thougt the point to have such statistics, would be to have statistics for hour/day/month/year so you can compare for instance Feb 5 2014 with Feb 5 2015. What I mean is, I want a peak for every hour/day/month/year. It's fine to have the total consumtion, but I would become more intresting to "split it up". Then you can really compare in that way you want. I want a curve like a temerature curve, but for energy.

Have I missunderstood everything? Can you please advice me how to get the statistics that I want? I ask for much, but I think you're a "Rock" Marcus!




/misterleffe
KHolm
Posts: 163
Joined: Fri Mar 17, 2023 9:45 am

Re: Wireless Energy Meter Plugin - BETA

Post by KHolm »

Hi,

It's all about selecting the right tool, or as Maslow said- if all you have is a hammer, everything looks like a nail.

Xively is not a data repository and were never meant to be used as repository for analytics. It's a tool where you can quickly visualize your data - and please be advised, with a developer account it'll only store 30 days of data for you.

So, for your purposes, Xively is not the tool. For me it's a tool to see how my daily consumption looks. I can now look at it in a cloud solution:

a.) per minute, to see how different devices uses energy

b.) per hour, to see patterns over they day; like that I'm using more energy mornings and evenings. I can also compare patterns over days.

c.) per day, to see how my home differ between weekdays and weekends

And as said, monthly/tearly will be N/A due to limitations in the developer account at Xively, and I will not pay $999 per year to store my meters readings - no way :)

But, to go into the data - you need to have it stored in some database. And as I said before, my next project is to get things into Google Spreadsheets. With the data into a spreadsheet, you can do whatever you want with it.
In fact, my first version is already working. My biggest problem right now is the limitations related to how Google works with date and time in the current version of Google Spreadsheets (a new is coming that will fix these impediments).

Here's a first screenshot what happens when the system Automagically finds events from the Wireless Energy Meter and uploads the result to Google Spreadsheets by appending a new row.

It depends on an external module (gspread) which may be a problem to get it into the main trunk as David is a bit hesitant to do to. Possibly I can get in into the "plugins/examples" at least.

Will have to work with it a bit more before I release it to the public domain.
googlespread_first_picture.JPG
googlespread_first_picture.JPG (47.99 KiB) Viewed 20673 times
/Marcus
misterleffe
Posts: 57
Joined: Fri Mar 17, 2023 9:45 am

Re: Wireless Energy Meter Plugin - BETA

Post by misterleffe »

Ok Marcus, I didn't understand that about Xively, the limitations... Thanks for sorting it out for me!

Ok, to have datapoints in spreadsheets, that will rise the statistic to a "higher level". Then you can make any diagram you want. Of course, I'm ready to test whenever you have a beta version on the go!
I think I'm leaving Xively for detailed statistics also because the pricing. This is just a hobby! Xively is fine for "live logging", so I will use it for accumulated energy and live energy consumption.

Thank you Marcus, for having patience with all questions popping up all time.



/misterleffe
KHolm
Posts: 163
Joined: Fri Mar 17, 2023 9:45 am

Re: Wireless Energy Meter Plugin - BETA

Post by KHolm »

I have a catch-22 issue with Google Spreadsheets, which makes it a temporary dead-end.

#1 - Catch 1
The current versions TIME function sucks - big time, and can barely be used.

I want to do this: Pmom = Energy*60*60/(Time_now - Time_previous) but cannot make Google calculate when I have a date wrap.

I need to move to New Google Spreadsheets.
https://support.google.com/drive/answer ... heets&rd=1

GOTO #2

#2 - Catch 2
...but New Google Spreadsheet does no support gspread module or any G-API at the moment, to I need to use TIME
https://github.com/burnash/gspread/issues/89

GOTO #1

#3 - Bonus catch 3
JavaScript in Google could calculate Pmom for me; but it can not be done while adding as row; needs to be done post-processing for each row - and that will take some time if your data grows.

Guess I need to do the Pmom calculation in wem.py. Had been hoping to leave that to post-processing...

Any ideas? Bueller? ...Bueller? ...Bueller? Anyone? Anyone?

/Marcus
KHolm
Posts: 163
Joined: Fri Mar 17, 2023 9:45 am

Re: Wireless Energy Meter Plugin - BETA

Post by KHolm »

Hi,

I've updated the plugin for the Wireless Energy Meter to version 0.6 BETA. It has not yet been provided as a pull request, I want to let it stabilize first.
The following things has changed:

#1 - Added output parameter Pmom [EXPERIMENTAL]
The plugin now calculates the Pmom = temp*(10000/blinksPerWatt)/1000*60*60/(Time_now - Time_previous)), as per instruction from http://foogadgets.tictail.com.
Formula under review and discussion, for now experimental
The signal to match should now be (for humanReadableTime = True, see below).
wem,data;status:%s;datetime:%s;seq:%d;energy:%f;packetloss:%d;pmom:%f;

#2 - Corrected a bug wrt. timestamp
The timestamp from the plugin was not correct. Now fault tolerant and correct.

#3 - Corrected a bug wrt. reporting blinks (in energy parameter) for systems with 10000 blinks per kWh.
The system always reported Int, now changed to float. See above new signal format.

#4 - Added a new setting, humanReadableTime.
When set to False it'll output seconds since EPOCH instead if ISOTIME. This helps a lot when working with e.g. Google Spreadsheet or other tools that has issues to manipulate time. It is reported as a float with 6 decimals (microseconds).
The signal to match should now be (for humanReadableTime = False)
wem,data;status:%s;datetime:%f;seq:%d;energy:%f;packetloss:%d;pmom:%f;
See picture below:
wem.JPG
wem.JPG (22.83 KiB) Viewed 20628 times
To change to the latest version, follow these instructions:

Code: Select all

$ sudo apt-get install wget
$ sudo apt-get install unzip
$ sudo service automagically stop
$ cd ~/downloads/
$ wget https://bitbucket.org/MarcusKarlsson/automagically/raw/1119b626e055f3c14f3f4746621e3a70345199ec/site/plugins/wem.py
$ cd ~/source/automagically/site/plugins
$ sudo rm wem.py*
$ unzip ~/downloads/wem.zip
$ sudo service automagically start
Please feedback if you find bugs and has an opinion on the Pmom calculations. I suck at those parts, gimme the formula and I'll make it work :D

/Marcus
KHolm
Posts: 163
Joined: Fri Mar 17, 2023 9:45 am

Re: Wireless Energy Meter Plugin - BETA

Post by KHolm »

KHolm wrote: #1 - Added output parameter Pmom [EXPERIMENTAL]
The plugin now calculates the Pmom = temp*(10000/blinksPerWatt)/1000*60*60/(Time_now - Time_previous)), as per instruction from http://foogadgets.tictail.com.
Formula under review and discussion, for now experimental
So, after a a few days of monitoring Pmom and having found "proof" for the formula (http://www.elektronikforumet.com/forum/ ... f=2&t=5022) I started to wonder how much we shall trust the time-stamp of the Pi.

The current formula looks like this:

Code: Select all

pmom = round(input[0] * 1/1000.0 * 60 * 60/float(time_delta), 5)
The formula uses all the way down to microseconds. The effect of a clock drift in the Pi+Wireless Energy Meter percolates down to be noticeable on the graphs of Xively.

Comparing E and Pmom for the same time period looks something like this:
pmom_e_diff.JPG
pmom_e_diff.JPG (16.87 KiB) Viewed 20603 times
They should of course differ, considering that the Wireless Energy Meter drifts. So, I was wondering if I should only trust the Pi down to milliseconds by rounding and compensating for the Pi's internal drift. Still, from the post below I think we can conclude that the drift is not enough to have to compensate for.
http://blog.remibergsma.com/2013/05/12/ ... keep-time/

So, my conclusion is that the Pmom included version 0.6 is correct and can be trusted and will now be provided to Automagically in a pull request.

/Marcus
KHolm
Posts: 163
Joined: Fri Mar 17, 2023 9:45 am

Re: Wireless Energy Meter Plugin - BETA

Post by KHolm »

Hi,

I noticed that Xively has some issues when sending data to frequently, typically the first message is lost for some reason.
So if you want to upload both E and Pmom to Xively you might want to send it in the message to Xively.

Use this signal transform to do so:
Pattern to match: wem,data;status:%s;datetime:%s;seq:%d;energy:%f;packetloss:%d;pmom:%f;
Send the following signal(s) on match: xively,report:Energy:$4:W,Pmom:$6:kWh;

The problem is not in Automagically or its plugins, it's with Xively - and likely connected to that we/I use developer accounts.

/Marcus
KHolm
Posts: 163
Joined: Fri Mar 17, 2023 9:45 am

Re: Wireless Energy Meter Plugin - BETA

Post by KHolm »

Version 0.7 has been committed to bitbucket and a pull request has been created for David to assess.

This version adds a protocol wrap that occurs when the energy meter has detected more that 2048 blinks and wraps to negative numbers on the protocol.

It can be downloaded using this command:

Code: Select all

$ wget https://bitbucket.org/MarcusKarlsson/automagically/raw/5bc589ba464f3d82992aacc86073bd807a1d93bb/site/plugins/wem.py -O wem.py
Thanks to Niclas F for informing me about this protocol quirk.

/Marcus
KHolm
Posts: 163
Joined: Fri Mar 17, 2023 9:45 am

Re: Wireless Energy Meter Plugin - BETA

Post by KHolm »

Version 0.8 has been committed to bitbucket and a pull request has been created for David to assess.

This version implements special case to handle -0.0 from energy meter.

It can be downloaded using this command:

Code: Select all

$ wget https://bitbucket.org/MarcusKarlsson/automagically/raw/7a8e04048203cf92059ce59ef3bb4e753111df34/site/plugins/wem.py -O wem.py

Thanks to Niclas F for informing me about this special case

/Marcus
KHolm
Posts: 163
Joined: Fri Mar 17, 2023 9:45 am

Re: Wireless Energy Meter Plugin - BETA

Post by KHolm »

Version 0.9 has been committed to bitbucket and a pull request has been created for David to assess.

This version implements logging to csv file.

It can be downloaded using this command:

Code: Select all

 $ wget https://bitbucket.org/MarcusKarlsson/automagically/raw/1a07b67f8b15975a16715d4ff637ee97f953f063/site/plugins/wem.py -O wem.py
/Marcus
Post Reply