Parser - hvordan få lest verdiene fra wind og rain sensor

[atlehaav]

Hei.

Har et fungerende oppsett med HA 3.0.1 (temp, hygro, 1-wire, nexa switcher, relay switcher, røykdetector) men gir meg ikke med det. Har nå en Oregon weatherstation (WMR88A med wind sensor WGR800 og rain sensor PCR800). Etter mye strul med TD tool (buffer owerflow WGR800 - men fikset etter kompilering av kildekode og innkopiering av ny tdtool) har jeg nå datene tilgjengelig. Tdtool -l gir:

PROTOCOL MODEL ID TEMP HUMIDITY RAIN WIND LAST UPDATED
mandolyn temperaturehumidity 11 16.0° 96% 2015-01-20 22:36:10
oregon 1984 17 ) SE, 1.1 m/s (0.9 m/s) 2015-01-20 22:36:58
oregon 2914 11 0.0 mm/h, 9.5 mm 2015-01-20 22:36:54
oregon F824 18 -19.9° 63% 2015-01-20 22:35:25
...
Kjøring av parser gir:
Array
(
[0] => oregon
[1] => 1984
[2] => 17
[3] =>
[4] => )
[5] =>
[6] => SE, 0.0 m/s (0.5 m/s)
[7] => 2015-01-20 23:09:53
)
Array
(
[0] => oregon
[1] => 2914
[2] => 11
[3] =>
[4] =>
[5] => 0.0 mm/h, 9.5 mm
[6] =>
[7] => 2015-01-20 23:09:48
)
Array
(
[0] => oregon
[1] => F824
[2] => 18
[3] => -20.6
[4] => 63
[5] =>
[6] =>
[7] => 2015-01-20 23:08:59
)
...

Hvordan får jeg dette videre fra parser og representert som sensorverdier i HA? Har sett inn i (og prøvd å forandre) telldusduo.php i /system folder (som jeg mener er nøkkelen til løsning) sammen med endring i telldusduo.php i /system_config, men får dette ikke til å virke . Noen tips?

[Daniel]

Vilken jättemärklig output tdtool ger ifrån sig, jag förstår att parsern inte lyckas så bra med den... Vad får du om du istället kör "tdtool --list-sensors"?

/Daniel

[atlehaav]

Hei, og takk for svar.

Vel, parseren virker fin-fint på alle andre "vanlige" sensorer. Får inn både temp og hygro (på de av mine som har dette). Ved å kjøre tdtool --list-sensors fås:

type=sensor protocol=mandolyn model=temperaturehumidity id=11 temperature=16.0 humidity=96 time=2015-01-21 20:28:10 age=51
type=sensor protocol=oregon model=1984 id=17 winddirection=157.5 windaverage=0.5 windgust=0.5
type=sensor protocol=mandolyn model=temperaturehumidity id=14 temperature=6.2 humidity=54 time=2015-01-21 20:28:31 age=30
type=sensor protocol=oregon model=2914 id=11 rainrate=0.0 raintotal=9.5
type=sensor protocol=oregon model=F824 id=18 temperature=-18.5 humidity=63 time=2015-01-21 20:29:00 age=1
type=sensor protocol=mandolyn model=temperaturehumidity id=13 temperature=6.4 humidity=60 time=2015-01-21 20:29:01 age=0

Jeg ser at denne gir noe mer oppløsning på verdiene, men jeg er fortsatt "stucked in the mud". Svar mottas med takk

[Daniel]

Ok. Problemet är att det var ett tag sedan Tom ifrån första början gjorde denna parser. På den tiden så fanns det bara ett sätt att få ut information ur tdtool och det var med --list, så det är det parsern använder. Uppenbarligen funkar inte det med alla givare. Vad vi skulle behöva göra är egentligen (och det enda sättet för att få det att fungera för dig) att ändra i vår parser så att den istället använder sig av --list-sensors.

Så svaret till dig är tyvärr att det fungerar helt enkelt inte i den nuvarande versionen. Jag ska göra ett försök att samla ihop kraft och tid (tror egentligen inte det är något jättearbete) att koda om parsern. Skulle dock inte tro att det händer närmsta veckorna.

/Daniel

[atlehaav]

Flott å høre. Jeg ser frem til en eventuell fix/endring. Hadde vært virkelig supert å fått inn disse værdataene også.

Takk for gode svar sålang, og keep up the good work

A

[Daniel]

I had to be home from work today because my kids have a cold, so I actually could spend some hours coding. Could you try downloading http://kolefors.se/telldusduo.php.txt, renaming it to telldusduo.php and placing it in parser/systems (replacing the existing telldusduo.php) and report back? With any luck it should work

(btw, anyone knows what file extension I should need to attach the file here in the forum? I gave up after trying ".php", ".txt" and ".whatever".)

/Daniel

edit: And why didn't the forum show my link as a link? I DID push the URL button. I wonder if this forum simply doesn't like idn...

edit (mod): URL doesn´t seem to handle åäö...

[atlehaav]

Hi, and great effort so far! Sometimes, It can really be more than OK to stay home watching over the kida

I think we are almost there, but.. Looks like the parser have some problems giving an ID to other values than temp: My output running the parser (have shorten it down a bit but still a lot of lines - sorry for that one) finds all the sensors and even write out the last ID 0 (with correct value). Temp sensor value (and ID) reading fine.

A lot to ask, but do you mind take another look at the codelines ?

Array
(
[0] => type=sensor
[1] => protocol=mandolyn
[2] => model=temperaturehumidity
[3] => id=11
[4] => temperature=16.3
[5] => humidity=96
[6] => time=2015-01-22 16:40:11
[7] => age=12
)
Writing id: mandolyn-temperaturehumidity-11, value: 16.3, time: 2015-01-22 16:40:11.
Writing id: 0, value: 96, time: 2015-01-22 16:40:11.
Array
(
[0] => type=sensor
[1] => protocol=oregon
[2] => model=1984
[3] => id=17
[4] => winddirection=157.5
[5] => windaverage=0.6
[6] => windgust=0.5
)
Writing id: 0, value: 157.5, time: 2015-01-22 16:40:24.
Writing id: 0, value: 0.6, time: 2015-01-22 16:40:24.
Writing id: 0, value: 0.5, time: 2015-01-22 16:40:24.
Array
(
[0] => type=sensor
[1] => protocol=fineoffset
[2] => model=temperaturehumidity
[3] => id=1
[4] => temperature=9.9
[5] => humidity=45
[6] => time=2015-01-22 16:40:05
[7] => age=18
)
Writing id: fineoffset-temperaturehumidity-1, value: 9.9, time: 2015-01-22 16:40:05.
Writing id: 0, value: 45, time: 2015-01-22 16:40:05.
Array
(
[0] => type=sensor
[1] => protocol=fineoffset
[2] => model=temperature
[3] => id=207
[4] => temperature=11.2
[5] => time=2015-01-22 16:40:05
[6] => age=18
)
Writing id: fineoffset-temperature-207, value: 11.2, time: 2015-01-22 16:40:05.
Array
(
[0] => type=sensor
[1] => protocol=oregon
[2] => model=2914
[3] => id=11
[4] => rainrate=0.0
[5] => raintotal=9.5
)
Writing id: 0, value: 0.0, time: 2015-01-22 16:40:24.
Writing id: 0, value: 9.5, time: 2015-01-22 16:40:24.
Array
(
[0] => type=sensor
[1] => protocol=oregon
[2] => model=F824
[3] => id=18
[4] => temperature=-15.6
[5] => humidity=63
[6] => time=2015-01-22 16:40:07
[7] => age=16
)
Writing id: oregon-F824-18, value: -15.6, time: 2015-01-22 16:40:07.
Writing id: 0, value: 63, time: 2015-01-22 16:40:07.
Array
(
[0] => type=sensor
[1] => protocol=mandolyn
[2] => model=temperaturehumidity
[3] => id=13
[4] => temperature=6.5
[5] => humidity=59
[6] => time=2015-01-22 16:40:02
[7] => age=21
)
Array
(
[0] => type=sensor
[1] => protocol=fineoffset
[2] => model=temperature
[3] => id=225
[4] => temperature=23.3
[5] => time=2015-01-22 16:40:06
[6] => age=17
)
Writing id: fineoffset-temperature-225, value: 23.3, time: 2015-01-22 16:40:06.

Found: mandolyn-temperaturehumidity-11, value: 16.3
Found: 0, value: 96
Found: 0, value: 157.5
Found: 0, value: 0.6
Found: 0, value: 0.5
Found: fineoffset-temperaturehumidity-1, value: 9.9
Found: 0, value: 45
Found: fineoffset-temperature-207, value: 11.2
Found: 0, value: 0.0
Found: 0, value: 9.5
Found: oregon-F824-18, value: -15.6
Found: 0, value: 63
Found: mandolyn-temperaturehumidity-13, value: 6.5
Found: 0, value: 59
Found: fineoffset-temperature-225, value: 23.3
Parsing 'telldusduo' completed

[atlehaav]

Success!!

Did some "try and fail" and even learned some basic .php codes as well. Was a small syntax mistake in the last argument (that list up the other ID's when not a temperature sensor). Corrected

$haId += $value["name"]
with
$haId .= $value["name"];
and bingo - everythings parse correct IDs.

Did a small addon as well (iot have the same ID outputs to my last setup - missing a "-" before the other sensors) Not really a artpeace of coding, so you might change that one into a better one
foreach($availableValues as $value)
{
$txt = "-";
$haId = $haBaseId;
if($value["name"] != "temperature")
{
$haId .= $txt;
$haId .= $value["name"];
}

Nevertheless; thanks a lot for your help. I couldn't do this without your effort!

Happy weatherlogger now

[Daniel]

and bingo - everythings parse correct IDs.

That's the spirit! Well done!

...and now everybody sees what happens when you don't stick to one language. In C++ "+=" works perfectly fine

Anyway, I changed your code slightly (keeping the dash in a variable isn't really needed) and I will now commit it and upload it to the HA homepage. Thanks a lot!

/Daniel

[badtastex]

Good to know that the Oregon sensors will be possible to get into HA, how will it be presented? Can it parse to m/s and mm?

Atlehav: As I understand it you are using HA as a weather station, have you looked into the possibility to backup the log of the weather data somehow? Since I use a raspberry Pi I am really looking for a good solution for this since every power out situation leads to the possibilty of lost data...

[Daniel]

Good to know that the Oregon sensors will be possible to get into HA, how will it be presented? Can it parse to m/s and mm?

The units in which sensors are presented are manually set up in settings->sensor types and then you simply choose for each sensor which type of sensor it is. In my HA for example I have the sensor types temperature, humidity and luminance. Does that answer the question?

/Daniel

[atlehaav]

I'm also using a rPI (with telldus duo and HA of course). All the things are stored on the same SD card, so not that redundant solution (power failure might seriously corrupt the SD card) - but I do make a new image of my setup/data from time to time.

The two oregon sensors (rain and wind) gives out storeable data for sure. Windspeed in m/s, bearing in degrees and rainfall in mm. The parser just take care of the real values. All of these are like Daniel says, just to put the desired label upon in HA - and shows up accordingly.

I believe HA does the things you're looking for - store the data in a DB for each sensor as long (and with "unlimited" data) as you like. My logging setup is rather agressive - with polling of 36 sensors (as of now..) each third minute. Logging among others a heat pump and some automation like switching out a heat exchanger when defrosting, start a dehumidifier in the basement when humidity reach a certain level.
That number of sensors and using "measurement tab" (to display ALL stored sensors simultanously daily, weekly, yearly) combined with the limitation of the rPI is from my point of view not usable (too much data/computations for the tiny rPI). But retriving data history for each sensor is used all the time.

Not that I'm into it (and does not bother to try out for the time being) , but guess you can backup the DB to another drive and/or use other applications to retrive stored values (sql queries). In that way you can use HA as a backup/weather data storage for sure (even though I don't think this was the primary goal when HA was made).

/Atle

[tom_rosenback]

The MySQL database is easily accessible. With this said you can use the 'mysqldump' tool to backup the complete HA database. If you don't have any computer running 24/7 you could just setup a job for the backup that executes at startup.

[badtastex]

Thanks Daniel, I was just wondering since I saw that the humidity was parsed (or at least shown) as degree centigrade after my update. Maybe you have fixed that or maybe it is possible for me to fix it myself.

atlehaav: I dont have that many sensors and I am not sure I will either but I still have had problems with some power outs that has corrupted my SD-card. My countermeasure for this has been the same as you use - to have a SD-card image handy for the latest known configuration. But that does not take care of the logged data.
We have an old house and are looking at doing some updates such as renovation and increased ventilation and I am keen on actually seeing the impact on humidity in the attic for instance, that is why I am not looking forward to a power out wiping out half a year of log data.

Tom: The mysqldump is the soulution I have implemented for now but the file that is being dumped is not straightforward to load into Excel or Matlab for data analysis, I would really want a solution that can dump a file in a CSV-format and only contains the data I want. There is a NAS on my network and a cron-job dumps the sql database for now. There is one possible vulnerability to this solution - a power out while dumping the file. I guess that I could solve this by using a date-based filename and create multiple files though...

[tom_rosenback]

Thanks Daniel, I was just wondering since I saw that the humidity was parsed (or at least shown) as degree centigrade after my update. Maybe you have fixed that or maybe it is possible for me to fix it myself.

atlehaav: I dont have that many sensors and I am not sure I will either but I still have had problems with some power outs that has corrupted my SD-card. My countermeasure for this has been the same as you use - to have a SD-card image handy for the latest known configuration. But that does not take care of the logged data.
We have an old house and are looking at doing some updates such as renovation and increased ventilation and I am keen on actually seeing the impact on humidity in the attic for instance, that is why I am not looking forward to a power out wiping out half a year of log data.

Tom: The mysqldump is the soulution I have implemented for now but the file that is being dumped is not straightforward to load into Excel or Matlab for data analysis, I would really want a solution that can dump a file in a CSV-format and only contains the data I want. There is a NAS on my network and a cron-job dumps the sql database for now. There is one possible vulnerability to this solution - a power out while dumping the file. I guess that I could solve this by using a date-based filename and create multiple files though...

The parser should not output any unit at all, or did you mean in HA? If in HA, HA doesn´t care about what name / id a sensor has, you have to explicitly select the unit as Daniel wrote.

I did not understand that you were seeking for a export instead of a backup. If it is ok to do the export manually you probably could do it with any of the free MySQL tools otherwise you would need a script to export the database to a given format.