So I have to admit that I was excited when I found out that the house that we would be renting had a PG&E SmartMeter installed. For whatever reason I happen to think its cool if you can find out just how much power you use.
However in order for this to be useful you need a fairly real time display of the data so that you can determine what actions cause what amount of power to be used. So once I got my first bill from PG&E I logged into their website in hopes of finding a nice real time interface like Google PowerMeter. Not so.
Well okay maybe they expose the data through an API. Nope.
The Challenge
At this point it had become a challenge, how can I get the data out of the system. I did a little handy work with firebug and spied what was going on (ASP.net applications are ugly!). Once I got the request parameters copied down I tried making requests through curl and kept hitting roadblocks. After spending several hours late one night last week I gave up in frustration.
Today I thought I would give it another shot. First let me give a shout out to mechanize I will never use another screenscraping library again. It’s uses nokogiri (so it parses all the html into a nice xpath accessible form), it handles all the cookie session legwork and it reads out the values saved in the forms, so you don’t have to. So awesome.
The Result
Well it’s done. It takes no less than 6 requests across 2 websites (apparently PG&E outsources the data display to http://www.energyguide.com/) to get the data with 10’s of post parameters along the way. I happen to think this is rather absurd, it should just be 1 request with about 4 parameters.
So to benefit the world I have made it into a small library hosted on github. Now I want to see some interesting uses of the data.
Sample Code:
api = SmartMeterService.newapi.login(USERNAME, PASSWORD)
samples = api.fetch_day(Date.today - 1)
puts samples.inspect
Code: