So I have to admit that I was excited when I found out that the house that we would be renting had a PG&E SmartMeter installed. For whatever reason I happen to think its cool if you can find out just how much power you use.
However in order for this to be useful you need a fairly real time display of the data so that you can determine what actions cause what amount of power to be used. So once I got my first bill from PG&E I logged into their website in hopes of finding a nice real time interface like Google PowerMeter . Not so.
Well okay maybe they expose the data through an API. Nope.
The Challenge
At this point it had become a challenge, how can I get the data out of the system. I did a little handy work with firebug and spied what was going on (ASP.net applications are ugly!). Once I got the request parameters copied down I tried making requests through curl and kept hitting roadblocks . After spending several hours late one night last week I gave up in frustration.
Today I thought I would give it another shot. First let me give a shout out to mechanize I will never use another screenscraping library again. It’s uses nokogiri (so it parses all the html into a nice xpath accessible form), it handles all the cookie session legwork and it reads out the values saved in the forms, so you don’t have to. So awesome.
The Result
Well it’s done. It takes no less than 6 requests across 2 websites (apparently PG&E outsources the data display to http://www.energyguide.com/) to get the data with 10′s of post parameters along the way. I happen to think this is rather absurd, it should just be 1 request with about 4 parameters.
So to benefit the world I have made it into a small library hosted on github. Now I want to see some interesting uses of the data.
Sample Code:
api = SmartMeterService.new api.login(USERNAME, PASSWORD) samples = api.fetch_day(Date.today - 1) puts samples.inspect
Code: git://github.com/MattColyer/smartermeter.git
8 Comments
Hey Matt,
I saw this pop up on planet olin. Since I follow this stuff for research, I’m entertained by the amount of work you had to put in to get out the data! I was wondering if PG&E was providing any support/interface for the new meters and if you know what they’re capable of (time of day pricing? real time pricing? interface for control of appliances?)
Hope all is well in CA.
Wow, I didn’t know I was still on the planet. I figured they had removed us old timers.
Yeah the amount of work that it required is truly stupid. I hear the main reason that PGE put the meters in isn’t for getting people to change their energy usage but to eliminate jobs for meter readers (humans cost alot of money, apparently enough to justify replacing all the meters).
PG&E doesn’t provide support at all. I am hoping that some clever individuals will figure it out. I believe that the meters can be used for time of day pricing because they make general statements about that in the literature (without any specifics about how I might get it).
I don’t think the resolution is real time, I am pretty sure the device batches up a bunch of data and then transmit it over a cell network. I am pretty sure it doesn’t interface with any appliances.
It doesn’t actually have to report real time data to the central server to get real time pricing. They would just work backwards.
For the higher functionality (like some types of demand response), they would generally want to be able to send you signals in addition to your meter sending out.
You might find looking at the NIST standards for ‘cybersecurity’ with smart grid interesting.
Great idea! I consistently get this problem, any ideas?
/usr/lib/ruby/gems/1.8/gems/mechanize-0.9.3/lib/www/mechanize.rb:294:in `click’: undefined method `[]‘ for nil:NilClass (NoMethodError)
from ./../lib/service.rb:32:in `login’
from /usr/lib/ruby/gems/1.8/gems/mechanize-0.9.3/lib/www/mechanize.rb:241:in `get’
from ./../lib/service.rb:26:in `login’
from graph.rb:36
Matt,
For your information, Arizona has time-of-day metered service. I do not know how APS implements the service. I do know that using the “right” plan will substantially reduce the electric bill. The following link provides information on the various rate plans
http://www.aps.com/main/services/residential/rates/Default.html
Since I do not have the time to explore and learn the all of the required programming and available apps/subroutines, I do appreciate the work that you have done. If you create an application that any reasonably intelligent person can install and run that will provide “a fairly real time display of the data,” please let me know.
Thank you for letting me know that it can be done
- – - Gene
Nice work. i hope it lights a fire under PG&E to make such an online meter available for all whom have a smart meter installed.
I heard PG&E will be changing the way they charge for power and start charging for VARs usage come 2012….
Read it now believe me later…
http://github.com/MattColyer/smartermeter does not seem to be a valid link. Where can I find your library?
Ah, found it. The actual link is… http://github.com/mcolyer/smartermeter