To me, ‘web 2.0’ is fresh, fast and fun. Gridspy is all about making power monitoring fascinating and accessible - that’s the goal. We’re talking numbers you can understand, graphs you can read and systems that you can intuit as you change the amount of power you use. Read on to learn how we deliver on our live data vision all the way from the Nexus to the browser.
Naturally like any sensor system, the data flow starts at the sensor. We have two sensor solutions - the wired Nexus and the wireless Gridspy. Both of these have several sensors that can clip onto the live mains cables in your house and the Nexus also connects to your home / office Ethernet. Every second each sensor takes 64 readings and uploads an accurate measurement of your power usage. Because home power uses the alternating current system to ease the transmission of power from generators to your house, power readings move up and down over time. The Gridspy system takes readings over several up and down cycles of your power and uses sophisticated mathematics to compensate for the alternating current to get nice smooth power readings.
Once plugged into a home or office Ethernet, the Nexus automatically detects the network configuration and establishes an outgoing connection to our servers. Once the connection is established, the nexus begins streaming live power data. What is really great about this scheme is that outgoing connections ensure that on most networks absolutely no configuration is necessary - our users do not have to expose the Nexus or any other device on the network to the general public to view their data on-line.
The nexus then collects those sensor samples from the nearby Gridspies, bundles it up with its own data and uploads it to our central servers. To collect the data we use a custom application written using the excellent Twisted framework. Every single reading gets instant attention: it is evaluated for real-time events, queued to be stored in the database and finally forwarded to watching dashboard users. Processing the data stream live on the server opens up many exciting possible features that I look forward to discussing in future blog posts. By the time that the sample reaches the server, it is about 0.2 seconds old.
Each time someone opens up a browser and starts viewing the data dashboard for a Gridspy (live power demo!), the dashboard makes a connection back to our server using a technology called Orbited. Orbited gives us a two way event-driven communication channel so our users can send messages to the server such as switching lights on and off while receiving those precious data samples back in real time. Through Orbited, our users subscribe to a “message queue” which efficiently sends power updates and other messages to all watching web browsers. We are currently using MorbidQ as our message queue for ease of deployment. If you want to set up a similar system, I thoroughly recommend this tutorial on Orbited, MorbidQ and Twisted.
So, the total time from taking a sample to seeing the result in the dashboard? It takes as much time as the power message takes to travel from your network to our server and the reading takes to travel back. For most people, this is a 0.2 to 0.4 second delay or less - it sure beats the old “poll the database every 5 minutes for latest readings” mechanism that I used in our early prototypes. The result? Power monitoring is fun and intuitive. If you hear the washing machine speeding up into a spin cycle, you will instantly see that reflected in your dashboard online. When you turn off those lights, you get instant feedback showing you exactly how much your power usage has dropped. This is the sort of feedback that motivates experimentation and power saving.
Power monitoring, meet web 2.0!
If you want more technical detail on any part of my system, please do drop me a note in the comments below. You can also follow me on Twitter.
I came across your website http://www.gridspy.co.nz/. It is very gool. I was wondering what front end and database do you use for your realtime site and historian.
I have been building an intranet real time website for our organization based on OSI-PI servers to archive SCADA data. It has been up and running for a couple of years and improving. My data site is restricted inside the firewall due to security concerns. It uses MS Sharepoint WSS 3.0 with Webparts but I don't feel it flexible enough unless I decide to dive into MS Webparts coding which requires a lot of time and effort. My graphics relies on SVG and currently I have no other alternatives. I am hoping you might shed some light on alternatives.
Since our firewall doesn't allow direct access to my site, I have to send data to our internet site. Here is an example of a map that refreshes every 10 minute, even though it could be real time. Here is the link: http://www.kingcounty.gov/environment/wastewater/CSO/RealTime/SeattleOverview.aspx
I am looking forward to hearing from you. Thanks for your attention.
Zhong
Hi Zhong.
The front end is custom, written in Python using the Django framework. The backend is postgresql, however I expect to move to one of the column-oriented databases as our load increases.
For graph rendering, we use Matplotlib.
I suggest that you create an application inside the firewall that is connected to your backend server and which connects to a public server to transmit live data. On the frontend server, you forward that data to users. In terms of UI, you could render a map and place data on top of it using relatively placed DIV tags.