One of the biggest questions facing the providers of energy disaggregation technology is how to prove that it works as advertised. Then there’s the inevitable follow-up question — how well does it work compared to the competition?
Over the past few years, we’ve been covering companies like Bidgely, PlotWatt, Smappee, Neurio (formerly Energy Aware), Navetas, Belkin, Intel, and others offering technology to disaggregate whole-home energy data into specific breakouts of air conditioning, water heating, appliances and other typical building electricity loads. But without large-scale, standardized testing methods, it’s hard for would-be users of the technology to know whether they’re getting what they’ve paid for.
Enes Hosgor, CEO and founder of EEme, says that his company wants to change that. Last week, the Carnegie Mellon University spinout released results from what may be the biggest disaggregation technology test out there — a comparison of the results turned out by EEme’s algorithms against the circuit-level, real-time data being collected from 264 homes that are part of the Austin, Texas-based Pecan Street research consortium.
EEme disaggregated a year’s worth of 15-minute smart meter interval data, and was able to achieve about 70 percent accuracy in its estimates of air conditioning, water heater, clothes dryer and dishwasher energy use, compared to the granular data Pecan Street was pulling into its supercomputers. That’s on par with what other disaggregation technologies have been able to get out of 15-minute whole-home meter reads, according to tests from the Electric Power Research Institute (EPRI) that we covered in late 2013.
But it’s also a lot more comprehensive in terms of the data it’s being compared against, he said. Other disaggregation tests have been limited to mock homes set up in labs, or at most, a couple of dozen homes equipped with expensive and hard-to-maintain circuit-by-circuit sensors. And importantly, those test results haven’t yet been made freely available to the public, as EEme is now offering to do, he said.
“If you don’t have that insight in public, you cannot have a benchmark, a reference point, to move the entire knowledge base forward,” Hosgor said in an interview last week. “And if you don’t know the accuracy reference point, you cannot put that in the context for use cases,” which can range from load forecasting, demand response and energy efficiency measurement and verification for utilities, to appliance-by-appliance energy use and cost breakdowns for homeowners.
To be fair to the other energy disaggregation companies out there, EEme isn’t revealing the algorithms and approaches it uses. “Everything we’ve built at Carnegie Mellon is our own — and we don’t share how we do things, just like Bidgely and PlotWatt don’t share how they do things,” he said. But at least it’s giving out the results from what’s most likely the largest, and thus most statistically significant, test of this kind of technology out there today, he said.
Sharing the wealth from a treasure trove of home energy data
Pecan Street offers a unique resource on this front. No other entity, to our knowledge, has sensored and monitored so many homes at such fine detail for as long as it has. Even so, EEme isn’t the first energy disaggregation technology vendor to use this resource as a test bed.
“We have done projects like this for several companies, including EEme,” Bert Haskell, Pecan Street’s CTO, told me in an interview this week. “It’s just the first to publicly release its results.”
Haskell declined to name the other companies that have used Pecan Street’s enormous residential energy database to test their disaggregation technologies. But he did say that they include at least three Fortune 100 companies — and while he didn’t mention it, Intel publicly announced in 2012that it was testing its disaggregation tech at Pecan Street.
Pecan Street has been releasing some of its own results on this front, like its Sol app, which monitors time-stamped energy data from solar PV-equipped homes to catch problems with their solar generation ranging from dirty panels to faulty electronics. It’s also making its data available to universities and researchers through its WikiEnergy platform.
“I’m glad that EEme has decided to publish its results, because I think it would be good for the industry to discuss what the appropriate methods are” for testing these technologies, Haskell said. “For this industry to move forward, there have to be some kinds of industry benchmarks that are commonly understood.” That’s a view shared by the Department of Energy, which could be a valuable partner in bringing energy disaggregation systems to broader use.
EEme wants to bring its own disaggregation technology into play as a behind-the-scenes addition to other partners, rather than as its own energy portal, Hosgor said. So far it’s piloted with one California utility and another in Texas — while he wouldn’t name either, it’s likely that Austin Energy is the Texas partner — as well as at a U.S. military base. But it’s looking for partners outside the utility space as well, he said.
“We want to be the go-to analytics company in the DSM [demand-side management] market, for everybody to make more targeted and intelligent decisions,” he said. The company has received funding from Carnegie Mellon’s technology transfer center, and “we’re currently raising capital to expand our team and operations.”
Pecan Street’s Haskell noted that large-scale industrial and commercial power users have been using energy data for diagnostic and analysis uses for years. “The kind of work we’re doing is really focused at lowering the hurdle for people to utilize this capability to the point where a mobile app can use this data to save you money in your house, without you having to do much of anything,” he said.
“We would love to facilitate that process,” he said. “I think the industry does need a set of benchmarks — not a single figure of merit, because there are different classifications of problems. We certainly have the data to facilitate a lot of those, particularly when they’re related to residential systems.”
