California’s Distributed Energy Challenge: Sharing the Data

California’s goal of merging rooftop solar systems, smart inverters, behind-the-meter batteries, real-time energy management systems and other distributed energy resources with its utilities’ grid plans faces a major challenge before it even gets underway -- the data-sharing challenge.

Over the past nine months, the state’s big investor-owned utilities and its big distributed energy companies -- SolarCity in particular -- have been staking out positions on how much data they need to share with each other to overhaul utility planning on the grid edge, and how little has been shared so far.

The debate is happening on a tight schedule, as dictated by AB 327, passed in 2013. That state law set a July 1, 2015 deadline for Pacific Gas & Electric, Southern California Edison and San Diego Gas & Electric to turn in distribution resources plan (DRP) proposals that consider the effects of distributed energy resources -- DERs, in industry parlance -- on utility infrastructure investments.

And under a February guidance document from the California Public Utilities Commission (PDF), these plans will need to share more data, at greater detail and at faster speeds, than utilities have ever had to provide before. Specifically, utilities are being asked to lay out a plan to provide feeder-level conditions such as “coincident and non-coincident peaks, capacity levels, outage data, real and reactive power profiles, impedances and transformer thermal and loading histories, and projected investment needs over the following 10 years” -- all of it “in as near-real time as possible.”

California's utilities have cited security, privacy and procedural concerns about turning over so much internal data to outside parties. They've also pointed out that the technology to make it happen may not be ready for some time, and won't come without significant costs.

But according to Ryan Hanley, SolarCity’s senior director of grid engineering solutions, “without access to this data, the industry is flying blind on whether they can expect to install distributed energy resources in particular locations in the future." That's a problem, he said in an April interview, because “none of it is available today."

In an April presentation before the CPUC (PDF), SolarCity identified a long list of utility data it believes is critical for its planning needs. At our request, the company extended that list to include information on whether this data is currently available to utilities internally, as well as whether or not it's publicly available in any fashion -- and according to SolarCity, most of it isn't, at least in a form that's usable to third parties. (Click on the chart to the right for an expanded view.)

SolarCity’s critique of California utility data-sharing practices has become a fairly common refrain among solar companies, energy storage vendors, microgrid developers, demand response providers, and other companies seeking to provide grid-facing capabilities for DERs on the customer side of the meter.

“I’d like to see more information transparency” from utilities, Audrey Lee, vice president of development for Advanced Microgrid Solutions, said during an April panel discussion in San Francisco. “Right now the utility has a lot of this data, but sharing that data with other parties” is critical, she said, for giving third-party grid players the information they need to guide their investments in alignment with utility needs.

At the same time, the CPUC is also asking companies like SolarCity and other DER providers to hand over their own data to California’s utilities. That data request list could include some that's subject to confidentiality concerns, such as individual customer system data, or long-range plans for which parts of the state and which classes of customers they're targeting over the coming decade.

Both sides have some claims to exclusive rights to their data, as well as legal and technical limits to how much they can share. But neither side has the complete set of data required for the task at hand -- to create a new model for how third-party distributed energy resources and traditional utility investments can share the responsibilities and rewards of an integrated, more efficient and greener grid.

And this means that these parties, which have often been at odds over California energy policies, will have to find ways to reach a data-sharing compromise. “It’s not a question of whether or not information should be exchanged,” said Paul De Martini, former smart grid chief at Southern California Edison and head of the More Than Smart working group that’s helping to guide the CPUC’s DRP proceeding. “The answer has been established as being yes.”

Today’s data-sharing regime and the barriers to what lies beyond

California’s utilities do share some data today, particularly when it comes to system-wide renewable energy integration issues. But the data-sharing status quo is limited to high-level aggregated views of what’s happening on individual grid circuits, Hanley said -- and that’s not enough to make decisions on a customer-by-customer basis.

For example, some of the most useful data tools available today are the utilities’ Renewable Auction Mechanism (RAM) maps, he said. These have been developed to guide large-scale renewable energy project developers on which parts of the grid are open to new interconnections and which are off-limits.

“We love the platform, and we think that’s a best-in-class platform to start from to share the data,” he said. But at the same time, it’s not updated that frequently, and “it only helps for one individual project,” he said. “If you’re doing a lot of rooftops, and you don’t have one address in mind, but rather thousands of addresses, that map doesn’t help us.”

Today’s RAM maps also lack the underlying data behind their determination of which areas are off-limits, he added. “I would love to be able to go to those maps, click on a substation, and download all the data, export it into a CSV file, or what have you. SCE and San Diego [Gas & Electric] are much closer to making that happen, because they have their maps in a GIS [geographic information system] platform,” he said. “PG&E’s map is just a web interface. […] You can’t do anything but scroll around and look at it.”

Getting from this high-level view to a more granular level, with the underlying data to back it up, will be a significant challenge for utilities, according to the CPUC. The most prominent barrier that utilities face is in protecting the data privacy of customers and the operational security of the grid.

In an emailed statement, PG&E spokesperson Ari Vanrenen said the utility’s DRP proposal is “applying existing CPUC and FERC [Federal Energy Regulatory Commission] rules for protecting confidential and sensitive data, including customer-specific data, critical infrastructure data, market-sensitive data and trade secrets. We will collaborate with stakeholders on methods to anonymize and/or aggregate data that is sensitive in nature so that we can maximize the access to useful data by DER developers and customers.”

CPUC’s February guidance noted that most distribution system data isn’t expected to be subject to the privacy restrictions it’s placed on sharing customer data. At the same time, California’s smart meters collect a lot of information that could be valuable to determining the interplay of DERs with the grid -- and the information coming from those smart meters, or the Energy Services Provider Interface (ESPI) systems utilities have created to store the data, may be subject to restrictions.

For example, Southern California Edison stated in a CPUC filing (PDF) that “the ESPI Customer Data Access System is designed to transmit customer-confidential interval usage data only to customer-authorized third parties. If the third party is not directly authorized by the customer, then ESPI may not be the correct tool for interval data transfer.”

Beyond customer privacy, the most often-cited reason for withholding data is grid security, Hanley noted. Utilities are under pressure to protect the grid from terrorist attacks, whether they’re physical attacks like that perpetrated by an unknown gunman on a PG&E substation in 2013 or cyber-intrusions aimed at stealing data or disrupting utility control networks.

But Hanley said that utilities aren’t being realistic when they cite security concerns as a reason for not making distribution grid data available. “Loading data, equipment settings, location of protection equipment, etc., is irrelevant to terrorists,” he said. “They would simply identify the substations on a map and go there. This fear is a red herring.”

The real-world challenges to data-sharing: Time, money, technology and data quality

Beyond data privacy and security, CPUC has cited the “lack of availability of feeder-level monitoring data” and the “difficulty of real-time provisioning of data for operational purposes between utilities and third parties” as key challenges.

That’s a nod to the fact that California’s utilities are in the midst of upgrading their own distribution modeling software and data-collection systems, and may be struggling to integrate legacy data sources into a system that can take real-time, edge-of-network influences into account.

SDG&E, for example, stated in a January presentation to the CPUC (PDF) that it’s upgrading its SynerGEE software platform to perform more advanced analytics on the effects of DERs on its distribution network, and expects full deployment to take place in 2016 -- a year after its DRP plan is due. That’s coming from one of the utilities furthest along in incorporating DERMS, or distributed energy resource management software, into its day-to-day operations.

Likewise, PG&E said in its DRP comments (PDF) that the plan it submits on July 1 will be “based on an initial framework of methodologies that will be further developed and implemented after the DRPs and associated incremental funding are approved." In other words, it won’t be ready to give the CPUC the full level of detail it has asked for by July 1. That means that “publishing to an online map may take some extra time and be geared toward feeder-level results,” PG&E wrote, and “inclusion of capacity, protection system limits and power quality will be limited to circuit-level results and may not be achieved down to the line-section level.”

This promise of filling in the details afterwards may be the only realistic way to deal with the timeline CPUC has set. But it also leaves parties like SolarCity concerned that the deeper data they’re looking for may not be forthcoming.

“The risk -- I’m not saying this is what they’re doing -- is that they’ll either not share enough data for third parties to arrive at insights and calculations themselves, or they'll put up barriers that will not allow for analysis,” Hanley said. As a former PG&E technology and integration manager, he knows that much of this data is regularly collected and used by utilities, which means it should be easily shared with third parties, he added.

That leads Hanley to voice an all-too-common speculation for why utilities may not want to share their data -- because utility “executives understand the threat of this data-sharing on their own business models. I would say it’s a risk to monitor, whether that will have any impact on their data sharing.”

How third parties could help fill in the data picture -- and the limits to their role

So why does SolarCity want such a fine-grained view of utility distribution grids? “At a high level, we want this data because we want to be able to do a hosting capacity analysis and arrive at what the capacity is on different substations,” Hanley said. That will allow SolarCity to direct marketing and sales efforts in areas where the grid has plenty of capacity, and “as much as anything, flag areas that might be closed off to DER,” he said.

But SolarCity also wants to use this data to help utilities, he said. “If they have some capacity challenges, we can identify those zones -- perhaps quicker than utilities -- and focus our assets there, and provide tailored grid services that can help them do their jobs better.”

For example, SolarCity could identify areas with capacity constraints as targets for its energy storage partnership with Tesla, and use that storage to balance out rooftop solar production to help meet local grid needs. In both cases, “data provides the bottom-up perspective to being able to do that intelligently,” he said.

In exchange, SolarCity can use its broadband-connected monitoring systems to bring in real-time data from its customers’ sites, at latencies that utilities would be hard-pressed to match with their smart meter networks, he said.

In fact, “we’ve had a standing offer for about a year with all the utilities in California and across the country -- we’ll data-swap. We’ll give you the detailed performance data you don’t have on solar assets and customer assets, if you provide us this loading data and grid information,” he said. “To date, every utility has said no to us. That’s why we’re really excited about the DRP -- it’s forcing the utilities’ hands and allowing us to uphold our side of the bargain.”

Likewise, microinverter company Enphase has been collecting data on its installations for years now, and has presented the CPUC (PDF) with examples of how it can share that data to help utilities gain better understanding of the impacts of rooftop solar on the edge of its grid. Enphase, like SolarCity, has already been working with Hawaiian Electric Co. on putting its solar inverters to use in helping the solar-impacted utility clear backlogs for new rooftop PV interconnections.

“There’s starting to be a realization in the industry that solar is connected all the way down to the home and street level. If you’re doing high-penetration studies, you can’t do it at the substation,” Ameet Konkar, Enphase’s senior director of strategic initiatives, said. “We have been making the rounds, having conversations with all three IOUs on the DRP proceedings, to say, ‘Hey guys, here’s the data. Pick a feeder.’”

CPUC’s guidance explicitly lays out the need for a process for DER owner-operators like SolarCity to share market data with utilities, along with policies that deal with confidentiality. Beyond that, however, “I would suggest that firms on the sideline, like Google, if they were given access to the data, could figure out solutions a lot faster than the utilities,” Hanley said. “We don't think utilities have a monopoly on understanding the grid. They used to, but there’s a whole new understanding.”

This view of Silicon Valley innovation beating out utility experience doesn’t hold water with all parties, however. As SDG&E stated in its CPUC filing, “Much of the analysis and tools that the guidance document envisions for the DRPs are still in development, despite the claims made by some vendors. As an industry, we have yet to see these tools used by utilities across the U.S. to make economic and reliability decisions for the grid.”

That view was echoed by Tom Osterhus, CEO of Integral Analytics. His company’s LoadSEER software is being used by PG&E to help model and predict the combined values of different DERs across the utility’s distribution grid. These calculations require data ranging from granular feeder-level voltage, reactive power and thermal ratings, to decades-out demographic forecasting and load-growth projections.

“That can only really be done holistically by the utility,” he said. “You can’t treat each resource as a silo, because you’re going to get sub-optimal results.”

Hanley doesn’t disagree with that assessment. “Someone like SolarCity has no access to this data and can’t estimate growth nearly as well as the IOUs,” he said. On the other hand, “All of the IOUs in California, if you start bringing in real-time data from all their meters -- their traffic can’t handle that. Ours can. We can bring back as much as we want.”

Finding the money value of distributed energy resources -- and fact-checking it with shared data

Behind all these discussions of data-sharing, there’s a lot of money at stake. California’s investor-owned utilities spend roughly $6 billion per year on distribution grid investments, which are made today without regard to how they’ll be affected by new grid-edge technologies. But California could see 15 gigawatts of these distributed energy resources come on-line this decade, including 12 gigawatts of distributed solar, 1 gigawatt of grid-scale energy storage, and another gigawatt of demand response.

The first goal of California’s DRP proceeding is to find ways to “minimize overall system costs and maximize ratepayer benefit from investments in distributed resources” by creating plans that don’t ignore the effects of the state’s dramatic growth in distributed energy. But beyond that, CPUC's guidance specifically cites the need to “animate opportunities for DERs to realize benefits through the provision of grid services” -- and that means money for the companies that can turn these potentially disruptive resources into cheaper and more efficient alternatives to new utility investments.

Last year, Southern California Edison signed long-term procurement agreements for hundreds of megawatts of distributed energy resources from companies including Stem, SunPower, Advanced Microgrid Solutions, Ice Energy and NRG Energy, based on the utility’s assessment of what those resources were worth to help it meet its local and system-wide grid needs. But these calculations were done behind closed doors. The state’s DRP proceeding, by contrast, is meant to open up the data that goes into these calculations to parties on both sides of the deal.

California’s utilities, like those in most vertically integrated utility regimes, rely on investing in capital projects that serve as the basis for calculating how much they’re allowed to charge their customers in new rate cases. Replacing that capital investment with third-party assets is problematic for that business model, which is why SolarCity CTO Peter Rive has suggested that the state consider policies that would allow rate-basing the grid services that DERs could provide, in what he called an "infrastructure-as-a-service" model.

“There’s a business model question here, about what would incentivize the utilities to want to do this,” Hanley said. “Can utilities get a return on capital in the ground, as well as third-party infrastructure?”

Likewise, DER companies need data-driven valuations for the services they can provide, Enphase’s Konkar said. “Nothing exists in terms of a rate, or market structure, to allow this to happen,” he said. “This is where the DRP proceeding, where you’re more formally attributing a grid value to these distributed resources, to create a value for it, could open a market overnight.”

Because Enphase provides its equipment to installers, it also has to have data to prove to those partners that it’s offering them an equitable share of whatever rewards it can receive from providing its microinverter data and control capabilities, or its new energy storage systems, a utility service, he added.

“We have the visibility into the grid, we have the ability to control -- but we do not have the right to control,” he said. “If we can get some of our installer partners to get on board, to see there’s a value stream for them, we can say, ‘Let us provide the service value to the utility.’”

There are significant challenges for considering third-party assets as utility infrastructure, however. For one, many of today's regulations don't allow customer-owned solar, energy storage or other inverter-connected energy assets to stay running during outages and grid disruptions, which limits their value at times when it would theoretically be highest. California's Smart Inverter Working Group, a consortium that's creating advanced inverter features to become part of the state's distributed energy regime, has submitted reports to the CPUC's DRP proceeding (PDF) to lay out what data-sharing would be required to overcome those obstacles.

Beyond that, however, there's the question of how different parties get to measure the cost-effectiveness of their own systems versus those from competitors, whether that's other DER providers, or the utilities themselves. That requires some kind of impartial oversight, and "the utility has to maintain that, they have to set the parameters," Integral Analytics' Osterhus said. "If a field crew changes something, they have to benchmark and validate that. It’s only good for today -- next month it changes."

Building methodologies for creating this impartial, commonly acceptable stack of costs and benefits for different DERs on the grid has been a big part of what the More Than Smart working group has been doing for the CPUC. Paul De Martini, who's been compiling the data being collected in this process for stakeholders in the DRP proceeding (PDF), noted that this work will continue long after utilities submit their first DRP plans in July.

“What we’ve identified in California is that this starts with planning, and moves to market opportunities for DER to serve utilities,” he said. “And information is the lifeblood of this approach. You need the data to be able to do the analysis, to identify the needs, and then be transparent to the marketplace on what that need is.”