Energy Web Data Exchange simplifies communication and business logic execution in complex markets by replacing heterogeneous point-to-point integrations among participants with a hub-and-spoke model in which all participants maintain a single integration to communicate with all other parties. Unlike conventional hub architectures that rely on a central broker to administer access and host the infrastructure, EW Data Exchange combines multiple open-source technologies to establish a shared platform that is owned, operated, and governed by multiple participants.

EW Data Exchange is tailored to execute high-volume, business-critical communications in electricity markets and e-mobility applications, but can be customized to support any business process involving data exchange between multiple organizations. Enterprise Data Exchange is protocol agnostic, and supports established standards including IEEE 2030.5, XML, OADR, and OCCP.

In its simplest form, Data Exchange is a secure, open-access messaging infrastructure that:

• Allows multiple parties to send, receive, and authenticate messages based on the roles that have been issued to and associated with their self-managed identity;

• Allows parties to exchange diverse datasets, ranging from real-time telemetry to bulk file uploads, in support of multiple DER and e-mobility use cases;

• Provides end-to-end encryption for all messages and data transfers, using cryptographic signatures from self-managed identities;

• Requires only a single integration mechanism with a central infrastructure in order to communicate via one:one (bilateral), one:many (broadcast), and many:many (multicast) channels.

Data Exchange can support a wide variety of use cases, including:

• Coordinating DER installation data among installers grid operators

• Streamlining DER registration in wholesale and/or local markets

• Enabling seamless roaming and advanced tariffs for electric vehicles

• Coordinating real-time grid operations between transmission and distribution system operators (operating envelopes, local constraints)

• Facilitating customer switching between retailers in deregulated markets

• Consolidating disparate market operations communications channels and processes (e.g. registration, nomination, bids/offers, dispatch, settlement)

• Establishing dynamic registries for DERs and EVs

A use case is defined by a particular configuration of a messaging channel (i.e. who is allowed to read/write, how messages are routed) and data schema (the format, frequency, and logic governing messages). Thus Data Exchange is a practically infinitely customizable solution; much like the way a road supports many types of transportation (from cars, to bicycles, scooters, trucks, etc.) Data Exchange provides a foundation for many types of business processes in electricity markets.

Who is it for?

EW Data Exchange can be applied in any energy market context where many-to-many data exchanges are critical. Companies or consortia can design how user roles are defined and acquired; what data structures and protocols are supported; and what types of integration options are made available to participants.

• Wholesale market & transmission system operators who need a solution to facilitate data exchange among market participants and distribution system operators in order.

• Distribution utilities who need to coordinate local services and non-wires alternatives with multiple vendors or aggregators.

• E-mobility service providers and charge point operators who want to reduce the cost and complexity of managing e-roaming and advanced charging programs.

• Industry consortia

EW Data Exchange allows companies seeking to offer / build / operate market-wide data exchange hubs to define the following aspects:

• Onboarding and issuing roles to organizations, systems, assets and users with DIDs, via the SSI-Hub

• Configuring role-based permissions, conditions, and restrictions for users

• Defining data structures and protocols

• Defining hosting and integration patterns (e.g. self-hosting clients, exposing APIs)

• Establishing communication channels between participants

• Scheduling jobs for batch processing and caching of data

• Use the channels for passing data between participants

Data Exchange Context Diagram

Background

Decarbonizing electric grids around the world is the single most impactful step we can take to mitigate climate change. Luckily, we are headed in the right direction: renewables and small scale clean energy assets called distributed energy resources or DERs—assets like electric vehicles, rooftop solar systems, batteries, and other flexible electric loads—are being deployed at an unprecedented rate. Unfortunately, it’s not fast enough: to achieve net-zero emissions by 2050, annual clean energy deployment needs to be three times higher than it is today through at least 2030. And if we want to get there, we have to overcome a serious obstacle: today’s electric utilities are not equipped with the tools needed to manage a renewable grid populated with millions upon millions of DERs.

The grid works by maintaining a precise balance between supply and demand. Today, utilities achieve this via a century-old model: generate power in large, centralized stations and feed it one-way to customers. This model assumes 1) utilities have direct visibility and control over generation (supply) and 2) customer demand for electricity is both passive and predictable. These axioms are no longer valid: renewable energy output is variable and largely a function of weather while demand for electricity from customers—who now own DERs capable of storing electricity, shifting when it’s used, or even injecting power back into the grid— is anything but predictable. A grid composed of vast amounts of renewables and DERs presents a complete paradigm shift for electric utilities.

Utilities have never faced a challenge like this before, nor are they equipped with the tools needed to manage this new paradigm. We aim to change this with a Digital Spine.

Digital Spine Overview

In its simplest form, a Digital Spine is a thin layer of interoperability that connects and communicates information between all of the hardware, software, and organizational systems comprising a grid in near real time. In contrast to the existing information technology landscape that utilities rely on today (which features limited information sharing between isolated and fragmented systems) a Digital Spine offers an open-access, cohesive infrastructure that is jointly governed and operated as a public good.

Today the concept of a Digital Spine - a common digital layer for transactions and interoperability for all actors and processes in an energy system - is being developed in multiple energy markets globally, most notably the UK.

Energy Web's Digital Spine toolkit includes four elements:

• Identity and Access Management (IAM): this component implements a unified authentication and authorization framework using self-managed, sovereign digital identities. This gives utilities and other grid participants the ability to mutually authenticate each other’s identity and authorize selective disclosure or communication of information based on their respective roles and responsibilities. A key benefit of this approach, in contrast to existing piecemeal systems, is delivering a “single sign on” user experience that improves interoperability and streamlines trusted integrations between devices, systems, and organizations without relying on a central administrator.

• Data and Message Exchange Module: this component is a secure, open-access messaging infrastructure that 1) allows market participants to send, receive, and authenticate messages based on the roles that have been issued to and associated with their self-managed identity; 2) allows market participants to exchange diverse datasets, ranging from real-time telemetry to bulk file uploads; and 3) requires only a single integration mechanism with a central infrastructure in order to communicate via one:one (unicast), one:many (broadcast), and many:many (multicast) channels.

• Data Hub Client Gateway: this component is an independent application that Digital Spine participants deploy in order to access the shared message broker. The Client Gateway provides a standardized interface to read and write messages in specific channels within the message broker.

• Joint Business Processing: for DERs to be fully utilized, in many instances information needs to be transmitted amongst three or more parties in a way that does not reveal all data to all parties. In these instances, Energy Web has developed an open source, decentralized technology called “Worker Nodes” that ingest data from external sources, execute custom workflows based on predefined business logic, and vote on results in order to establish consensus without revealing or modifying the underlying data. This technology borrows concepts from public distributed ledger solutions, namely distributed consensus protocols which use cryptographic techniques to establish provably correct and timely results.

Digital Spine Use Cases

Collectively, these four components provide a shared digital infrastructure that facilitates communication and coordination between utility hardware and software systems–from smart meters to network planning tools–and DERs and the companies who manage them. Ultimately, a Digital Spine exists to enable new applications provided by other software vendors and utilities themselves, including:

• Demand Response / Transactive Energy / Virtual Power Plants: Though there are many different names for it, the concept of using market mechanisms and dynamic pricing to influence DER behavior is well established globally. The Digital Spine enables distribution utilities to construct sophisticated transactive energy programs that procure grid services from DERs at specific locations and/or at specific times of day. For this use case, the role of the Spine is to facilitate data exchange and validation between the utility’s operations center and multiple third-party DER operators.

• Network Optimization via Operating Envelopes: Operating envelopes (also called Dynamic Line Ratings) are an emerging solution for dynamically modifying import (consumption) or export (generation) of DERs to the distribution network. The core function of an operating envelope is to define limits on how much power can be injected or drawn by DER based on physical constraints within the distribution system. For this use case, the role of a Digital Spine is to ingest and partition (i.e. route) operating envelopes to the appropriate DER operators so DER can safely sell services wholesale and to the distribution utility.

Moving forward, our full vision for a Digital Spine includes additional applications where consortium building and technology integrations with existing vendors is critical:

• Distribution network modeling / analytics solutions: Utilities need partners that can ingest both traditional system data and DER data to construct digital twins of entire networks in different ways and provide advanced analytical capabilities to utilities to support both operations and planning.

• Optimization solutions: Dynamic line ratings (also called operating envelopes) are one way of optimizing dispatch of DER at the distribution level. There are many other companies that offer solutions to help optimize dispatch and management of DER who can use a Digital Spine to efficiently communicate distribution network conditions and constraints to other market participants.

• Forecasting solutions: Today many companies offer forecasting capabilities (e.g. system demand, power flows within specific network lines, renewable output, etc.). A Spine can enhance forecasting capabilities by making additional datasets that are currently too complex or costly to integrate available to forecasting providers.

• Real time operations solutions: Today there are many companies offering DER management systems and advanced distribution network management systems, but they are not ubiquitous. A Spine could make it easier to bring these solutions to utilities who currently lack them.

Leading examples of Data Exchange Implementations include:

• E-Mobility Management [coming soon]

Enterprise Data Exchange is a customizable solution for authorizing, delegating, and delivering messages between a multiplicity of companies, systems, and assets operating in a common market environment without relying on a central administrator or broker. It is an extension of electricity market flexibility and e-mobility solutions developed in the Energy Web community over the last five years.

Background

Adoption of electric vehicles (EVs) is growing exponentially around the world, and EVs are on pace to represent a significant share of the global transportation sector this decade. Collectively, EVs represent an opportunity to both accelerate decarbonization in the energy sector and introduce innovative new business models. Emerging opportunities for EVs include:

1. Grid balancing and demand response: EVs can provide multiple grid services (e.g. peak shaving, voltage support, etc.) by adjusting their charging patterns based on grid conditions. Smart charging systems can optimize charging schedules to avoid overloading the grid during peak demand periods, or consume excess variable renewable energy duriong periods of low demand. By acting as demand response resources EVs can help flatten demand peaks and reduce strain on the grid, thus enhancing its efficiency and reliability.

2. Grid flexibility and storage: EVs can act as distributed energy storage units. With smart charging infrastructure and vehicle-to-grid (V2G) technology, EV batteries can be used to store excess electricity during times of high renewable energy production. This stored energy can then be fed back into the grid during periods of high demand or when renewable generation is low. V2G technology allows bidirectional power flow between the grid and EVs, enhancing the grid's flexibility and stability.

3. Accelerating deployment and increasing utilization of renewables: As a major consumer of electricity, EVs can drive demand for carbon-free electricity through clean energy tariffs, purchases of environmental attribute certificates, or even carbon-optimized charging. EV charging infrastructure can be colocated and powered directly by renewable energy installations, and EV charging strategies can be optimized to charge EVs during hours with plentiful supply of renewables.

While these opportunities are promising, fully realizing their potential is challenging due to complex relationships between multiple stakeholders, the highly distributed and diverse nature of EV and EV charging infrastructure, and the sheer volume of EVs.

EW Data Exchange builds on the capabilities first introduced in the Open Charging Network to deliver EV drivers, grid operators, charge point operators, vehicle manufacturers, retailers, and other EV service providers with a comprehensive and cohesive solution for sharing and validating data.

Reference Implementations

• Open Charging Network 2.0: A next-generation implementation of the OCN is currently under development. OCN2.0 will share many architectural features with the Digital Spine toolkit, but include several e-mobility specific features including built-in standards (e.g. OCPI) within the DDHub Client Gateway, as well as the ability to run the gateway within EV and EVSE equipment, to simplify integration between EV equipment and channels hosted in the DDHub Message Broker. OCN2.0 will also feature dedicated Worker Node networks to verify and validate message routing between participants, as well as execute custom business logic involving multiple companies (e.g. smart charging, V2G, etc.).

• Real-time Locational Green Charging: Combining the Data Exchange solution with the 24x7 Green Proofs toolkit enables advanced green charging tariffs and programs that match locally-available carbon-free generation with specific EV charging events in near real-time.

Contents

1. Prerequisite Check

   1. Configure MetaMask settings

   2. Secret Engine credentials

2. Resource Provisioning

   1. Launching Digital Spine Integration Client instance

   2. Config DSI Client

3. Client UI walk through

4. Creating Channels

   1. Creating a Publish Channel

   2. Creating a Subscribe Channel

5. Demo Integration - test Sending / Receiving data

   1. Example Post data to a Channel

6. Conclusion

1. Prerequisite Check

1.1. Configure MetaMask settings

To integrate the DSI Client with the Energy Web-hosted message broker, you will need to acquire and manage roles on Switchboard (https://switchboard.energyweb.org/). Metamask is used to log into the Switchboard dApp and sign transactions.

Go to Settings > Networks and click Add Network button to add Energy Web Chain. Enter the details below to add Energy Web Chain as a new network.

You can also use ChainList to quickly add Energy Web Chain as a new network to your MetaMask. To use this tool, please follow this link and click “Add to MetaMask” button.

• Select Energy Web Chain from the dropdown (by default, the Ethereum Mainnet will be selected) before navigating to the Switchboard dApp homepage.

1.1.a Funding Account

The Switchboard dApp does require users to use Energy Web Tokens (EWT) in order send transactions, sign messages or acquire credentials on the Energy Web Chain.

To learn how to acquire EWT in your wallet, click here

1.2. Secret Engine credentials

Azure KeyVault

Suppose you have a role-based-access-control service principle (sp) created, which is granted full Read and Write permissions to the KeyVault, your sp details may look like

{
 “appId”: “32xxx725-7xx9-46d1-b288-c552xxxxxab35",
 “displayName”: “vault-demo-sp”,
 “password”: “SLxxQ~2ul1J~4wxxNZ-o~sb65l4KuNhxxXaK9c4j”,
 “tenant”: “14e3fxx-fd1f-4exx9-a2aa-4a9xxxxxxx129"
}

You will need above details and the KeyVault uri on the resource creation page.

Learn more about

• Azure Key Vault documentation

• Azure Service principle

Hashicorp Vault

You just need

• vault server url

• The access token

2. Resource Provisioning.

2.1 Launching Digital Spine Integration Client instance

Login to Azure portal and go to ‘Marketplace’ page.

On the marketplace page, search ‘energyweb’ in the search box as shown below:

You will see the ‘Digital Spine Integration Client by EnergyWeb’ in the search results.

Click ‘Create’ on the Offer tile. It will take you to the resource creation page.

Follow the user guide on the UI to fill in the required fields for ‘Basics’ and ‘Virtual Machine Settings’.

On the ‘Digital Spine Integration Client Settings’ tab, you can set the configuration for the DSI Client.

Most configurations are pre populated, you just need to select the desired application to integrate with from the `Application Name` dropdown .

At the 'Vault Configuration' put in the secret engine details, below is the example for KeyVault.

Once all the required fields have been filled. You can proceed to ‘Review + create’ tab, you can double check all the inputs. Then click ‘Create’ to start creating the resources.

After a couple of minutes, the resource creation will complete. You'll find a Virtual Machine under the resource group.

On the VM’s overview page like below, the Digital Spine Integration Client can be accessed at the VM’s DNS in the browser.

2.2 Config DSI Client.

2.2.a Visit Client Application

Open a browser and type in the DSI VM’s DNS; you will be directed to the DSI Client landing page as shown below:

2.3 Configure DID on the DSI Client

On DSI Client landing page, enter the private key for your DID you will use for enrollment (note: the wallet address will be the DID) and hit ‘Import’.

It can take up to a minute for the DID to be verified and configured (assuming that your DID already has the role 'user'; if not, complete the enrollment form here).

If your wallet does not have sufficient balance (a minimum of 0.1 EWT is recommended) you may receive an ‘insufficient fund’ error message. To learn about transactions and transactions costs, click here; to learn how to acquire EWT in your wallet, click here.

Once you have a sufficient balance of EWT to cover the transaction cost, the application will proceed to the next stage of access validation.

2.4 Enroll the DSI Client with DID

If your DID doesn't have the required 'user' role to the application you selected for the Integration Client at resource creation step. You'll see an ‘Unauthorized’ error.

To proceed, click ‘Enroll’ to submit the enrollment request.

When the user role for your selected application is approved, you will see the status is changed to ‘waiting for role to sync’.

2.5 Manually sync role on Switchboard (dApp)

After submitting the enrollment request for your DID, the Energy Web team will contact you via email to notify you of approval. Upon receipt of the approval email, you can publish the newly issued role(s) to your DID document by following the instructions below.

In a new browser tab, visit Switchboard (https://switchboard.energyweb.org/), click on the Use MetaMask button on the welcome screen of the Switchboard dApp to log in.

The MetaMask plug-in will pop up in the top right corner of the browser and request a signature to log in.

• If you are logging in with a MetaMask account, you will be prompted to sign the message in the MetaMask Extension.

• If you are logging in with a hardware wallet account (via MetaMask), you will also be prompted to connect to your hardware wallet and sign the message on that device.

After providing the signature, you will be logged in into Switchboard dApp.

The top right menu icon has a notification. Click to open, you will find one publication available.

Click ‘publish’, on the confirmation modal click ‘CONFIRM’.

The MetaMask plug-in will pop up in the top right corner of the browser and request a signature for the transaction.

Follow along to sign the transaction in Metamask, then you'll see the ‘publish is successful ‘ notification.

When the publish is completed, you can return to the the DSI Client browser tab.

3. Client UI walk through.

At the time of revisiting the DSI Client, it should successfully redirect you to the dashboard page. If not, check to make sure your Metamask extension is connected to the correct (enrolled) account. If the correct account is active, you may need to refresh the DSI Client landing page and re-enter your private key as described in the previous section.

When you can see ‘Application refresh’ and ‘Topic Refresh’ with ‘Success’ status, you can navigate to ‘Topic management’ OR ‘Channels’ > ‘My apps and topics’ from the left navigation menu.

Your application will be shown on the list. There is one sample topic created for the application (*one is available at the time of writing, but additional topics will be added over time. You may see more topics available for your application).

If you click on the application, it will show you all the topics

You can also view the topic by clicking on it

Topic is a definition of the data schema. Which will be strictly validated during data exchanging.

You can learn more about topics here

Next we'll explain how to use the topic.

4. Creating Channels.

You can have a better understanding about Digital Spine channels here. Few summarised Key points to bear in mind.

• "Publish" type channel is for sending data and "Subscribe" type channel is used for retrieving data

• Channels can be configured to enable one:one, one:many, and/or many:many communications by changing settings for who can receive or send data in the channel scope

• Topics are used by channels, and it defines what kind of data can be send/retrieved to/from a channel, A channel can have multiple different topics

On the UI, from the left navigation menu, under ‘Channels’, navigate to the ‘Channel management’. Here is the section you manage all your channels

4.1 Creating a Publish Channel

To create a channel, simply click ‘Create’ on the page.

In the pop-up modal, you can define the channel details.

The example below shows creating a ‘Messaging’ Publish channel named ‘demo.pub’; when you have defined your channel details click ‘Next’,

At the ‘Restrictions’ tab, you are able to restrict the recipients by `DID` or ‘Role’, this will make sure only defined recipients will receive messages from this channel.

Make sure you click the ‘Save’ button after you put in the recipient DID or Role. To add multiple recipients, repeat this step as necessary.

To proceed to the next step, click “Next”.

On the next screen, you will define what topics to add to this channel.

You can select your desired application from the ‘Select Application’ dropdown, then the available topics belonging to the application will be shown in the ‘topics’ dropdown, click on the topic to add, you can select multiple topics to this channel.

At the last step you can review all the channel details; if you need to revise any settings, click back. When you have confirmed all settings, click "submit" to create the channel.

After clicking ‘submit’, The channel will be shown on the channel list.

4.2 Creating a Subscribe Channel

Creating a ‘Subscribe’ channel is exactly the same process as 'Publish' channel, but selecting the channel type ‘Subscribe’

And at the ‘Restrictions’ section, we can restrict whom we only want to receive data from.

For demonstration purposes, we restrict to only receive data from senders who has role ‘‘admin.roles.marketplace.energyweb.iam.ewc’

On the Topics tab, same concept as above publish channel, here we restrict what kind of data we want to retrieve from this channel.

And review all the details and submit our ‘Subscribe’ channel.

We should have both channels shown under ‘Channel management’

This is all about channel creating and how to implement a topic.

5. Demo Integration - test Sending / Receiving data.

The Digital Spine Integration Client has an API swagger page, which shows you the list of endpoints available for integration.

You can access the swagger integration APIs page from ‘Integration APIs’ from the left side nav.

Click ‘Open’ Rest API.

On the swagger page, scroll down to the ‘Messaging’ section

You can find detailed information about how to integrate with those endpoints.

5.1 Example Post data to a Channel

To post a message to the newly created ‘demo.pub’ channel, the example payload will be formatted as shown below:

{
    "fqcn": "demo.pub",
    "topicName": "sample_topic",
    "topicVersion": "0.0.1",
    "topicOwner": "marketplace.apps.energyweb.iam.ewc",
    "payload": "{ \"message\": \"This is my first demo message\" }"
}

You can POST above payload to the D.S.I Client’s endpoint `/api/v2/messages`

You can get all the payload info from the channel details on the D.S.I Client’s UI, click to open the 'demo.pub' channel from the channel list under ‘Channels’ > ‘Channel management’

Here are the channel and topic details

So payload info has below relationship with what is shown on the channel detail and topic detail modals.

5.2 Example Get data from a Channel

To receive data from a channel, by following the documentation, You can retrieve data from GET endpoint '/api/v2/messages'

An example request path can be /api/v2/messages?fqcn=demo.sub&amount=10&topicName=sample_topic&topicOwner=marketplace.apps.energyweb.iam.ewc&clientId=demo_user

Query breakdown will be

• fqcn = demo.sub

• amount = 10

• topicName = sample_topic

• topicOwner = marketplace.apps.energyweb.iam.ewc

• clientId = demo_user

The query information can also be retrieved from the channel detail as explained in the previous step.

6. Conclusion

Those are the main steps to get you started with the Digital Spine Integration interface, You can start communicating with other DSI clients integrated with the same application.

And contact us at digital-spine.support@energweb.org to acquire the 'topiccreator' role to create topics to test scenarios based on your business requirement.