This document outlines a data model and syntax for expressing a set of ordered events in a decentralized system in way that can be cryptographically verified. This technology is useful when coordinating the recording of events, such as financial transactions, transfer of property, or time-stamped data that must be shared among coordinating parties. A primary goal of this ledger format is flexibility, allowing the pluggability of consensus algorithms, data structures for recording history, and the type of data that can be stored in the ledger.

There are a number of ways that one may participate in the development of this specification:

Introduction

Decentralized systems often need to coordinate information on their operational state. While the entire state of the system doesn't need to be shared, the parts that affect the operation of the entire system are important. At their most basic, a decentralized system can be viewed as a state machine where events modify the state of the machine resulting in a new state:

A decentralized system can be viewed as a state machine where events transition the machine from one state to the next.

There may be many different types of events in the system. Some events may handle system configuration, defining what the system is and isn't allowed to do. Some events may store data in the system that must be accessible by the entire system while other events snapshot the state of the system for performance and auditing purposes. Other events record the consensus of the system as a whole at a given point in time so that participants in the system can know that other participants agree with their view of the system.

There may be many different types of events that trigger state changes in a decentralized system.

Each event may affect the global system state in some way. This means that the current state is a representation of all the states and events that came before it.

As the system processes each type of event, its underlying state changes.

In certain decentralized systems, like financial or healthcare systems, auditing these states and events is important to ensure that they are operating in an acceptable way from a regulatory perspective. Thus a verifiable record of these events and states is an important component of the system design, but has not been a practical endeavor until recently.

Due to recent advances in cryptography and decentralized systems design a new set of technologies called cryptographic ledgers have enabled the recording of state and events in a ledger where the ledger can be distributed and independently verified using cryptography. Some of these ledger systems are trustless, meaning that no trust (other than a trust in mathematics) is necessary in order to participate in the system. Other ledger systems are permissioned, meaning that some trust in third parties is required to ensure the accuracy of the ledger.

A cryptographic ledger is a log of states and events that can be independently verified using cryptography.

This specification details an extensible data model and syntax for expressing cryptographic ledgers. The data model in this specification is designed to be protocol-agnostic and flexible, supporting pluggability of consensus algorithms, various data structures for recording history, and enabling a wide range of data types to be stored in the ledger.

About this Document

This document is a detailed specification for ...

How this Document is Organized

This document is organized as follows:

Terminology

cryptographic ledger
A method of recording changes in a state machine where the accuracy of the ledger can be mathematically proven to be correct.
decentralized system
A system in which lower level components operate on local information to accomplish global goals. For example, an ant colony is a decentralized system as there is no central control determining when food must be collected or how new tunnels should be excavated.
genesis event
The first entry in a ledger.
state machine
A system that when given a particular input changes from one internal state to another internal state. For example, a computer system that operates a light bulb starts with the light state set to off. When an input to turn the light on is given, the internal computer system state sets the light state to on, thus illuminating the light bulb.

The Data Model

To provide the greatest amount of flexibility, the data model is a graph. If a formalized data model is desired, RDF may be utilized but is not necessary.

A ledger consists of a series of entries. This section outlines the basic types of entries that all Web Ledgers support. A particular Web Ledger MAY contain more than these basic entries.

Configuration Event

A configuration event specifies which software algorithms should be applied when processing a particular Web Ledger. The first entry in a Ledger is typically an entry called a genesis event. The genesis event has the following basic layout: 

{
  "@context": "https://w3id.org/flex/v1",
  "id": EVENT_ID,
  "type": "LedgerConfigurationEvent",
  "ledgerConfig": {
    "id": LEDGER_ID,
    "type": "LedgerConfiguration",
    "name": "example",
    "description": "This is an example ledger.",
    "storageMechanism": STORAGE_DATA_STRUCTURE,
    "consensusAlgorithm": CONSENSUS_ALGORITHM,
  "previousEvent": {
    "hash": "urn:sha256:0000000000000000000000000000000000000000000000000000000000000000"
  },
  "signature": SIGNATURE_VALUE
}

In the example above, LEDGER_ID is used to uniquely identify the ledger (e.g. did:f6ea280f-8011-4502-a29f-464954de3427). EVENT_ID is used to uniquely identify the event in the ledger (e.g. did:f6ea280f-8011-4502-a29f-464954de3427/events/1. STORAGE_DATA_STRUCTURE is used to identify the storage mechanism that is used in the ledger (e.g. SequentialList) so that serializing and deserializing the contents of the ledger remains consistent across ledgers. SIGNATURE_VALUE is used to perform the cryptographic proof that the ledger entry was created by the entity identified in the signature. A more complete example of a genesis event is available in the appendix.

A standard configuration entry only differs from a genesis event in that the previousEvent value refers to the event before the current event: 

{
  "@context": "https://w3id.org/flex/v1",
  "id": EVENT_ID,
  "type": "LedgerConfigurationEvent",
  "ledgerConfig": {
    "id": LEDGER_ID,
    "type": "LedgerConfiguration",
    "name": "example",
    "description": "This is an example ledger.",
    "storageMechanism": STORAGE_DATA_STRUCTURE,
    "consensusAlgorithm": CONSENSUS_ALGORITHM,
  "previousEvent": {
    "id": PREVIOUS_EVENT_ID,
    "hash": PREVIOUS_EVENT_HASH
  },
  "signature": SIGNATURE_VALUE
}

The PREVIOUS_EVENT_ID is the identifier of the previous event in the ledger (e.g. did:f6ea280f-8011-4502-a29f-464954de3427/events/1) and the PREVIOUS_EVENT_HASH value (e.g. urn:sha256:abd465d34f7a3f0f7d849550eb9fc32c17d12881da6b524da0a96e12cc984538 is a hash of the previous event in the ledger.

Storage Event

A storage event stores data in a ledger and has the following general layout: 

{
  "@context": ["https://w3id.org/flex/v1", MARKET_VERTICAL_CONTEXT],
  "id": EVENT_ID,
  "type": "LedgerStorageEvent",
  "replacesObject": [ UPDATE_OBJECTS ],
  "previousEvent": {
    "id": PREVIOUS_EVENT_ID
    "hash": PREVIOUS_EVENT_HASH
  },
  "signature": SIGNATURE_VALUE
}

When storing data in the ledger, one may use a MARKET_VERTICAL_CONTEXT to add semantics to the objects stored in UPDATE_OBJECTS (e.g. https://w3id.org/vaccinations/v1). UPDATE_OBJECTS are one or more JSON objects that can be stored in the ledger. It is expected that each object has a unique identifier to enable searching based on that identifier. A detailed example of a storage event is included in the appendix.

Consensus Event

It's currently not specified in a configuration event how often consensus events should occur. The options are: per event, or every N events. This is an active area of research and debate.

A consensus event is an assertion that there is agreement on a subset of entries in a ledger. Some ledgers do not require consensus events as each event establishes an acceptable level of consensus. Other ledgers require consensus events after a pre-determined amount of time. An example of the basic layout of a consensus event is shown below: 

{
  "@context": "https://w3id.org/flex/v1",
  "id": EVENT_ID,
  "type": "ConsensusEvent",
  "previousEvent": {
    "id": PREVIOUS_EVENT_ID
    "hash": PREVIOUS_EVENT_HASH
  },
  "signature": [ SIGNATURES ]
}

A consensus event has a relatively simple definition. It consists of a number of SIGNATURES from notaries as determined by the previous configuration event. A more complete example of a consensus event is located in the appendix.

Checkpoint Event

A checkpoint event may be used to quickly bootstrap new mirrors for a ledger. An example of the basic layout of a checkpoint event is shown below: 

{
  "@context": "https://w3id.org/flex/v1",
  "id": EVENT_ID,
  "type": "LedgerCheckpointEvent",
  "checkpointLog": CHECKPOINT_LOG_URL,
  "checkpointLogHash": CHECKPOINT_LOG_HASH,
  "signature": [ SIGNATURES ]
}

A checkpoint event declares a CHECKPOINT_LOG_URL (e.g. https://example.org/ledgers/example/checkpoints/132) and a CHECKPOINT_LOG_HASH (e.g. urn:sha256:7fa3b9eaa8d92d2b87abf83d88a92ff23). All notaries typically download the checkpoint log and check it before providing signatures noting that the checkpoint log is valid. There is a detailed example of a checkpoint event in the appendix.

How checkpoints are generated are a current topic of debate. The configuration event doesn't specify how often checkpoints can/should happen. It is also debatable that a checkpoint should be included in a consensus event and shouldn't be a separate event.

The HTTP API

The Suggested Syntax

To ensure that the syntax will be easily adopted by developers, and supports a graph-based data model, it is RECOMMENDED that the implementation syntax is JSON-LD (or similarly compatible syntax). Other RDF-compatible syntaxes MAY also be used to express the ledger semantics.

Examples

Configuring the Ledger

{
  "@context": "https://w3id.org/flex/v1",
  "id": "did:f6ea280f-8011-4502-a29f-464954de3427/events/1",
  "type": "LedgerConfigurationEvent",
  "ledgerConfig": {
    "id": "did:f6ea280f-8011-4502-a29f-464954de3427",
    "type": "LedgerConfiguration",
    "name": "example",
    "description": "example ledger",
    "storageMechanism": "SequentialList",
    "consensusAlgorithm": {
      "type": "ProofOfSignature2016",
      "approvedSigner": [
        "https://blue.example.com/keys/j37f8",
        "https://red.example.com/keys/83hfk",
      ],
      "minimumSignaturesRequired":1
    }
  },
  "previousEvent": {
    "hash": "urn:sha256:0000000000000000000000000000000000000000000000000000000000000000"
  },
  "signature": {
    "type": "LinkedDataSignature2015",
    "created": "2016-10-05T11:47:26Z",
    "creator": "http://blue.example.com/keys/j37f8",
    "signatureValue": "RG+t2EbnMKr4zIwgncLHTb/JvAZ/EIXmTXrrad7DBdlchKPPC9j91N793eHY5UYx4zx7PTDnU2HlW9zqQD1EMBwI7TtqE7BfvxOFodjKJ/Aw16vpAmw2h2Jo9NBu4BvFWsLPCiMLHuW7Vxo7XDkGVfaZUqk/X2q/xpRmqJ1tOlNgivUH8bIm/TUAG3g90DMN9s5zImVyB4B20zMJk8K50BbpHYKUVNWCaHnv9F4VvUUa+AdNeNlC8W1F5j2WChyXzd+51bervxQErZUDFyocW/xEYPvJXPSGnL0ARBDggKQDe1M3bg8w7GnMGrJBieNZAuK2mxOtF2rOPDU3oMk3ZQ=="
  }
}
    
{
  "@context": "https://w3id.org/flex/v1",
  "id": "https://example.us.gov/ledgers/emergency-response/1",
  "type": "LedgerConfigurationEvent",
  "storageMechanism": "SequentialList",
  "consensusAlgorithm": {
    "type": "ProofOfSignature2016",
    "approvedSigner": [
      "https://fema.example.us.gov/i/ofp",
      "https://tsa.example.us.gov/i/vclaims",
      "https://dhs.example.us.gov/i/credentialing",
    ],
    "minimumSignaturesRequired": 1
  },
  "nextEvent": "https://example.us.gov/ledgers/emergency-response/2",
  "signature": {
    "type": "LinkedDataSignature2016",
    "created": "2016-09-19T17:19:52Z",
    "creator": "https://fema.example.gov/i/credentialing/keys/7f6",
    "signatureValue": "pjeyuFq9idhwHGfGFzv...9fjsk=="
  }
}
    
{
  "@context": [
    "https://w3id.org/flex/v1",
    "https://w3id.org/cryptocurrencies/v1"
  ],
  "id": "bytecoin:block:23fa73bb201c738d",
  "type": "LedgerConfigurationEvent",
  "storageMechanism": "MerkleTree",
  "consensusAlgorithm": {
    "type": "ProofOfWork2016",
    "proofOfWorkAlgorithm": "Bitcoin",
    "targetDifficulty": "199312067531.243"
  },
  "signature": {
    "type": "LinkedDataSignature2016",
    "created": "2017-01-24T02:10:21Z",
    "creator": "bytecoin:keys:23f3b2c8183bdd52aff689",
    "signatureValue": "90D4cFqT/yKPiwd/GFzv...NJGL=="
  }
}
    
{
  "@context": [
    "https://w3id.org/flex/v1",
    "https://w3id.org/cryptocurrencies/v1"
  ],
  "id": "https://example-consortium.com/private-ledgers/loans/real-estate/1",
  "type": "LedgerConfigurationEvent",
  "storageMechanism": "SequentialList",
  "consensusAlgorithm": {
    "type": "ProofOfBallot2015",
    "minimumQuorumPercentage": "0.51",
    "minimumVotePercentage": "0.67",
    "approvedSigner": [
      "https://regulator-a.gov/i/bank-oversight",
      "https://regulator-b.gov/i/consumer-protection",
      "https://bank-x.com/i/housing",
      "https://bank-y.com/i/corporate-loans",
      "https://bank-z.com/i/swaps",
    ]
  },
  "signature": [{
    "type": "LinkedDataSignature2016",
    "created": "2018-04-16T19:34:18Z",
    "creator": "https://regulator-b.gov/i/consumer-protection/keys/2f73",
    "signatureValue": "Piw90D4cFqT/yKd/JGLv...NGFz=="
  }, ... // there will be at least 3 more signatures here
  ]
}

The use of nextEvent is unlikely here, it is more likely to be prevEvent and potentially include a hash of that event depending on configuration.

Storing Data in the Ledger

{
  "@context": ["https://w3id.org/flex/v1", "https://w3id.org/dhs/v1"],
  "id": "did:f6ea280f-8011-4502-a29f-464954de3427/events/2",
  "type": "LedgerStorageEvent",
  "replacesObject": [{
    "id": "https://hhs.gov/vaccinations/6080026",
    "type": ["Credential", "VaccinationCredential"],
    "claim": {
      "id": "did:e277ce82-bf9d-413a-a6fe-fde877fd19b1",
      "vaccination": {
        "type": "IPV",
        "vaccinationDate": "2004-07-17",
        "vaccinationProvider": "Alameda County Hospital, California, USA"
      }
    }
  }],
  "previousEvent": {
    "id": "did:f6ea280f-8011-4502-a29f-464954de3427/events/1",
    "hash": "urn:sha256:f04524659baa027ea49e7d0635bc6b1869d3e7d811f6bfefeb3aae1adcae0362"
  },
  "signature": {
    "type": "LinkedDataSignature2015",
    "created": "2016-10-12T18:37:27Z",
    "creator": "http://dhs2016ledger.digitalbazaar.com/keys/fema-key-1",
    "signatureValue": "ITv0b9tTIjgBMgu4jUOsuU9Y2t9rsq+rKrbJHfcKWmDQZE1X6VpqKowv3l27WN0w5NyECIkJx6oy94nlM375ivmhQf6iYd6R7IizVBiRiLl3HeNXdaxBpyYMn6OiP8MYVdJt2g5vj+xhP9AadXnRL0TYmE6qH1LtOw42SQtfseQDb8xF1Bsjie7KmKK/G+B1yoKqg1L34NLSAnHpnqRGAen9OEMi0hwBrVD8HIRFkdCehh4T07HRrZe2mwhfY0gOA54j02qWZxidvYUYAV8vax4AhFyk0Vsaibo9yt85uU8Tc8a4e5hjS7nCAya8FMzDd0xEOp1q9MeC5vgJHKz29g=="
  }
}
    
{
  "@context": [
    "https://w3id.org/flex/v1",
    "https://w3id.org/housing/v1",
  ],
  "id": "https://vhda.va.us.gov/ledgers/webville/houses/2",
  "type": "LedgerStorageEvent",
  "previousEvent": "https://example-consortium.com/private-ledgers/loans/real-estate/1",
  "replacesObject": [{
    "id": "https://vhda.va.us.gov/properties/3829344",
    "propertyAddress": {
      "street": "263 Main Street",
      "locality": "Webville",
      "region": "VA",
      "postalCode": "24736-3726",
      "country": "US"
    },
    "owner": {
      "name": "Jane Smith",
      "postalAddress": { ... }
  }],
  "signature": {
    "type": "LinkedDataSignature2016",
    "created": "2016-02-22T02:10:21Z",
    "creator": "https://webville.va.us.gov/i/planning-department/keys/1",
    "signatureValue": "cNJGLFqT/d/90D4GFzv...yKPiw=="
  }
}
    
{
  "@context": [
    "https://w3id.org/flex/v1",
    "https://w3id.org/dhs/credentials/v1",
  ],
  "id": "https://example.us.gov/ledgers/emergency-response/2",
  "type": "LedgerStorageEvent",
  "previousEvent": "https://example.us.gov/ledgers/emergency-response/1",
  "replacesObject": [{
    "id": "https://example.us.gov/credentials/234234542",
    "type": ["Credential", "EmergencyResponseCredential"]
    "claim": {
      "id": "did:3f32-23425-53-241442",
      "emsLicense": {
        "id": "FF-37-48573",
        "status": "valid"
      },
  }],
  "signature": {
    "type": "LinkedDataSignature2016",
    "created": "2016-09-19T17:19:52Z",
    "creator": "https://fema.example.gov/i/credentialing/keys/7f6",
    "signatureValue": "pjeyuFq9idhwHGfGFzv...9fjsk=="
  }
}
    
{
  "@context": [
    "https://w3id.org/flex/v1",
    "https://w3id.org/cryptocurrencies/v1"
  ],
  "id": "bytecoin:block:83ba663df28372718",
  "type": "LedgerStorageEvent",
  "previousEvent": "bytecoin:block:271863df2883ba637",
  "addsObject": [{
    "transaction": [{
      "https://w3id.org/bytecoin/v1"
      "source": "bytecoin:2892316f63778",
      "destination": "bytecoin:983bba526fd6",
      "input": [...], // the inputs to the transaction
      "output": [...], // the outputs from the transaction
      "transfer": {
        "amount": "0.3928621",
        "currency": "bytecoin:currency:bco"
      }
    }], ... // there could be hundreds to thousands more of these
  }]
  "signature": {
    "type": "LinkedDataSignature2016",
    "created": "2017-01-24T02:11:16Z",
    "creator": "bytecoin:key:23fa73bb201c738d",
    "signatureValue": "90D4cFqT/yKPiwd/GFzv...NJGL=="
  }
}
    
{
  "@context": [
    "https://w3id.org/flex/v1",
    "https://w3id.org/real-estate/v1"
  ],
  "id": "https://example-consortium.com/private-ledgers/loans/real-estate/2",
  "type": "LedgerStorageEvent",
  "previousEvent": "https://example-consortium.com/private-ledgers/loans/real-estate/1",
  "addsObject": [{
    "id": "https://example-consortium.com/private-ledgers/loans/real-estate/transactions/238947234",
    "transaction": {
      "source": "lei:HB7FFAZIO0MZ8PP8OE26", // compliant to ISO 17442 standard
      "destination": "lei:PP8OEHB7FFAZIO0MZ826",
      "transfer": {
        "loanId": "lei:FAZIO0PP8OEHB7FMZ826"
      }
    }
  }],
  "signature": [{
    "type": "LinkedDataSignature2016",
    "created": "2018-04-16T19:34:18Z",
    "creator": "https://regulator-b.gov/i/consumer-protection/keys/2f73",
    "signatureValue": "Piw90D4cFqT/yKd/JGLv...NGFz=="
  }, ... // there will be at least 3 more signatures here
  ]
}

Recording Consensus

{
  "@context": "https://w3id.org/flex/v1",
  "id": "did:f6ea280f-8011-4502-a29f-464954de3427/events/100",
  "type": "ConsensusEvent",
  "previousEvent": {
    "id": "did:f6ea280f-8011-4502-a29f-464954de3427/events/99",
    "hash": "urn:sha256:0877a494ce1705117f917a3af3fc48190b1fdbca04fdba8e995fd6c15fb2fe71"
  },
  "signature": [{
    "type": "LinkedDataSignature2015",
    "created": "2016-10-12T19:54:12Z",
    "creator": "https://notary1.example.org/keys/124",
    "signatureValue": "skeJ4...lBUiI858="
  }, {
    "type": "LinkedDataSignature2015",
    "created": "2016-10-12T19:54:32Z",
    "creator": "https://notary2.example.org/keys/f31",
    "signatureValue": "iamOjOGncO...gRXfsG0="
  }, {
    "type": "LinkedDataSignature2015",
    "created": "2016-10-12T19:54:56Z",
    "creator": "https://notary3.example.org/keys/35",
    "signatureValue": "cPehH...iI858="
  }, {
    "type": "LinkedDataSignature2015",
    "created": "2016-10-12T19:54:23Z",
    "creator": "https://notary4.example.org/keys/ugref2",
    "signatureValue": "hMaD22...ONwzsgo="
  }]
}

Checkpointing Ledger State

A checkpoint log is an entry in the ledger that

Don't really know if checkpoints are /that/ useful without prevId entries. Don't really know if checkpoints are that useful at all? Seems to be the only use is a link to a compressed/checksummed file? 

{
  "@context": "https://w3id.org/flex/v1",
  "id": "https://example.org/ledgers/example/6372463",
  "type": "LedgerCheckpointEvent",
  "checkpointLog": "https://example.org/ledgers/example/checkpoints/132",
  "checkpointLogHash": "urn:sha256:7fa3b9eaa8d92d2b87abf83d88a92ff23",
  "signature": [{
    "type": "LinkedDataSignature2016",
    "created": "2018-04-16T19:34:18Z",
    "creator": "https://example.org/i/master/keys/32",
    "signatureValue": "Piw90D4cFqT/yKd/JGLv...NGFz=="
  }, ... // there may be multiple counter-signatures here
  ]
}

Linking to Other Ledgers

{
  "@context": [
    "https://w3id.org/flex/v1",
    "https://w3id.org/cryptocurrencies/v1"
  ],
  "id": "https://example.org/ledgers/example/5",
  "type": "LedgerStorageEvent",
  "previousEvent": "https://example.org/ledgers/example/4",
  "addsObject": [{
    "transaction": [{
      "source": "https://example.org/accounts/jane/7",
      "destination": "https://foo.com/accounts/bob/3",
      "remoteLedger": "https://foo.com/ledgers/blah/3445",
      "transfer": {
        "amount": "23.45",
        "currency": "USD"
      }
    }], ... // there could be hundreds to thousands more of these
  }]
  "signature": {
    "type": "LinkedDataSignature2016",
    "created": "2017-01-24T02:11:16Z",
    "creator": "bytecoin:key:23fa73bb201c738d",
    "signatureValue": "90D4cFqT/yKPiwd/GFzv...NJGL=="
  }
}

Acknowledgements

This specification, a part of the "Credentials on Public/Private Linked Ledgers" project, has been funded in part by the United States Department of Homeland Security's Science and Technology Directorate under contract HSHQDC-16-C-00058. The content of this specification does not necessarily reflect the position or the policy of the U.S. Government and no official endorsement should be inferred.

The editor would like to thank the Web Payments Community Group

Thanks to the following individuals, in order of their first name, for their input on the specification: ...