The Stellar Consensus Protocol David Mazières Stellar Development Foundation

Tuesday, April 21, 2015

Financial infrastructure today • Financial infrastructure currently a mess of closed systems • High transaction costs - Average fees for Remittances to Africa 12%, within Africa fees reach 22% [guardian’13] • Limited reach, high latency - How to get aid to Mali? Solidarités International literally forced to send “boatfuls of cash” [guardian’15] • 2 Billion people are unbanked [WorldBank’15] - Traditional banks are unlikely ever to fill this void completely - E.g., costs banks upwards of $150 + $250/year to setup and maintain a checking account [ABA’10][AmericanBanker’11] 2 / 25

An inter-financial network

• We need new types of financial institution to innovate - Smaller, more nimble organizations can increase financial access • How will we connect new entrants to banks and to each other? - Need an inter-financial network, akin to the Internet - Must be open to any new startup organizations • How will we ensure transaction integrity? - Today: trust financial institutions and attempt to regulate them - Goal: myriad small startups; can’t trust them all

3 / 25

Transaction integrity

• Straw man: financial-network operator enforces integrity - Who appoints the network operator? - What single party would be trusted world-wide? - Imagine trying to put a single party in charge of the Internet. . . • Better approach: consensus - Participants agree on validity of one another’s transactions - Can’t roll back a transaction one everyone agreed on it - But can we realize an Internet-level consensus protocol?

4 / 25

Background: Byzantine agreement Quorum A v1

...

vN −T

...

vT

...

vN

Quorum B • System comprises N nodes any T of which constitute a quorum - Some nodes experience Byzantine failure (behave arbitrarily) • Don’t externalize (act on) any transaction until a quorum agrees • Safety if < 2T − N failures (2 quorums share honest node) - No two nodes externalize conflicting transaction histories • Liveness if ≤ N − T failures (an entirely honest quorum exists) - Means termination always possible (no blocked states) 5 / 25

Byzantine agreement (continued) • Byzantine agreement has long been a solved problem [Castro’99]

+ Low latency – by human standards - E.g., might require 5 communication rounds in common case

+ Flexible trust – N nodes can include anyone appropriate - E.g., small nonprofit can help keep big banks honest

+ Asymptotic security – based on standard cryptography - Can resist attackers with arbitrary computational power

– Centralized control – who chooses then N nodes? - Imagine if one party dictated all tier-one ISPs worldwide - Makes Byzantine agreement totally unsuitable for job 6 / 25

Background: Proof of work • Digitally sign transaction history (a.k.a. block chain) with a DMMS - Dynamic Membership Multi-party Signature - Expensive to compute, demands worldwide collaboration - Each signature compounds security of block chain • Motivate would-be attackers to participate in DMMS if rational - Reward participation with coin distribution or transaction fees

+ Completely decentralized and open membership - Bitcoin introduced it, achieved astounding impact - Clearly the missing ingredient from past consensus approaches

– Lacks benefits of Byzantine agreement - High latency, trust determined by computing power, modest computational security assumes rational attackers 7 / 25

Federated Byzantine agreement • Goal: Byzantine agreement with decentralized control • Idea: determine quorums in a decentralized way - Each node v picks one or more quorum slices - v acts on a statement only if one of its slices unanimously agrees Definition (Federated Byzantine Agreement System)

An FBAS is of a a set of nodes V and a quorum function Q, where Q(v) is the set of node v’s quorum slices. Definition (Quorum)

A quorum U ⊆ V is a set of nodes that encompasses at least on slice of each of its members. (∀v ∈ U , ∃q ∈ Q(v) such that q ⊆ U ) 8 / 25

Quorum slices v4

v2

v3

Q(v1 ) = {{v1 , v2 , v3 }} Q(v2 ) = Q(v3 ) = Q(v4 ) = {{v2 , v3 , v4 }}

v1 • Visualize quorum slice dependencies with arrows • v2 , v3 , v4 is a quorum—contains a slice of each member • v1 , v2 , v3 is a slice for v1 , but a quorum - Doesn’t contain a slice for v2 , v3 , who demand v4 ’s agreement • v1 , . . . , v4 is the smallest quorum containing v1 9 / 25

Quorum slices v4

v2

quorum for v1 , v2 , v3

v3

Q(v1 ) = {{v1 , v2 , v3 }} Q(v2 ) = Q(v3 ) = Q(v4 ) = {{v2 , v3 , v4 }}

v1 • Visualize quorum slice dependencies with arrows • v2 , v3 , v4 is a quorum—contains a slice of each member • v1 , v2 , v3 is a slice for v1 , but a quorum - Doesn’t contain a slice for v2 , v3 , who demand v4 ’s agreement • v1 , . . . , v4 is the smallest quorum containing v1 9 / 25

Quorum slices v4

v2

v3

Q(v1 ) = {{v1 , v2 , v3 }} Q(v2 ) = Q(v3 ) = Q(v4 ) = {{v2 , v3 , v4 }}

v1

quorum slice for v1 , but not a quorum

• Visualize quorum slice dependencies with arrows • v2 , v3 , v4 is a quorum—contains a slice of each member • v1 , v2 , v3 is a slice for v1 , but a quorum - Doesn’t contain a slice for v2 , v3 , who demand v4 ’s agreement • v1 , . . . , v4 is the smallest quorum containing v1 9 / 25

Quorum slices v4

v2

v3

Q(v1 ) = {{v1 , v2 , v3 }} Q(v2 ) = Q(v3 ) = Q(v4 ) = {{v2 , v3 , v4 }}

v1

quorum for v1 , . . . , v4

• Visualize quorum slice dependencies with arrows • v2 , v3 , v4 is a quorum—contains a slice of each member • v1 , v2 , v3 is a slice for v1 , but a quorum - Doesn’t contain a slice for v2 , v3 , who demand v4 ’s agreement • v1 , . . . , v4 is the smallest quorum containing v1 9 / 25

Tiered quorum slice example 3/4

v1

v2

v3

v4

Top tier: slice is three out of {v1 , v2 , v3 , v4 } (including self)

v8

Middle tier: slice is self + any two top tier nodes

2/4

v5

v6

v7 2/4

v9

v10

Leaf tier: slice is self + any two middle tier nodes

• Like the internet, no central authority appoints top tier - Don’t even require exact agreement on who is a top tier node 10 / 25

Cyclic quorum slice example v1 v6

v2  Q(vi ) = {vi , v(i mod 6)+1 }

v5

v3 v4

• Traditional Byzantine agreement requires ∀(i, j), Q(vi ) = Q(vj ) - Means no distinction between quorums and quorum slices • Federated Byzantine agreement accommodates different slices - May even have disjoint slices if you have cycles - Shouldn’t necessarily invalidate safety guarantees 11 / 25

When is safety impossible? Q(v1 ) = Q(v2 ) = Q(v3 ) = {{v1 , v2 , v3 }}

v1

v2

v4

v3

v5

v6

Q(v4 ) = Q(v5 ) = Q(v6 ) = {{v4 , v5 , v6 }}

• Suppose there are two entirely disjoint quorums - Each can make progress with no communication from the other - No way to guarantee the two externalize consistent statements • Define a term for necessary safety property Definition (Quorum intersection)

An FBAS enjoys quorum intersection when every two quorums share at least one node. 12 / 25

When about Byzantine failures? Q(v7 ) = {{v7 }} Q(v1 ) = Q(v2 ) = Q(v3 ) = {{v1 , v2 , v3 , v7 }} v2

v1

v4

v7

v3

v5

v6

Q(v4 ) = Q(v5 ) = Q(v6 ) = {{v4 , v5 , v6 , v7 }}

• Suppose two quorums intersect only at Byzantine nodes - Byzantine nodes behave arbitrarily - Can feed inconsistent data to different honest nodes - No way to guarantee safety • Revised necessary property for safety:

Need quorum intersection after deleting malicious nodes - E.g., reduces above diagram to one on previous slide 13 / 25

When is liveness impossible? 3/4

v1

v2

v3

v4

Q(v1 ) = v1 plus two of {v2 , v3 , v4 } Q(v2 ) = v2 plus two of {v1 , v3 , v4 }

• Suppose each v1 ’s slices contains a Byzantine node - Byzantine includes crashed—might not agree to anything - Impossible to guarantee liveness for v1 Definition (v-blocking)

A v-blocking set contains at least one node from each of v’s slices. • Necessary property for liveness:

Failed nodes cannot be v-blocking for any correct node v - Equivalently, correct nodes must form a quorum - Otherwise, can’t guarantee all honest nodes will remain live 14 / 25

FBAS security • A optimally safe FBAS guarantees safety for any set U of nodes iff: 1. U is unanimously well behaved, and 2. U enjoys quorum intersection after deleting all nodes not in U . • An optimally live FBAS guarantees liveness for nodes in U iff: 1. U fulfills the above requirements for safety, and 2. U is a quorum—nodes outside U are not v-blocking for any v ∈ U . Definition (intact)

Intact nodes are those that can be guaranteed safety and liveness. • Result: quorum intersection gives one maximal set of intact nodes - Without quorum intersection, must conceptually delete nodes with bad quorum slices to set security goals 15 / 25

The Stellar consensus protocol (SCP) • First FBA protocol • Guarantees safety and liveness for all intact nodes • Also guarantees safety for nodes enjoying quorum intersection - Nodes that conservatively include have large quorum slices can enjoy safety even if bad nodes are v-blocking • SCP does not optimize all axes, however: - Communication complexity (# messages/rounds) - Recovery from liveness failures (after reconfiguration) - Efficiency in the face of Byzantine network behavior • The core of the protocol is federated voting - Nodes exchange votes for statements including their quorum slices - Dynamically discover quorums while assembling votes 16 / 25

Federated voting • The core of SCP is federated voting • Each node v can vote for a statement a provided 1. a is valid and consistent with past statements v accepted, and 2. v has never voted against a and promises it never will. Definition (ratify)

A quorum Ua ratifies a statement a iff every member of Ua votes for a. A node v ratifies a iff v is a member of a quorum Ua that ratifies a. • Result: well-behaved nodes enjoying quorum intersection cannot

ratify contradictory statements • Problem: intact node v may be unable to ratify a after others do - v might have voted against a, or - Some nodes that voted for a may subsequently have failed 17 / 25

Accepting Definition (accept)

Node v accepts a consistent statement a iff either: 1. Each member of a v-blocking set claims to accept a, or 2. A quorum containing v all either voted for or accepted a. • Intuition: say a v-blocking set claims to accept statement a - Either all are lying (a is not intact), or a quorum accepted a • Result: two intact nodes cannot accept contradictory statements • Now a node can accept a statement it voted against! • But still have two problems: - Still no guarantee all intact nodes can accept a statement - We’ve weakened safety—what about non-intact nodes w. quorum intersection? 18 / 25

Confirmation

Definition (confirm)

A quorum Ua in an FBAS confirms a statement a by ratifying the statement “I accepted a.” A node confirms a iff it is in such a quorum. • Idea: Hold a second vote on the fact that the first vote succeeded - Intact nodes may vote against accepted statements - Won’t vote against the fact they were accepted • Result 1: Confirmation provides ratification safety • Result 2: Once a single intact node confirms a statement, all intact

nodes can eventually confirm it

19 / 25

Summary of voting quorum votes for/accepts a a is valid

voted a

uncommitted voted a¯

quorum confirms a

accepted a

confirmed a

v-blocking set accepts a

• Once a node v confirms a, it can safely act on a - Nodes with quorum intersection will not contradict a - If v is intact, all nodes will eventually confirm a • Given above two guarantees, we say system has agreed on a 20 / 25

System states a agreed

a-valent

bivalent

stuck

a¯-valent

a¯ agreed

• Initially, system might agree on a or its opposite a ¯ (bivalent) • At some point a quorum can no longer ratify a ¯ (a-valent) • Eventually, system may agree on a (a agreed) • But a statement can also get stuck at any point along the way - E.g., split vote prevents quorum, or nodes fail after voting 21 / 25

From voting to consensus • Ultimately Stellar wants to agree on sequence of ledgers • Can’t just vote for ledger n’s value—vote might get stuck • Instead, vote to commit or abort ballots - Each ballot is hn, xi where x is candidate ledger - Committing hn, xi choses x for the ledger • To avoid getting stuck, before voting to commit hn, xi: - Agree to abort all hn 0 , x 0 i < hn, xi such that x 0 6= x (prepare hn, xi) - If ballot hn, xi stuck, try again with hn + 1, xi • Common case of protocol is two rounds of federated voting 1. Agree to prepare a ballot (accept+confirm set of aborts) 2. Agree to commit a ballot (accept+confirm a commit) 22 / 25

Another application of SCP

google.com

google.com

• Problem: One bad certificate authority undermines TLS - E.g., turktrust issued fake google certificates [ArsTechnica’13] • Can be address by certificate transparency [RFC6962] - Agree Symantec issued certificate before Turktrust • SCP can strengthen certificate transparency, for web and email 23 / 25

Summary of properties • Summary of consensus properties: mechanism

decentralized

low latency

! ! !

maybe

Byzantine agr. proof-of-work proof-of-stake SCP

! !

flexible trust

! !

asympt. security

!

maybe

!

• But note consensus is not a cryptocurrency! - SCP does not do coin distribution - SCP has no intrinsic incentives for good behavior - Many more possibilities for trust configuration than proof-of-work, some good some not 24 / 25

Thank you

https://www.stellar.org/galaxy/ 25 / 25