Fluffy: ultra-light client for Ethereum
How can we design networks that allow for clients to show up with small amounts of data, contribute to the network, and for those contributions to be meaningful?
– Piper Merriam
We are delighted to announce that Nimbus will be joining the Ethereum Foundation’s Portal Network team as one of the launch clients for the portal network.
In a sentence, the Portal Network is an in-progress cross-team effort to re-imagine the ethereum light client, and develop a practical and functional light client experience.
To quote directly from the spec:
The “Portal Network” is an in progess effort to enable lightweight protocol access to resource constrained devices.
The term “portal” is used to indicate that these networks provide a view into the protocol but are not critical to the operation of the core Ethereum protocol.
The Portal Network will be comprised of one or more decentralized peer-to-peer networks which together provide the data and functionality necessary to expose the standard JSON-RPC API.
These networks are being specially designed to ensure that clients participating in these networks can do so with minimal expendature of networking bandwidth, CPU, RAM, and HDD resources.
The term “Portal Client” is used to describe a piece of software that participates in these networks and exposes the standard JSON-RPC API
In particular, our aim is to work together with the EF to develop a new set of Ethereum protocols alongside the existing Eth protocol that will specifically serve this new way of accessing Ethereum data.
The overarching goal is to provide a mode of operation for Ethereum that serves common usage patterns, as opposed to keeping track of the full state at all times, as current clients do.
What we are talking about building here is the perfect client for a wallet. An ultra-light client that contributes to the network and does not require syncing (i.e. is immediately usable from a fresh install or after being offline).
it shouldn't be as difficult as it sounds. i imagine most wallets will have light clients directly embedded inside. for example, @ethstatus will come integrated with an @ethnimbus light client. so there's a sense in which most users will run light clients without even knowing it. May 24, 2021
As such, one of the end goals for us is to embed this sort of client directly inside the Status app.
This has the potential to both improve the security and privacy of our users (we will no longer have any reliance on Infura), as well as improve Ethereum’s resilience by allowing each user to contribute back to the health of the network.
The Portal Network has its roots in Piper Merriam and the Trinity team’s original goal to build a lightweight client on the existing network. It was born from the realisation that the existing network simply wasn’t flexible enough for the type of client they had in mind.
In Piper’s words:
When we set out creating Trinity the goal was to build a lightweight client. After spending almost three years deep in the protocol, exploring lots of different avenues for building the type of client we wanted to exist we eventually came to the conclusion that it couldn’t be done on the existing network.
This is where the portal network project came from. Starting back at first principles for what we wanted the client to look like and then building out the network functionalities necessarily to make that happen.
Trinity is no longer going to be developed. We are working on a separate portal client called ‘Trin’ written in rust that will be one of the launch clients for the portal network.
The exisiting DevP2P LES network is designed using a client/server architecture – with light-clients as the clients, and full nodes as the servers (today, the term light-client is usually used to refer to a client of the existing DevP2P LES network).
Since this architecture puts all the load on full nodes, and full nodes are already heavy to run, most node runners don’t activate this option.
So while the current network design serves it’s original purpose well, it is severely lacking from a light-client perspective.
How do we fix this? As Piper’s Trinity team found out after close to three years of hard work, it turns out there is no simple solution. The existing network is simply not flexible enough to allow for an effective light-client design.
Fixing this requires going back to the drawing board and redesigning the core of the protocol.
A light-client friendly network must be designed in a way that is conducive to nodes being able to show up with a small amount of storage, offer it up to the network, and do their small part, rather than requiring every node to have a very high burden of responsibility (i.e. hold the entire history of the chain in memory).
In other words, such a network needs to allow light-clients to actually contribute to the network, in the sense that each additional client that joins the network should add additional capacity to the network.
Practically speaking, this means coming up with a network design that reduces the overhead to verifying the data that you request on demand as it shows up, and the baseload of passing messages around the network.
The Portal Network aims to achieve this by deconstructing the monolithic Eth protocol into 3 separate networks: Gossip (transaction and block), State, and History, with initial development focusing on the State network.
These networks will exist side by side with the Eth protocol - unlike the Eth protocol, they don’t have to be totally bulletproof, but they should work almost all of the time.
The hope is that these new networks will become more intertwined with the existing protocol over time. For example, one can imagine a world in which full clients use the history network to give node runners the option to only store the history that they care about (perhaps 200MB) instead of the whole chain (> 200GB). Similarly with the state.
In sum, this modular architecture – in which the data (state and chain history) is shared in a p2p model, and in which transactions and blocks are gossiped – allows light-clients to be selective about how much state/historic data they store and serve.
When they need to access data that they are not storing locally, they can request it ad hoc from the relevant network (in a similar manner to BitTorrent, a DHT is used for finding the requested data).
To borrow from Piper’s excellent essay – The winding road to functional light clients - part 1 – most wallets, including ours, are standardized on the JSON-RPC API.
For the Status wallet to function correctly we need the following JSON-RPC endpoints:
eth_blockNumberfor tracking the tip of the chaineth_getBalanceandeth_getTransactionCountfor account informationeth_callfor reading contract informationeth_estimateGasandeth_gasPricefor gas estimationseth_sendRawTransactionfor transaction sendingeth_getTransactionReceiptfor transactions once they’ve been mined.
If we dig down further into what is necessary to actually fulfill this functionality we get these lower level requirements:
- Access to the accounts and contract storage (State) to support
eth_call,eth_estimateGas,eth_getBalanceandeth_getTransactionCount - Access to the gossip network (Gossip) for tracking the tip of the chain and
eth_sendRawTransaction - Access to the chain history (History) for
eth_getTransactionReceipt
By allowing lightweight access to State, Gossip, and History, the Portal Network opens up the door to a wallet-embedabble light-client that satisfies these requirements, doesn’t require syncing, and doesn’t sacrifice privacy and security.
This is a huge improvement over the current state of things, where we unfortunately have to rely on Infura for certain JSON RPC calls (e.g. eth_gasPrice) as well as for sending transactions - without access to the State, we can’t serve the majority of the JSON-RPC API or send transactions, because we can’t participate in transaction gossip.
We have started work on a mode of operation for Nimbus, orginally called nlpn but recently renamed to fluffy, that will exist side-by-side with our Ethereum 1 client.
fluffy will enable the nimbus-eth1 client to operate as an ultra-light node in the network, in addition to the existing modes that are being developed.
Work initiated on the Portal Wire protocol - a subprotocol build on top of Node Discovery v5.1 protocol.
Basic support for this protocol is implemented and a couple of weeks ago we successfully completed handshakes with other clients, including ddht (a Portal client prototype written in Python) and the Trin client.
The next step is to start serving data through the Portal Wire protocol. We are tackling state data first.
This will require bridge nodes to help pipe state data into the Portal Network: Work has already started on a Nethermind plugin that serves a custom JSON-RPC API to provide this data to any Portal node that wants to act as a bridge node (see the draft specification here).
Eventually our light-clients should support a (subset) of the Ethereum JSON-RPC API so that wallets can integrate directly with them.
- The Nimbus Light Portal Network client (nlpn) can be found in our nimbus-eth1 repository: https://github.com/status-im/nimbus-eth1/tree/master/fluffy
- The Portal Wire protocol is added to the
nim-ethrepository, as Node Discovery v5.1 sub-protocol: https://github.com/status-im/nim-eth - Specifications: https://github.com/ethereum/stateless-ethereum-specs/
- Website: https://www.ethportal.net/
- Some quick notes regarding first Portal Wire interop tests with ddht and trin. https://gist.github.com/kdeme/36795f5deae7d02ce1785e9c7d501e53
- The winding road to functional light clients blog post series of Piper Merriam
- This video talk about the subject.
N.B Conveniently, all of the infrastructure necessary to realise functional light clients naturally extends to stateless clients as well, so there’s a lot of cross-over with stateless Ethereum. In fact, allowing stateless clients to serve the vast majority of the JSON-RPC APIs is a core part of the Portal Network’s overarching motivation.
If you want to help contribute towards the design and implementation, join the ongoing discussion happening on the #portal-network channel over on Ethereum R&D discord server. There is also a weekly call, Tuesdays @ 3PM UTC, where we discuss the latest developments and open questions.
If you’d like to get involved with Nimbus specific development, we encourage you to join our discord here.
All images in this post are credited to Piper Merriam - they are taken from his wonderful talk: Democratizing Ethereum / Ethereum Core Protocols: Breaking Down the Monolith
