The Portal Network: You Cannot Get There from Here

Transcript

foreign this talk is titled you can't get there from here I'm on like 36 hours awake right now so if there's some rambling or incoherence I apologize hopefully by the end this makes sense um this talk is kind of about client diversity um it's also about a project that I've been working on for a long time here we go uh that's me I work for the ethereum foundation I build things I've been doing this for a while um so what is the portal project um for anybody who's heard me talk about this before

um we're there's a little bit of naming change here you may have heard this called portal Network um switching how we're talking about this just a little bit um for me this is very much my white whale I've been at this since I started quite a few years ago um hopefully the whale doesn't eat me uh this guy this guy back in 2016 I had the bright idea that I was going to write a lightweight ethereum client um written a lot of python in my career it's where my expertise lived for a long time um and we sat down and it went great we spent five years working on this [Music]

um didn't understand the protocols that we were going to build on hadn't built peer-to-peer networking protocols at the time I just gotten done kind of writing this beautiful python evm and we get into Dev P2P Dev P2P is the peer-to-peer networking protocol that all execution clients are built upon and as we implemented the protocol what we ended up with was something that looked a lot like go ethereum it's looked a lot like Geth it turns out that when you go to implement an execution layer clients and you do so on dev P2P which is what all of the execution clients are based on

you end up with something that looks a lot like go ethereum um roughly two years into that we realized python is kind of slow and we can't keep up with block times um that we're struggling to make something lightweight um and then we spend the next two years or so kind of iterating in every direction we possibly can trying to use Les trying to use novel syncing strategies trying to figure out can we build something that's lightweight and ultimately this is the conclusion we came to um I I pulled this phrase from a from a book that I read recently called the power um I really like it it's this concept of trying to transform a system that kind

of doesn't want to be changed that is maybe stuck in a local maximum uh that's that that has to get worse before it gets better um they're being lightweight access to the ethereum protocol and here being the existing peer-to-peer infrastructure that we have today so let's talk about Dev P2P uh Dev P2P is actually a generic uh peer-to-peer networking networking protocol very similar to live P2P um I'm going to talk about it generically as if it's ethereum but there's there's these sub protocols called Ethan Les um Dev P2P there we go

it's not though oh it is it's just a different color neat all right here we go man pew there we go uh so uh the eth protocol and Dev P2P is the foundational layer for all execution ethereum clients um and at its core it is a very simple transport layer between full nodes full nodes being these guys um it has these kind of networking Primitives for syncing the state for asking for for block number whatever or the receipts from that block or things like that and then there's a couple of

things for following the head of the chain a lot of it is about freshly installed clients becoming full nodes after which they use very small much smaller subsets of the protocol but full nodes are these big heavy things I love this picture I've used it in my slides too much but I I love it um being a full note is a very big nasty heavy thing I don't run one they're too big I don't want that software on my laptop it eats the whole thing um but this isn't a fault of the execution clients this is what Dev P2P makes you do if you implement what a full node looks

like if you implement the protocol as it's designed for for the eth protocol and Dev P2P This is the End state and what we the conclusion that we came to is that full no Dev P2P is too demanding Dev PDP demands full nodes uh the dev P2P protocol is built on this fundamental assumption that you can just talk to any node at all in the network and they will be able to return any piece of this data any piece of History any piece of state that sort of thing if you want to be part of that protocol you must be that tall to ride and if you were not you were this sad little girl Dev P2P is too demanding because Dev P2P

says You must be a full node and it is unreasonable to expect normal people to operate full nodes and so this is what the world looks like in kind of the ethereum protocol today if you want to access ethereum you have roughly kind of these two choices you can do the heavy thing which you know there's some hobbyists out there that do it and if you run infrastructure you might do it but everybody else lives in this kind of like privacy sacrificing centralized brittle thing which is centralized providers these are your options effectively I'm glossing over [Music] uh Les Les is the light ethereum sub

protocol it is not delivered on lightweight protocol access um Les depends on full nodes we have this recurring theme of dependency on full nodes in ethereum's protocols and being a full node in the Les Network pew pew is like this you get hammered by all of the light clients just demanding data from you and demanding data from you and there's no benefit to you as a full node and even as a light node there's nothing you can do to contribute to that Network um Les has not delivered lightweight protocol access it is decentralized however it is decentralized around full

nodes serving all of this data you can't get there from here we spent a lot of time trying to build lightweight protocol access with the protocols that we had available to us this is the conclusion so out of all of that work we came with some design principles around what if we can't do it with these tools what tools do we need to be able to do this um this is where the portal project came from um things like inferior Alchemy those are your centralized options Les is decentralized but each one of the centers of these stars is a full node getting just hammered we we went towards this distributed

model we wanted to eliminate this kind of tiered system of full node or not full node or anything like that we wanted a homogeneous homogeneous Network where all of the people all of the participants in it are able to contribute and much like BitTorrent that the small contributions of mini add up to a large amount of network capacity we wanted we needed to build something in this corner of the graph where nothing exists something that is lightweight and decentralized and lets you actually get at the ethereum protocol so what is it uh the portal project is a new set of peer-to-peer networking protocols it's

built on the Discovery V5 protocol for those of you who care to get down in the weeds of things we took a page from the way that the beacon chain and kind of consensus specs and clients were built with a focus on understanding what kind of resource requirements are networks and data structures and everything we're going to put on our clients so that we could make sure that being a participant in these networks did not have those you must be this tall to ride sort of filters in them um we needed to build something that was friendly to be run on people's boring consumer laptops and the old Android phones and Raspberry Pi's and what the

portal project is is a set of peer-to-peer networking networking protocols that Implement a distributed storage Network for the core of the execution layer ethereum data the ability to follow the beacon chain because that's how you follow the chain uh the history of execution headers bodies and receipts and the state here's a mouthful globally available highly accessible maths massive Cooperative archive node globally available with super peer-to-peer network if you have access to like standard sending UDP packets you can be part of this highly accessible in it being designed to be friendly towards low resource devices massively

Cooperative in those low resource devices can contribute back to the resources of the network if you want to show up with 50 megabytes of hard drive space to give us wonderful and we're shooting for it being an archive node essentially being able to access arbitrary historical state current State Historical history this is like the base data set for ethereum and you can build just about everything else on top of that cats um so so where are we when can you uh when can you use this uh you can start using some pieces of it now uh we have

the history Network generally feature complete we're in the bug squashing stage teams are starting on Beacon chain uh on our kind of Beacon chain sub protocol for being able to follow that and similarly State [Music] um I don't know I should have titled this lies and projections for software delivery uh it'll be done when we get it done we're delivering stuff as we go um I just want to acknowledge that I didn't talk in any detail about the details of how we do this and how we store this stuff um that's not exactly what this talk is about if you want to talk about the details of how the networks are built I would love to do that I just don't want

to lose half the room getting into nitty-gritty details um this is what I really want to talk about because what this project is is kind of these two sides of the same thing lightweight protocol access is very much at the core of what we're going for but the piece that is kind of under the surface is execution layer client diversity which has been a show for quite a while uh this is what our execution layer Network looks like that giant blue Pac-Man eating everybody else's go ethereum this has improved over the years but not not by a lot um and I have a lot of ideas and opinions why um these are opinions um

take it or leave it but if you want to fix client diversity you have to beat go ethereum um that I'm using intentionally adversarial language here because it's more attention grabbing but nonetheless this is this is the truth of it if you want client diversity in the execution later you have to beat go ethereum so how do you do that uh one thing that we tried to do when we built our clients and that it seems some of the other clients are sort of trying to do that's not entirely fair is be better at being deaf than Geth is which is a losing game in my opinion um Geth has had years to be good at

being deaf heck they are a Geth um and it is my understanding that these two clients just look a lot like Geth it's not their fault if you remember I said we went and we implemented the dev PDP eth protocols and we got something that looked a lot like Geth that's what the network makes you do um so I think if you want to beat go ethereum you have to play a different game than them Aragon is sort of playing a different game if gethson we're here in the middle and and the portal project is going off in this direction saying let's move the complexity into the networks let's take it out of the clients

Aragon said we're going to Sprint this direction and we're going to make a really high-tech piece of software Aragon is not a lightweight client to run but I do think it differentiates itself more than the other clients from Geth they're still not winning I think if you want to beat go ethereum you have to play a fundamentally different game and this is where the rub comes I think that what we are doing with the portal project is a fundamentally different game we'll see but here's my Logic for how I get here if you want client diversity you need diverse clients as in you need clients that expose a diverse feature set different feature sets that focus on

doing different things well otherwise you're just playing a zero-sum game against the same set of users if the clients aren't different they're just all fighting for the same set of users and if you want diverse clients you have to have flexibility in how you build your client and as I said the network says You must do these things and so if you want to be able to have flexibility in how you build your client you need some networking protocol changes and it turns out that since Geth owns the network if you want networking protocol changes you have to get Geth on board with those and this just mostly doesn't happen we've been in roughly the same state for

years it has improved recently I will say that but that Pac-Man has been eating the other clients for a long time and hence you can't get there from here I don't think that you can get execution client layer diversity using the protocols that we have hence trying something new this is what the portal project is we're aiming to deliver lightweight protocol access and I think we're also looking at hoping to fix client diversity by allowing execution clients to do something different than what they've essentially been forced to do for a long time questions

[Applause] how's work going I mean I'm 40 I got divorced and in the middle of some kind of midlife crisis oh sorry this how's this going uh it's going great um uh I've got three wonderful client teams um that are engaged and excited about this thing we've been kind of hidden working on it for a while on our own um I'm cautious to approach the execution

client devs they're busy with other things and frankly I've been a little burned in the past and them not getting on board with this stuff and I understand why they're busy with their own things it is hard to run an execution client but the Project's going great we have the history Network largely up and running um it's really cool to be able to grab data out of this network and to start running nodes and seeing them scoop up data and serve it to other people and to have the thing actually working because this thing has lived as a theoretical thing in my brain for too long so what are other ways to implement this

like clients would a CK approach help in the sense that you would just have like a light node a very fine to Transitions and then the data availability problem you shoot it outside of ethereum well so first of all I don't know what you mean by living outside of ethereum because it's a little ambiguous there um in theory yes ZK stuff can help ultimately um we've just focused on kind of the Nintendo approach to things which is old boring stuff that's known to work if that makes any sense um all of the data that we serve and retrieve from the network has served in

provable formats right now instead of going for things that I didn't know and didn't understand it didn't know how to build it went with pretty basic stuff like Merkel proofs and things like that that I know how to use there's got to be so much room for improvement here but we went for something that we could run for deliver with no unknowns with the knowledge that as soon as we got there we would start iterating on it and making it better um cool so you said you have three clients currently in in progress those

are execution layered clients to be clear and like the uh these are brand new clients different we call them portal clients um I think that the distinction between an execution and a portal client is going to get fuzzier and fuzzier as time goes on we'll see um one's built by the status team it's called fluffy they're kind of our inroad to execution clients because they have Nimbus eth which is their execution client we've got one by the JavaScript team inside of the eef and that one's largely experimental but looking at things like being able to run them in a browser and fancy stuff like that and then one built by what is effectively my team in Rust which is just designed to be kind of

like this kind of core system level um I don't know what else to call that cool is the ultimate idea that they also start contributing to the decentralization of ethereum and they end up in that pie chart that you showed up there before as well um the way that I think and that I hope this affects execution client development is that when these networks are up running live serving all of that data and the cost at getting at it is just really low level Network participation that doesn't cost you much at all that that's gonna be tempting from any hopefully for many of the execution client developers

as they look at man it'd be really nice to ditch all of these old block receipts and Implement Eep 44444 however many fours are in there um and you know free up hundreds of gigs of disk space I could just fetch this data on demand from Portal that's the hope thank you all happy to talk with any of you about this [Applause]