The Portal Network / Piper Merriam

Transcript

[Music] so Piper over to you all right let's see if this clicker works all right it looks like it does cool well hello everybody uh I'm here to talk to you today about a project I've been working on for a bit me and a couple of other people called the portal Network I'm Piper Miriam I work for the ethereum foundation I guess researcher might be the right title here I've done a lot of things um I've been doing ethereum stuff since the network launched and well it's been quite a ride [Applause] uh goal is at the end of this you guys have a sense for the answer to this question uh both kind of conceptually and the like what problems are we trying

to solve what problems are out there as well as just pragmatically what specifically technology wise are we building uh this is kind of my white whale I've been at this problem for like five or six years now and it's got some more to go um turns out lightweight protocol access is not easy which 2016 me had no idea about uh I thought yay I'm gonna build a light client uh around 2016 uh parody is the bed and gef is pretty much the only show in town and client diversity is a problem and I thought maybe I can fix that it went great not really

um we spent five years building a ethereum client called Trinity I was written in Python uh we knew almost nothing about the protocol when we started and we learned a lot along the way um we built and built and we ended up with something that looked kind of like Geth but was really slow and then we spent another like two years iterating on different things that we could do with les which is the the established light client protocol at the time uh ways that we could use the existing peer-to-peer protocols that all ethereum clients use and ultimately we ended up letting the boulder roll back down the hill

um the conclusion was you can't get there from there from here or if you want to put that this is a concept from a science fiction book that I read recently that I like a lot and it's just this phrase to try to capture when you were essentially trying to do something the wrong way trying to to transform a system into something that it doesn't want to turn into that sort of thing so we learned some stuff during this time and one of the big ones is that the dev P2P protocols that ethereum runs on are far too demanding uh Dev P2P is a peer-to-peer networking protocol um all ethereum execution layer ethereum clients use it to communicate and Dev

feed a p the way that execution clients use it for is Rel basically a dumb transport layer between full nodes and full nodes are Hardware intensive pieces of software this is like rough stats of what a full node looks like today you need something generally like a full terabyte of fast disk to run these things you need large amounts of ram mini cores of CPUs to run the transaction pool itself is an expensive thing to to be part of and if you want to be part of the network that execution clients are part of you have to be this tall to ride and if you aren't then we'll hear this sad

little girl at the fair um Dev P2P Demands a ton from the clients that are part of its Network um and the end result is that all of those clients end up looking about the same and they're all really heavy pieces of software to run on the other side of that we have this protocol called Les the light ethereum subprotocol and the end result after many years of developed development is that it doesn't scale well um this is what Les is like Les is a client server model again we have this dependence on full nodes this sort of fundamental Assumption of full nodes and all of the light participants in this network just get

their data from Full nodes and being a full node in this network looks something like this um you just get hammered with requests from all of the white nodes and the amount of capacity in the network is a function of how many full nodes there are so every new light node that comes online is eating up some amount of fixed capacity and being a full node in this network has no benefit to you it is purely a a cost to you and the white nodes have no ability to contribute to this protocol there they are purely takers um so after spending a bunch of time building what was effectively a copy of Geth and realizing we weren't going to

do better than being Geth and then spending a bunch of time trying to build options around in different directions of how can we make this client shape differently we ran into we came to this conclusion you can't get there from here um and so we came up with a certain set of like well if you if we if we can't build it from this what do we need to build uh one of the things that we centered on was kind of going well first of all we weren't going to build a centralized solution centralized Solutions are boring to me um go have fun building one of those if that's what you want to do uh Les fits this decentralized model but the client server model we already kind of looked at that and and thought that this isn't working well we don't want to

depend on full nodes and so we we aimed for something that fit this distributed model it it it's you know sticks with the core of our of our uh ecosystems values right we care a lot about centralization um but we wanted something that didn't have this kind of like full node light node distinction um we wanted something where we were all the nodes to the network uh uh contributed capacity a lot like BitTorrent um we aimed to to build something and we have built something that instead of each new light client coming on taking up some amount of fixed capacity um we wanted something where as we added

new nodes to the network it got more powerful um and then this is a graph that's been in a bunch of my slides because it's really what we're aiming at uh today you only get the uh decentralized heavy option or the lightweight centralized option and there really should be something in the upper yep not mirror to cool left corner of this graph and that's what we're aiming to build that's what we are building um so what is it so the portal network is a completely new set of peer-to-peer networking protocols for for ethereum um it's built on the Discovery V5 protocol for anybody who cares about

those kind of specifics there's some really nice things about that protocol that we that we like one of them is that it's connectionless it's another thing in the dev P2P world that we didn't like uh in the lib P2P world as well that they are uh lib PDP and Dev PDP are are connection based protocols you establish a connection with somebody and then you hold on to it and that itself is a little bit discriminatory towards light nodes who want to pop into the network and do something in a network where they have to establish long-lived connections they're they're already at a disadvantage so we were looking for a a protocol that was that was well suited for light nodes and the Discovery V5 protocol did that for us

um this is a distributed storage protocol we looked at what data you need to to do lightweight protocol access and then started looking at how we map that data onto it onto a distributed storage Network and the whole thing is designed from the ground up aimed at being resource friendly towards low resource devices so uh aiming for building protocols that don't have kind of these implicit height requirements these these you must be this tall to ride uh aiming for a protocol where the base level of participation uh is is accessible to Consumer level Hardware consumer level internet connections that sort of uh that sort of thing

and then I came up with this mouthful a globally available highly accessible massively Cooperative archive node that's what we're building in a mouthful of that uh globally available it's a peer-to-peer Network you can get on it from anywhere in the world you have an internet connection and that you can connect to another Peter highly accessible targeting low resource devices uh ideally you can flash a Raspberry Pi and and run it on our Network you can run it on your laptop you can run it as a Daemon process that doesn't eat up your CPU massively Cooperative we're looking for something where we bring hundreds of thousands of nodes online around the world and that capacity gives us what we need to not only source

serve all of ethereum's historical data but even archive node level State data uh we've been at this for a little bit uh we're on track to deliver a fully functional uh history Network so that's a network that serves all of the historical headers block bodies and receipts um probably in the next couple of months but betting on software delivery is risky business so by the end of this year we should have a fully functioning chain history Network we're hoping to see experimentation on top of this for eip44s um one of the things that's blocking

most of the ethereum clients from dropping all of the deep history is there isn't an alternative place for them to get it we're working on state um we're just getting started on that that's going to be a harder nastier problem but we're looking at delivery sometime towards the end of this year or Q4 uh sorry or 2024. um there's a this is a this is like a long ongoing research project we're delivering pieces as we go um and we're really just hitting our first kind of like live networks with live data where these things are really working um

I encourage you guys to join our Discord or anything like that if you're interested in getting involved we are right close to the point where we start talking to the community about turning nodes online we're not quite there yet so I'm not going to tell you to do it if you think that this is a project that you're interested in that you'd like to learn more about I'd love to get you hooked up talk to you whatever but I want to zoom out a little bit um if you remember I kind of started this way back in 2016 with this idea that I was going to fix client diversity um this project this I never want to know what to call it because it is a

project it's a multi-team project we have three different client implementations the spans Beyond just the EF this is something that is uh you know infrastructure for kind of like our entire ecosystem and lightweight protocol access is is a huge thing it is it is one of the primary things we're focused on um but I want to talk a little bit about execution layer client diversity because that's kind of the piece that's hidden under under the under the surface level here this is roughly what client diversity looks like today it's the guest show it's been the guest show for as long as it's been it shows no sign of not being the guest show anytime soon and so the answer so I

asked the question like why is it this way and and what is it that we need to do to fix this because because this is a graph that we should want to fix and the answer if like if you can come up with an answer to this question that's that that's my thesis here is that if you want to fix client diversity you have to figure out how to beat go ethereum and I'm using intentionally adversarial language here nobody wants to actually ruin go ethereum anyway we're talking about go ethereum does a really good job so you have to do a better job than go ethereum but it's more fun if I say beat them so how do you beat go ethereum well we tried this be better at being deaf than

gef is um and I think that there's some other clients that are doing similar things um and I don't really blame them for this if you remember what I said earlier the dev P2P protocol dictates a lot of functionality about what your client must do in order to be part of that Network if you make a list of what that functionality is you basically get a list of what Geth does and it turns out if you go to build a client you end up building roughly the same functionality of what gef does you build a client that walks its way through the header chain to figure out what the front of the chain is that walks its way back downloading the whole history and then uses some process to

grab a snapshot of the state and then execute blocks to keep it up to date that's what an ethereum client looks like because that's what the network basically makes you do uh these guys have been slowly gaining some market share but this graph has not changed a lot for a while um it's my thesis that if you want to beat go ethereum you need to play a different game than they're playing go ethereum has had what five six seven years to get good at being who they are like you you got a lot of ketchup to do if you're going to beat them at their own game

Aragon has come along and said we're going to play a slightly different game it's not a fundamentally different game but it is different and they have differentiated selves but they're still not quite winning right nobody is is beating that 70 that Geth has on this graph um it's my thesis that if you want to beat go ethereum you've got to play a fundamentally different game than they're playing and we'll see if I'm right but the thesis goes something like this if you want client diversity then you need diverse clients as in you need clients with a diverse feature set if all of your clients look the same then

you're just playing a zero-sum game against each other to get diverse clients you need flexible client design you need the ability to actually make your client different than the other ones it turns out our networking protocols don't really allow that and so if you want flexible client design in the ethereum ecosystem you're going to have to actually change some of the networking protocols that all of these clients live on and it turns out that in order to get buy-in for changing the network protocols you've got to get guests buy-in and I worked on trying to get them to do some of this stuff for a while repeatedly for for a while it turns out they don't care that much

and I don't mean that as any negative thing towards them their job is holding up the core of the protocol and the networking protocols that we have they serve that just fine for them get guests doing just fine with the network protocols that we have and so this is where coming in and trying to play a fundamentally different game comes in and so somewhere a couple years ago I had the bright idea of well let's build a whole new set of networking protocols so that's what we're doing the portal network is a fundamentally different game uh hence this you know you can't get there from here concept um

I tried to evolve the networking protocols that ethereum clients use into something new it wasn't a viable path to get to where we needed to go and thus building this whole new set of protocols that that come at the problem from a completely different direction yes we're going towards lightweight protocol access and light clients and functional things like that the other thing that I hope to unlock from you know up on top of this fantasival beautiful mountain with gold on top um that none of the other clients are on top of um no the the thing that I'm hoping to unlock here is the option for other ethereum clients to change what

their client looks like to develop different functionality that doesn't look like the other clients to allow them to Branch out in different directions because right now the protocols that they're all stuck on and they all use they dictate a lot about what their clients look like I'd like a world where that's not the case and I'd also like a world where we have like clients this is supposed to be the view from the top of that mountain um somewhere towards the end of this year I expect to do that Community call because one of the things if you remember we're building a network where not only do the people with big pieces

of Hardware like full nodes get to contribute to this but if you want to turn one of these on on your laptop you will actually be contributing to something tangible and valuable to ethereum's ecosystem um and we're going to hit that point somewhere towards the end of this year um hence the Little Star in this graph um that's what I got to talk to you about today anybody got any questions very very interesting Piper [Music] I think we have one question right here in the back shoot uh very easy you talked about Raspberry

Pi can you precise on which version of Raspberry Pi um we don't have hard Hardware numbers right now so I'm just gonna be really upfront that that these are targets and not things that we've measured but uh you should be able to make a meaningful you should be able to make a meaningful contribution to this network with half of a core of a CPU and a gig of RAM and 500 Megs of disk okay so I couldn't do anything by try this week without at least to The Benchmark it or to see how it works yeah I mean with like the the goal for each of these networks is no height requirements that

if you want to show up with a core of CPU and a slow DSL connection and 100 megas of disk to give us great okay thank you that's a pretty good answer who's got another question right here in the front or yeah maybe wow so is it incentivized BitTorrent style in that in order to get some service from the network you have to provide some service back or is there some other incentivization and there is no uh quality um our clients are dumb at the moment because we're we're kind of just getting off the ground so at present uh there there are no incentive systems there

there is no um there's nothing in there built for that the idea is to make the barrier to entry low enough that contributing to the network is uh disappears into the kind of Gap space in your machine where it's not really a big deal to leave one of these on in the background it's not saturating your your bandwidth it's not running your CPU at full tilt right it's it's a it's a intended to be something that is lightweight and the networks themselves are designed to allow you to tune your participation so if it is eating too much of your machine's resources and you want to turn it down

the networks are designed to have knobs so that you can turn that participation knob down um it's a thing that we're going to have to iterate on over time this is a like kind of a we're in kind of this tight iteration cycle Loop and the first goal is get something up and functional and then figure out where the where the slow parts are and attack those thank you cool and over here sink here so you mentioned this V5 as one of the protocols that you are building on is it part of the protocol stack where you are building on this grief we have built our protocol on disk V5 it's a UDP based protocol

connectionless it does exclude browsers but we do really like the connectionless aspect of it and not having to build a whole base protocol along with this was a was a nice starting point thanks all right any other questions up here around no can I ask you one question myself the team group Community working on this what how is it going are you looking for more contributors help or you have a lot of a lot of help already um I think that the kind of help that we're looking for is left the like harder development help and more of the community building soft

skills uh that sort of thing as well as people who are interested in building more like hackathon project type things on top of this um I think that's an answer okay so and as you move towards the end of the year with launch and all this kind of stuff you're gonna look for the hype the spread the word I imagine that we start looking for more help around that time okay very cool if there's no other questions thanks a lot Piper very interesting welcome good all right thank you foreign