PeerDAS, Ethereum danksharding v0

Transcript

[Music] hello everyone I'm d lion cord at ethereum I'm building the lighthouse the RAS clion in Sigma Prime thank you for coming even though it's raining and we just had lunch and uh yeah just to make it more fun I'm a lion today with a tail so today I'll be talking about scaling scaling ethereum and this really exciting feature that we're bringing very soon that's called pieras so just to make sure that we are all on the same page I'll try to bring everyone that are like really experienced with sharding people that are don't understand yet what we're

doing so first just to Define what is scaling scaling means increasing the capacity of the chain to make transactions cheaper right now as you know ethereum has very expensive fees and that restricts the type of activity to this high financialized um just basically training if we manage to scale ethereum to a very high degree we will allow other types of activity to be possible in the chain things that are more social more like social networks gaming Etc so the path that ethereum is taking the big road map goal is d scaling I love the name is such a meme but we we just we just run through it and justify why

we're doing this so so if you have been following ethereum this has been a a big shift in the road map in the last 2 3 years that's called the rollup Centric road map so we have figured out that rollups are the way to scale ethereum we would offload computation to an L2 and then let L1 just scale data with with sharding so let's justify a bit why this is the case consider ethereum Bitcoin networks that value the centralization and they achieve that by limiting the total capacity of a note that allows any participant to run nodes at home with very constrained

environments but that limits the total throughput of the chain to these very constrained notes whoops test test I can get a mic otherwise whoopsie yeah yeah it's all the audio do

[Laughter] it a electricity is gone H yeah sure pH okaye

because the lion so my my zodiac sign is lion and from there it just grew I was doing music before and my artist name was also lion yeah no for for

hello good thank you guys thank you thank you thank [Applause] you okay I hope you enjoyed the boss it was a schedule and expected so rewind back we we we're talking scalability why doing rollup centri road map and just going through why why that's the right option so theum we have this self-imposed limitation that we try to make nodes really easy to run by limiting the total amount of work they have to do but that in term limits the capacity of the chain and leads to these very expensive fees if we want to scale

we could do some strategy by some chain that I'm not going to name but we could just make what that note has to do just more resources and that scales the network that works that's a valid strategy but it sacrifices on decentralization less and less people can run notes it becomes more expensive more manwith so you tend to centralize in data centers and some very few professionalized participants they can start to then if they want to they can caliz and do bad things like 51% attacks and and like minting funds bad bad things if that happens then the social layer has to come and say hey that's not okay we don't like that and hold the

chain and do what's called a social recovery that's kind of like the da hard Fork that's really really annoying it's very expensive in terms of capital and reputation so ideally we we don't want to do that so the key to scalability is to automate automate that and we that's what rollups do rollups allow computation to be proved on chain so that invalid State transitions are impossible optimistic rollups do it with economic games and valid rups do it with cryp crazy cryptography so that solves the scalability in the computation chain but we still have data if data is not available everything breaks and here

when I say data I mean transactions like any input that goes into the rollup like could be state divs or deposit transactions so why is that a big problem for optimistic rollups if you don't know these transactions you cannot do this dispute bisecting game that we saw in the previous talk anyone could submit any s transition and it will just be valid because no one can dispute it for validity rollups that's not possible because the transition is guaranteed to be correct but you can put the rollup on a state that no one else but this malicious actor can can progress it doesn't sound that bad but it's really bad it means that if you manage to pull off this attack you can go to hey you I

will only move your money out if you give me 50% just terrible so now the role of L1 becomes to guarantee data availability so that these attacks are not possible and then you would ask okay who has to do these checks and unfortunately everyone that's that's why this is such a hard problem and what what I'm going to explain now this is what I want to make you from this presentation is this slide really understand that because that's the key of everything why we're doing sharding why the RO of centric Roat makes sense so data availability is very problematic because it's a non-uniquely attributable fault so let's let's run through this we have on the left a scenario where the

proposer is malicious it holds data and then someone in the network say a slasher comes and points hey you are malicious I'm going to slash you but then immediately later the proposer publishes the data that's case one and then case two the proposer is not malicious it publishes everything but the slasher incorrectly points that the data isn't available then you as an observer at T3 you have no way to tell these two scenarios apart so who is at fault you don't know and that's why it's called a non-uniquely attributable fault and you cannot trust anyone else on datab availability except except for

yourself so that's why everyone has to do these checks so the only way to scale data is we have to be able to convince yourself that data is available without checking all the data and that's where sampling comes in so the naive approach and what we have been doing so far is everyone downloads everything very easy scales it's safe well it doesn't scale that's the whole point point so Point number two we will not download everything we will just download some bits of the data and if those bids are available then you say well some of the data is available fine this is terrible this is not okay and it's because you

don't have to convince yourself that some of the data is available you have to convince yourself that exactly 100% is available even if a single bite becomes unavailable all these like nasty nasty attacks are possible you can you can break optimistic rollups you can break valid rollups just by holding this tiny tiny tiny bit of data so this not safe so what we're doing is idea number three probabilistic sampling over extended data so what we do is we take the data we compute a polinomial and we compute more points we extend it with with this now only having half the data

you you can reconstruct everything so sampling becomes from a game on number two of everything is available to half is available and now this is possible that's something that we can do and it's safe and very very very scalable so this new data the extended data we will not put it in blocks we put it in a blob is this something called a side car that has different properties and we expire them sooner than blocks uh so we can price them differently they have an independent fee market and this construction this is already live in ethereum and it has enabled a bunch of scalability thanks to this independent

fee market and faster expiry so what we want to get eventually is to this D sharding road map where we have blobs with this cryptography Advan sampling and very cheap but figuring out the sampling part is very very difficult but rollups rollups one scalability now things are very expensive so what we have done and is live in the network since I think March is the Denon hard Fork where we bring Protank sharding it's just introducing the cryptography The Blob space and thanks to this independent fee market and expiry they they they have better stability I think as of today blob space is already is all

still free like the fees are so low because we haven't reached capacity yet cool so so far we understand why we're doing it but now it's the time to actually do sharding Proto than sharding it's it says sharding but it has no sharding everyone has to download everything what we're going to do now is start to Shard so we're just going to drop the requirement of everyone having to download everything and what I want to show you with this picture is that the higher the sharting requirement so the more that we dilute data among all participants the harder things become it's more difficult to find P it's more difficult to defend

against attacks when you go into these like really hard regimes things are really hard to do in a way that's like really safe liess proof so initial steps we're going to go face by face the initial phase with like a moderate charting requirement this is what we call pias and saetas so just to understand you what pias means this is what we have today we have blobs and everyone downloads everything with pias we would take these blobs and extend it this is the part we were we were talking before with the kcg cryptography and people when they want to sample they would choose random columns in this grid the one selected in

red don't load them verify them and then if enough columns are available they can convince themselves that everything is available just to give you a taste when we want to go to these very high sharding regimes we going to use 2D Matrix it sounds really scary we're going to not going to do on the details but the important point you should do is here red area big here red area small that's better because it means that if you want to sample you only have to download this tiny tiny tiny bits of data and then you can make the block crazy big without breaking that rule that nodes have a constant load so to recap sharding is hard

because we have to do a lot of things that are network based have failure rates and we need to do them in a very short period of time and that's hard if we fail to accomplish the sharding requirements within this like really tight Windows the network breaks down we have safety issu ISU we can have liveness issues it's more like reor prone so coming up with a way to do this crazy sampling and everything very quickly is hard and we haven't figured out that yet so pias the key innovation of this design is accepting the complexity of sharding and doing something that's like not as ambitious and reuses existing Network components so just to go over

the flow of how it's going to work we'll have the proposer the proposer will choose some transactions in the block including blob transactions it will compute this extension that we were talking about and then chop the data into columns and distribute to the custodians we'll do that over gossip app which is a really well understood and battl tested protocol that we use today in production and then the new part would be the sampling but we will do sampling reusing again existing components that we know and love and are secure so you would find your custodians with this B5 we will overload the no ID with the initial bites having some

meaning of who has what and then once you find your custodian you query the data with the rec rest protocol which again really well tested we understand it we have been using for a while so so far so good the issue is that this initial design wanted you to complete proposer distribution to coents to samples in 4 seconds that's really really really tight and what's going to happen is that we will have some people sliding outside of this window and the people that would slide here they will be probably the solo stakers and those participants that don't have access to a high bandwidth and they will be penalized for voting the wrong

thing so this this is not great because we would be economically incentivizing people to move to Data Centers and that kind of defeats the whole purpose so a massive simplification of the design thanks to Francesco was accept that this is not okay and then we move the attestation deadline to the custodian section now from this line to here that's what we call subet sampling you subscribe columns you get them from the proposer and if you get enough of them you consider the block to be valid this is really weak compared to full down sharding but it's sufficient if the cost of the requirement is high enough so at

this point you can be convinced that the block at the network level is available but not at the local level because your queries can be linked and the proposer can send you the columns that you are expecting because you reveal that information and trick you into believing that the block that should be available it's actually not so that's why step number two you still sample but you only use that sample signal for the critical role of boarding for justification transaction confirmation and proposing and because now you have a longer window we can do that safely even though you are a solo Staker and you don't have a very fast

connection so so far Pi progress has been really good we like a bunch of ceps almost everyone we met in Africa some weeks ago and we hacked really hard on on pias and we went from really crude implementations to a working defet so that was really exciting we also decided all of these things so we finalized the spec and I think now pias is in a very solid state that we just have to to implement then these next months we're going to be refining the implementations doing more tests hope fully cleaning everything up we have to do a lot of parameter choosing there are many

trade-offs that we have to do to ensure that sampling is safe and we have a good compromise between liveness bandwidth consumption and safety and then hopefully by the end of the year we'll have that in mainnet it's very exciting because pieras has been considered for inclusion into the hard Fork as of this Thursday so that's hot news and yeah it's the giraffe Fork which just that on neota cool so yeah I'm really really bullish on pier Das I think scalability is by far the most impactful thing that we can do in ethereum today it's a democratizing force to ensure that all participants reach and pool can make use

of this chain that we allow more economic activity so we should definitely invest as much resources as we have into making that a reality and we will do that slowly progressively but still increasing decentralization step by step thank you so much thank you and thank you so much for pushing through the technical difficulties I appreciate that I wonder are there any questions since you have to extend the block um how does that play into uh storage requirements or other types of problems so

if you extend the blob you're you're actually doubling the data but because we are sharding we are dividing the amount of data so say that we have a cost of the requirement that's a factor of eight of the block then the actual sharding increasing would be would be a factor of four so if we turn the parenthesis right you as a ster should see your dis space requirement reduce wonderful I think we do have time for another question if there is one thank you for the presentation and does this also introduce new roles in

terms of you presented proposer uh custodian and so on I mean new worlds in terms of new clients no this is baked into validator or full node software or yeah everything would be baked on the current current software you you would still have to run the same software they will just be doing a few extra things so you you will really notice if we tune it right you even should not as as he was asking your this requirement hopefully should go down and your bandwidth depends on on what we tune but yeah same software same responsibilities you just have to do some extra work on the network okay thank you wonderful so a round of applause for de line thank you so much for joining us

on stage thank you for having me we going to be read for uh our next talk in a bit