ep 24 – Of Data Availability & Danksharding

Transcript

[Music] hi everyone Robert Hackett back here again with another episode for web 3 with a16z I recently chatted live with some of our researchers on the topic of data availability sampling and dang sharding which is relevant to blockchain scaling as well as Paving the way for more advanced blockchain networks and user applications while much of the discussion is especially relevant to ethereum some of the concepts we cover are also applicable to advances in Computing and networking generally this discussion involves some specific math which we cover and briefly explain in the episode for quick context as you listen polynomial commitments which you'll hear

more about are a tool that helps reduce the amount of data needed to verify complex computations an interpolation another term you'll be hearing is a way to reconstruct data from a limited set of data points be sure to check out the paper referenced in this episode for a deeper explanation at a16zcrypto.com d-a-s that's Das for data availability sampling as always none of the following is investment business legal or tax advice see a16z.com disclosures for more important information including a link to a list of our investments okay so today we're going to be talking

about data availability sampling and dank sharding now if you're unfamiliar with those things don't be scared because I'm here with a couple of experts who are going to break things down for you we've got Dan Binet Stanford Professor eminent cryptographer and a16z crypto research advisor and Lara nikolenko research partner at a16c crypto great to be here thanks Robert thanks Robert great to be here thanks Lara thanks Dan so as mentioned Dan and Lara recently wrote up a great post and have a proposal related to ways to improve proto-dink sharding which is this upgrade that is planned for later this year in ethereum there is a lot of Rich detail in that post so we'll dig in

there but before we get into all the details I thought maybe we could zoom out and back up and start with a bigger picture here so let's start with Dan help us understand the subject of today's Talk data availability sampling and maybe you could do it without breaking anyone's Minds here maybe we could keep it as you know succinct and accessible to a broader audience as you might be able to sure so um maybe we can zoom out a little bit before we talk about data availability sampling maybe let's talk about the general goal here so this is part of the efforts to scale ethereum and so one of the ways to scale ethereum is using Roll-Ups and Roll-Ups actually

need to push a lot of data on chain so our rollup allows you to take say a hundred or a thousand transactions and process them all as a single transaction on the ethereum layer one and then all the data associated with those transactions basically gets pushed on chain and as a result there's a need to actually store quite a lot of data on chain and the question is how to do that and that's exactly where the Eng sharding comes into play so it's a really beautiful beautiful idea it came from the ethereum foundation particularly from Duncan advice it's a really elegant construction and basically Lara and I wanted to give an overview of how this construction works and potentially look at some options in which it may be can be improved a little

bit yeah Roll-Ups essentially allow execution to scale for ethereum but the question is how to make the data scale and then sharding that availability sampling basically adds this missing piece to ethereum that will allow to achieve full scaling this post is kind of quite technical some aspects we don't explore like networking which is also interesting to research and to write about but we're mainly focusing on kind of cryptographic aspects of index charting and I would actually even add that there's really beautiful open questions that are still left to think about for researchers if you are interested in this area there's like beautiful questions around coding and polynomial commitments they're really quite interesting questions if

they could be resolved we would end up with even more efficient systems that's excellent context I definitely want to ask you a little bit about those open areas for potential inquiry in a little bit but before we do that let's talk a little bit more about this proposal that would free up space on the ethereum blockchain so the blockchain is basically this giant record of transactions happening from the time that this system launched a way back when to the Genesis block how are developers thinking about freeing up space making it get greater throughput scalability cheaper transactions all of these things that sound great and make it sound like the system will be much

more usable how do you actually get there what is required to get these efficiency gains I think the main challenge is that you cannot just ask validators to store more data that won't get you too far if you want to scale the blockchain to increase the block size by several orders of magnitude you have to split your block and disperse it to validator so that each file data Only Stores some fragment of the block and there were the ideas from error correcting and racial coding comes in that allows you to do that so basically increasing the block size without putting too much burden on the validators I would say that's the main technical difficulty that's what then shredding is solving you mentioned Erasure coding and it sounds like that's a key piece of this technology that

enables it to work maybe you could provide some more detail there what is Erasure coding how does it work and how does it apply in this context sure absolutely so you you basically take a block with users data and you expand it you you raise record it turn a smaller block into a larger block and this larger block can tolerate some emissions from it so you can lose some portion of the Block in case of doing trading you can lose 25 of the block and still be able to reconstruct these missing pieces from what you have so when you this person this expanded block to the validators and the validators go down because some of them are Byzantine or faulty you can still reconstruct with those validators and that's okay if they

go down and they lose their pieces the rest of the body just who are still online and Still Honest can recover those missing pieces and that's why the properties of Erasure coding are useful here just to substitute for Byzantine or faulty validators and maybe we can even explain it a bit more with an analogy it's like you know if you're watching a Netflix movie and say you know the Netflix servers are sending packets to your computer and you're watching the movie well imagine I don't know like 10 of the packets actually don't make it through and your computer only gets to see 90 of the packets so you know normally he would start to see all sorts of lossy video and degradation and performance with the ratio coding what happens is if the movie is coded using an Erasure code

even if only 90 of the packets get through the laptop has enough information to actually reconstruct the entire movie what's interesting is so Yoshi coding is used everywhere like communication networks wouldn't be able to work without Erasure coding and maybe again just another example you know when you have a deep space probe you know it's sending messages back to Earth there's a lot of noise and a lot of them gets either dropped or they get garbled and yet on Earth we're able to recover the signal and get those Crisp Images from Mars and that actually also is done using a slightly stronger technique called are correcting codes where we're not only losing packets but also we're recovering from packets being distorted being changed in value what's interesting in the context of the

blockchain is that all the data is being signed so we actually don't care so much about data Corruptions because the signature layer will detect data corruption so really all we care about the only thing that's sort of a malicious node can do that someone is trying to prevent the data from being reconstructed the only thing that that node can do is in some sense remove pieces that make up the data yeah so we don't care so much about recovering from actually Corruption of the data because that's taken care of by the signature but we do worry quite a lot about pieces of the data simply missing and as a result maybe whoever is trying to reconstruct the data is not able to do it and so that's exactly where erase recording comes in where we know that the only thing that can happen is that a

certain piece of the data is missing it can't be garbled so if we got it we got the right one and that's because of the signature but if it's smithing we have to somehow recover and that's exactly as Laro was saying that's the idea of Erasure coding and the way you do that is basically you take your original data in the case of this hearing you would take a block and you would expand it a little bit actually you expand it by like a factor of four to get sort of more data in the block so that the data is now redundant and the block and now you break it into little pieces and now you can ask every validator oh you know you don't have to store the entire block you only have to store this small piece of the block and if enough validators

actually do their job and store those little pieces and when we need to reconstruct a block they send those pieces back to us if enough pieces get back then we are able to actually reconstruct the entire block and in tank sharding in particular again it's a beautiful beautiful proposal the recovery rate is 75 so if 75 of the validators respond and we're able to recover 75 of the pieces then we're actually able to reconstruct the entire block that's kind of the core mechanism that's used in Deng sharding that's really useful so it sounds like Erasure coding is this kind of technique that enables you to apply some redundancy and backups so that you don't lose all of the data so you can still sort of

assemble it get access to it see that it exists Dan you mentioned that the reason for doing this data availability sampling is to prevent Bad actors from doing certain things like getting rid of some of the data what exactly are we defending against here yeah that's a great place to go next so in fact what happens is with these proposals for solving the data problem in ethereum there's going to be a new transaction type that's going to be introduced this is called a blob carrying transaction and what it would allow folks to do is basically embed blobs each blob is 128 kilobytes yeah so embed blobs into blocks so normally

blocks are made up of transactions to just do all sorts of things to the ethereum state so now in addition to just the regular transactions that we know and love there's also going to be a blob carrying transaction or a few blob carrying transactions per block and as I said Each one is going to be 128 kilobytes now the long-term plan and actually maybe Lara can talk about the transition to how we get there but the long-term plan is there could be quite a few data carrying blobs in every block which would actually make the block quite large right I mean each blob is 128 kilobytes if you put a bunch of those together you could end up actually you will end up with like blocks that are 30 megabytes and it's just not reasonable to ask validators to store

these huge blocks today the blocks are only like 100 kilobytes or so so these will be much much larger blocks and so the idea is to basically break up these large blocks into little pieces every validator or every node will actually store only this little piece and now the problem is well what happens if they say that they store the piece but in fact they didn't right what do we do then and that's exactly where data availability sampling comes in which is a very efficient way to test at block creation time that in fact everybody received their pieces everybody currently has their pieces and currently the block can be reconstructed even though it was broken into many many small pieces and these pieces are stored distributively across the network which is why it takes

over I just want to make sure that I understand something here so you said blocks today are about 100 kilobytes and the idea is that after all these upgrades they're going to be about 30 megabytes and part of the reason for that expansion of the block size is to accommodate this new type of data this blob data that has a different purpose than sort of what you usually shove into blocks which is just pure transaction data this blob data is really related to helping these off-chain layer 2 networks store some data in a more ephemeral manner did I get that right yeah I'm glad you reiterated that because that's important so these blobs basically are going to be used by these Roll-Ups right so the Roll-Ups have to store the rollout data and so instead today what

they do is they store it as what's called call data which is somewhat expensive and kind of not what quality that was meant for so instead they're going to store this data as blobs and blocks and what's interesting is these blobs are actually not going to be available to the execution layer of ethereum they're just going to be stored as blobs and blocks the execution layer will only see hashes of these big blobs they won't be able to access actually individual bytes or elements in the blobs which is a new mechanism today the way this is stored is as we said in call the data and call data is all available to the execution layer of ethereum so because of this simplification in fact storing blob data is going to be a lot cheaper than call data so in principle the goal of this is to reduce the costs

of layer 2 systems right because today they have to pay quite a lot to store all their data as call data in the future once Deng sharding is deployed or even once prototype charting is deployed in fact the cost will be a lot lower so L2 systems will become much much cheaper and easier to use that's great I love that we're having a sophisticated conversation about cryptography and very technical software and yet we keep using the word blob reminds me of the anti-sci-fi movie Attack of The Blob but Larry yeah maybe you could chime in now and talk about this trend and how we get from where we are now to that Vision in the future of expanded block sizes yeah for sure before I dive into that just to add two comments to what then said I

want to mention that it's important that blobs are going to expire and the validators don't give any guarantee that they're going to be storing these blogs forever right now the expiry is like set roughly the 30 to 60 days that's what I guess ethereum Foundation is thinking about so after this period during which you will have an opportunity to download all of these blobs and store it locally you know the network will drop them but the commitments to those blobs will persist and if you need so if you have the blobs themselves you can always resupply them to the execution layer with call data so as long as you store the blobs you can prove that those are the correct blobs that you have by resupplying them just because the chain continues storing the hashes of those blobs the commitments I also want to

mention another important thing is that the fee market for this blobs is going to be different so it's going to be another fee Market they're going to be priced a little differently so ethereum is kind of going to have these two pipes one pipe if it gets congested you're paying larger fees say for execution and if a data pipe gets congested you're paying larger fees for storing the data so we don't yet know how expensive the storage is going to be the intuition is that it must be less expensive than call theater today but again we have to experiment to exactly see how much cheaper it's going to be and protodin trading is actually it's a step towards for dank schroding but it's like an

experiment that we're gonna carry to see how validators are going to handle this additional load and how expensive uh going to be the fees for storing those data blobs so on the way to the next Friday we're gonna do this experiment with basically you don't apply in any Erasure coding or error correcting all you do is you add this special transaction type that carries data blobs and those blobs are going to have an expiry and that's it so the block size is going to be increased by a little bit in ethereum so right now Stan was saying it's around 100 kilobytes with product charting is going to be around 500 kilobytes or so so it's not a big of an

increase but we're still going to test like all of the hypothesis that hopefully we'll check out and ethereum will continue moving to full then charting so Larry you said a number of things in there I want to make sure that all those points came across you mentioned that the blob data is going to have a different C Market so is the right way to think about this like you have different Highway systems or perhaps you have a lane on a highway where like you have an easy pass or something and maybe it's cheaper for someone in this HOV Easy Pass style Lane you get lower fees to put this kind of data on the blockchain versus someone who's just a regular commuter having to pay a toll I know I'm kind of mixing some metaphors here but I'm wondering if

that's like a physical analogy to describe these differences in fee markets for different types of data yeah I would say that it's hard to imagine this new data fees being more expensive than what we currently have so the hypothesis is that it's always going to be cheaper to put your data in those blobs if you don't care about accessing this data from the execution layer so it's as if opening you know another Lane on your Highway that will just increase traffic but for certain types of transactions you will go on to this Lane and for those transactions that are kind of data heavy and for transactions with execution heavy it will continue going through the main Lanes if that makes

sense yes it does and it sounds like one of the reasons why it can be cheaper is because it has this expiration date as you mentioned I think you said that the current ideas maybe that's going to last 30 to 60 days for this blob data at which point it would simply I don't know vanish and there would just be a trace remaining a sort of commitment as you described yes exactly maybe to add to that basically what happens is when you submit transactions there's a fee Market as Lara was saying in that if there are lots of transactions being submitted all at once say there's a big enough team end and everybody wants to issue transactions then of course the price per transaction immediately goes up with blob data is going to be a parallel fee

market so presumably there are fewer people submitting blobs than there are people submitting transactions so hopefully there will be less congestion over blobs but in principle it could happen hopefully not but it could happen that all of a sudden for some reason there's huge congestion over blobs and in the flea market for blobs will actually go up but in principle again because there's less demand for submitting blobs than is for submitting transactions the hope is that the cost for blobs will be lower than the cost for transactions okay that's really helpful to understand as well so we've laid out a sort of timeline for these updates perhaps people listening in maybe you're familiar with the merge which happened last year in the fall this big ethereum upgrade that basically eliminated the

environmental impact of ethereum in terms of its energy consumption and now we're entering this new period of upgrades which I think vitalik co-creator of ethereum has termed The Surge meaning that all of a sudden these updates are going to enable the blockchain to scale a lot more powerfully than it's been able to before so part of this update one of the first ones is Proto dang sharding it's happening later this year what happens in between that upgrade and full on dank sharding what are the differences between the two and when do we get that fully formed vision of dank sharding in the future that's a great question I think there are still some research problems to

figure out along the way especially around the networking because when those validators are storing only fragments of the blog they need to help each other reconstruct those fragments if some validator falls asleep for a while it wakes up once the help of other validators to help it reconstructs its missing fragment so it's quite involved the networking protocol that I think is still in the making for doing shredding and there are other exciting research problems that potentially can improve the scheme but I think so far it looks like quite a clear path to get from product and chart into Dent charting and the question is just maybe how to make it better how to improve different aspects of it make it more efficient

maybe it's worthwhile adding that in the product sharing approach there are at most four blobs per block which is why each blob is 148 kilobytes four times 128 gives us half a megabyte which is why that's kind of the limit on block sizes in proto-denk sharding and that's actually imminent that's supposed to happen later this year and then yeah going all the way to Deng sharding still uh takes some work in fact there was a very big event recently with generating parameters jointly for tank sharding and so um there's a lot of work to do in preparation of getting to full tank charting yeah that's quite exciting I think there's two accepting contributions to participate in the trusted setup ceremony yeah so it's still ongoing it's

a large community effort that's really fun to watch and people come up with lots of creative ideas to contribute so check this out definitely that's super cool I actually participated in some of the uh trusted setup Ceremonies for some of the early privacy coins is this trusted setup ceremony as ostentatious as some of those where you had people kind of like burning laptops and exploding hard drives and things like that from what I've seen is just people come up with different creative ways to generate entropy some using their pets dogs and cats um some are creating some sophisticated marble run machines and such there was even one contribution done from a satellite the highest altitude contribution there so yeah

actually we have to mention that it was pretty cool actually that company cryptosat actually has satellites and orbits that sample noise up in space and then contributes and participate in the trusted seller protocol so that was pretty cool pretty cool to see wow that is awesome did not know there was crypto in space just yet Dan you said that Proto dank already is going to have four blobs per block what is uh dank sharding going to enable how many blobs are we talking here yes by the way it's protogenics up to four blobs per block they're actually targeting two but up to four and then thing starting as we said they're targeting at most 30 megabyte blocks so just divide 30 megabytes by 128 kilobytes and that tells you it's you know on the order of

100 I guess it's about a half yeah I think the current plan is 128 Target and maybe up to 256 blobs per one block great and that's exactly when the blocks become quite large and then it becomes a little difficult for the validators to keep the entire blocks themselves and that's where we have to start breaking them up into pieces and he's validator will store one piece got it I appreciate the uh basic division maybe some of the more technical math that you get into in your post would be a little bit harder to convey here but that makes sense maybe we could talk a little bit about some of the proposals that you made in your recent post so you did this research and

you found that with some adjustments you could potentially get even more efficiency out of eip4844 which is the more technical name for proto-dank sharding yeah the bulk of the work was just to understand the Dank shortening proposal in full and then we observed some different ways to look into the mathematics cryptographic components of it that hopefully unravel new toolkits that we can use there so the rough idea not going too much in depth is you fit a polynomial through your blog it's a bivariate polynomial because your block is rectangular and then you evaluate the spawn them a lot more points kind of

expanding the block and the idea is that if you've used this as bivariate polynomials instead of as dank writing was viewing it as a list of multiveried polynomials you can then apply you know techniques for bivariate evolution bivariate interpolation possibly even try to apply bivariator correcting codes to that but that's very much an open door for more research and exploration so so in this post we try to explain where more research can be done to improve the scheme in that direction yeah maybe I can add to that I mean Dan Clarity is such a beautiful idea really the ethereum foundation deserves a huge amount of credit for it and you know done card in particular it's really quite a elegant construction initially we were just kind of trying to

understand the details and you know it took some time to kind of recover exactly all the details of how everything works and we figured maybe it'll help the world to also have another write-up that explains how the mechanism works and initially I guess the original dunk sharding the way it's described is all using univariates polynomial commitments where we take each row we view it as a polynomial Erasure coding is all done using polynomials maybe I can even teach a little bit of Erasure coding here in one sentence in that suppose you have two points in the plane and you want to do Erasure coding on them what you can do is you can just pass a line through those two points and now you can just publish instead of just those two points

you can publish those two points plus maybe two additional points on the line so now you have four points total then you know that if two out of those four points make it to the receiver the receiver can use the two points that it received to recover the line and then recover the original two points that's the whole idea of Erasure coding so of course instead of lines we use higher degree polynomials to achieve different thresholds but that's the idea basically we have two points we pass a line we get more points on the line if only two points of the line make it to the receiver the receiver can reconstruct a line and recover the original points and so then starting actually does this really really quite elegantly by looking at the Block as a matrix as a

rectangular set of data and then it basically extends using these line ideas both horizontally and vertically and that actually gives the quota block where then pieces of that rectangle is sent to the different validators what's interesting is now that you have a rectangle you can kind of think of it as a two-dimensional object that very naturally leads to thinking about this as a bivariate polynomial as exactly as Lara was saying and what tank sharing does is it gives actually a very interesting way to commit to these bivariate polynomials so it kind of builds a way to commit to bivariant polynomials by using commitments to univariate polynomials and so it turns out that the Reconstruction mechanism that was done in Bank charting is also based on construction along lines and

columns also construction as done using univariated polynomials and then I guess as we were working through this there was this realization that hey everything here really is about rectangles and bivariate polynomials maybe it is a way in which the Reconstruction can also be done by using sort of interpolation of bivariate polynomials so in fact you take click your blog and you expand it to X by the factor of 2 in both directions so you have 4X more points as a result but those four x points only encode a small quadrant so in principle you only need one quadrant in order to interpolate and recover all the rest of this encoded block so 25 of the points

should be enough but as Bank sharding works by doing Universe interpolations it needs 75 percent of the block to do you know column and row wise reconstruction if you do bivariate interpolation directly you should be good with just 25 instead of 75 so that will improve the number of elements you need in order to reconstruct to recover the block I'll also improve communication that availability sampling but it's all due to improve the Reconstruction now the question becomes kind of a mathematical question of how do you do efficient by variate interpolation there's an obvious need for kind of a better algorithm there and we were researching this a little bit

and it appears so far to be a little bit underexplored so maybe there were no applications for bivariate interpolation before maybe it's just a hard problem or we don't know but that's definitely an interesting direction to basically try and improve bilinear interpolational algorithms so what I love about this is just like Lara was saying for the more algorithms folks in the audience is that there's been a ton of work on doing univariate interpolation if I give you points on the universe polynomial like you know points on a line and I ask you to reconstruct that polynomial there are very good algorithms for univariate polynomial interpolation and it turns out bivariate interpolation problem somehow it seems like it received less

attention and what's really cool here is all of a sudden you know the blockchain ethereum Dank sharding is creating an application for this really natural algorithmic problem of bivariate polynomial interpolation we really need it here if we had a good algorithm for bivariate polynomial interpolation we could actually make down charging better because reconstruction as Lara was saying would go down from 75 to 25 so to me this is really one beautiful in that the blockchain ethereum Dank sharding is creating this new area of research or at least prioritizing this new era which research showing you know we really need better algorithms new algorithms you know efficient algorithms to do bivariate polynomial interpolation so

hopefully that will encourage and spur a lot more research on these types of algorithms and presumably they will be actually very useful for this problem so I love that we dug into the methodology here and didn't shy away from the mathematics I know some of it might sound a little complicated you know bivariate polynomials and univariate polynomials but I especially appreciate how you sort of describe these in terms of geometry and shapes because I think everybody here can really picture a line going through some points or how a rectangle functions so I think that really helps ground the kind of work that you're doing I want to double click on this statistic that you mentioned where uh where the current proposal as it exists would require

75 of samples in order to be reconstructed whereas what you're proposing would trim that down to 25 so that's a giant difference 75 percent of 25 but it also sounds like just for a casual Observer if you're only having 25 the fact that it's like just less than 50 it sounds like is that really enough to to make assurances that this data is available and was made available you know when you go down to 25 it sounds like I don't know you might be cutting some corners or stuff so how do you assure people that in fact just having 25 of data samples is actually enough and that things can work at that level

yeah that gets us to the topic of that availability sampling and what it achieves I guess because this reconstruction thresholds 75 or 25 basically determine how many samples you need to get high assurance that the data is there the way you do the sample is that you ask the network of violators to give you back one element random element of this encoded block and if you get it back successfully and you can verify also what's the validators give you back the proof of the validity of this piece and you can verify it against the commitments that the chain persistently stores so when you get back the successful sample that makes you sure that the data is available with

probability that is 1 minus either one quarter or three quarters depending on how your reconstruction algorithm Works whether it requires 25 of the data 75 of the data so every time you do a random sample and it comes back successfully your basically false positive the probability that you think the data is available when it's not drops down exponentially and the rate at which it drops down depends on how much data you're required in the Reconstruction so if you reconstructionally request 95 of the data you do less samples and your Assurance the false positive rates goes down quicker than if you only have a reconstruction album that requires 75 of

the data so depending on how efficient is your reconstruction you might need fewer samples in order to have the same assurance that the data is available so that's why you not only improve the Reconstruction here but you also improve the number of samples you need to do for your data availability sampling and that availability sampling is interesting because it's probabilistic so the most samples you do the higher is your Assurance the data is available right and you can always amplify this probability by doing more samples to make it clear I think actually Lira what you just explained is really really important so that's like the heart of the angstarting and data availability sampling so I'll just say it one more time just so that the audience will hear it twice because that's really got the heart of it so maybe think of uh the block we said the

block is going to be encoded as a rectangle right so somehow we go from a block to a rectangle the specifics of that is done using Sim ratio coding but let's pretend we went from a block of data to our rectangle so now imagine this rectangle is literally a rectangle of dots every dot corresponds to like you know one piece of the data that's going to be distributed to one validator so now to do data availability sampling what somebody wants to verify you know that enough of the dots are actually available we know that if more than 75 of the dots are available then the block can be reconstructed using the Erasure coding method or maybe if what we're saying would be used then only 25 of the dots are sufficient to reconstruct the

original rectangle but how do you know that 75 percent of the dots are available so that's exactly the data availability sampling mechanism what you do is you can kind of Imagine like you're throwing darts at this rectangle right so every time you throw a dart you hit a random point in the rectangle and then the validator that holds that point has to prove yes I really have that point now you want to verify that 75 of the dots are available so imagine you have this rectangle maybe only 75 of the dots are there some of the dots disappeared for some reason you want to verify that 75 of the dots are available because if 75 are available you can reconstruct the entire rectangle and so what are you going to do to verify the 75 are there you're going to throw a

bunch of darts at the rectangle and for every time the dart hits a DOT the validator that it hits has to prove that the dot really is there and so if you throw a hundred darts in all 100 darts come back saying yes the data really is there that gives you a pretty good idea that more than 75 of the data is available because you know if less than 75 is available and you throw four darts you expect one Dart to hit a missing Dot and so if you throw a hundred darts and all of them come back saying the data is available you have pretty good assurance that more than 75 of the dots are there and so that's the idea of data availability sampling you just try lots

and lots and lots of random points like 100 of them if all of them are there then you have pretty good assurance that more than 75 are available and you can reconstruct the data and you see if you want to get 75 Assurance you need to throw a hundred darts if you need only 25 Assurance you would need to throw fewer darts than that so that would basically reduce the amount of data that's needed to satisfy the sampling mechanism now maybe it's worthwhile saying that once the data availability sampling check succeeds so all the 100 darts come back saying yes the dots really are there then the validator says Ah the data is available and it goes ahead and signs the block as saying this block passed data availability sampling

yeah and that's used later on in consensus so that's what the test is the test basically data availability sampling is a very efficient way to test that enough data is available to reconstruct a block without actually reconstructing the block completely so I think it's good to hear it twice and maybe even a third and a fourth time but that's kind of the core idea that makes this all work I love that and I especially love the physical analogies that you're using with a dartboard and throwing darts I think that really brings it home for people so we're nearing the top of the hour here I want to get ready to close out but before we do that just maybe I'll throw it to you both in one sentence what is the upshot of all of this like

why does it matter to get these efficiency gains well I would say the ultimate goal of course is to scale the blockchain and this new techniques would allow to do that for ethereum to achieve full scaling that's really interesting approach I would say because in the beginning Syrian was thinking about doing sharding full sharding and arriving quite complicated designs but having rollups help scale execution of ethereum and leaves it up to ethereum to scale its data availability layer basically increase the space while Roll-Ups increase of the execution capacity and that piece together will give us cheaper and faster blockchains yeah Lara said it perfectly I mean

really the scaling story for ethereum is Roll-Ups and the way to get rollups to be more efficient and cheaper to use is by solving the data problem and dank sorting is a very elegant and efficient way to do that so the upshot is a scalable version of ethereum where roll ups are much cheaper to use than they are today that's great and if you get cheaper transactions and the ability to make more transactions I think that opens up ethereum to do all sorts of new applications that weren't possible before with high gas costs and fees so this has been great thank you all for joining us I hope that you all learned a little bit about data availability

sampling and bank sharding if you'd like to learn more as mentioned you can check out Dan and Lara's post it's a really really great piece so I highly recommend reading it and checking out all the resources contained in there thank you all for joining us I'm looking forward to this weekend I'm going to go to my local bar and teach everybody about Erasure coding so thank you all and take care that's great thank you thank you this has been fun bye [Music] thank you for listening to web3 with A6 and Z you can find show notes with links to resources books or papers discussed transcripts and more at asicsncrypto.com

this episode was technically edited by our audio editor Justin golden credit also to moonshot design for the art and all thanks to support from Asics and Z crypto to follow more of our work and get updates resources from us and from others be sure to subscribe to our web 3 Weekly Newsletter you can find it on our website at asicsnzcrypto.com thank you for listening and for subscribing let's go [Music]