ZK7: Hyperscale Data Availability for ZK Systems - Sreeram Kanan - UoW / Layr Labs

Transcript

[Music] good afternoon everybody um i'm sridham kanan i'm an associate professor at the university of washington seattle i run the uw blockchain research lab and i've been i've been on leave over the last year building a startup called layer labs today's talk is titled hyperscale data availability for zero knowledge systems and we'll tell you about some of the work we've been doing in how to provide hyper scale and capital efficient data availability for zero knowledge systems

first things first i need to thank my team um shaubik robert bowen jeffrey gautham vyas gautaman vyasa here um thank you to all of you for this team effort okay so we start with the basic premise and the problem that much of xero knowledge rules are solving the scaling problem of blockchains so when you have a blockchain you usually have a leader based system where there is a single node the proposer which is proposing a block and it needs to execute all the transactions so it does that and then it sends it to the other nodes all of them also have to verify all the

transactions however zero knowledge systems solve this scaling problem so that's the big advantage of zk systems is instead of all the nodes having to verify the entire set of transactions the sequencer generates a short proof that it did execute these transactions correctly and then send it to all the other nodes that verify these proofs so while zero knowledge systems solve the computation scalability problem they do not solve the communication

complexity problem why is that all the data all the transactions have to be sent to all the different nodes to ensure that the data is available why is this a problem because if you did not do this if you did not have all the nodes have access to this data then even though these nodes are verified that the state transition has occurred correctly they may not be able to reconstruct the state and if they're not able to reconstruct the state then the sequencer which is not necessarily trusted or a proposer who's not necessarily trusted can completely halt the system and do an extortion attack okay so

this so the solution to this problem both in zk systems which are separate layer ones and in zk roll ups is to make sure that these transactions are available in like a large truss group so that's the solution so while zero knowledge systems scale computation they do not scale communication these transactions have to be flooded to all these nodes that incurs a cost and a scalability penalty so zk roll ups which opt to run as layer 2s on top of a big layer 1 system like ethereum

in these systems the scalability problem translates into a cost problem why because you have to pay for the non-scalable cost that you're letting the others incur so in these ek rollups what's happening is the uh you have some application and the sequencer collects all these data and they're storing the data into the data layer of the core protocol like in ethereum just store it as call data and you provide your validity proofs the short zero knowledge proves that you did do the execution correctly into the settlement layer the validity

proofs at least at scale are not costly because you can have a lot of transactions in the block and a short valet proof but the amount of data you're storing on ethereum is actually quite large because that scales with this with the number of transactions in your system so summarizing this the data availability problem is a scaling problem that translates as a cost problem to rollups because you have to pay for the non-scalability and to compound this problem for a zero knowledge roll-up

they are competing in a landscape of many alternative layer ones and these alternative layer ones can offer near zero fees by paying their validators in their native token by inflating their native token which is basically just paying in future revenue and while it's an attractive model for them rollups cannot play this model why can't they play this model because they have to pay for the non-scalability in real hard eat or whatever the layer one that they're using of course these alternative l1's may not offer the same decentralization and

security as ethereum the question is can we get the best of both worlds can we get ethereum security or a large scale security of whatever layer one that you're using i will scope the discussion by discussing about rollups but the methods that i'm proposing here equally apply to separate layer one zero knowledge systems okay can we get near zero fees and ethereum security the best of both worlds okay one solution that has been proposed to solve this problem is a separate data availability layer a separate data availability layer

so what is the idea here the idea is instead of the sequencer storing all the data into ethereum they store the data in a separate data availability layer that somebody builds and of course the benefit is now you get low cost because this data availability layer may be custom built for your particular roll-up and of course the validity proof can still be written into ethereum and the data availability layer reports to ethereum that hey i have checked that the data is available

okay some people call this a validium in these validiums the big gain is near zero fees very low fees because you know it's built for this purpose but there are some problems first the whole point of a rollup was to derive the full security of the layer one system like ethereum but you do not get it in this kind of a method there is also a large capital cost of staking if this data availability layer had its own staking

you have a new token for this layer and you have to stake this token in order to provide the service why do you need a token because you want to create some crypto economic cost if the data was unavailable i want to slash these guys if you wanted to create this crypto economic cost you need staking and if you want staking you have to pay capital cost what is capital cost suppose i want to lock suppose this is a highly bootstrap data availability layer worth the same as ethereum maybe a few hundred billion dollars this is only sustainable if this data availability layer is able to draw an annual revenue

of 10 of that value the 10 or 5 or 10 percent whatever is the apr that's the market macroeconomic rate so you need to be able paying fees to this data availability layer not for the actual cost of storing data but for the capital cost that somebody is incurring to provide crypto economic security a separate data availability layer any separate proof of stake network you have to immediately think what is the capital cost capital cost dominates all costs in proof of stake networks okay so who's paying for this is this layer going to be continuously cheap something we need to think about

okay so there is a third problem which is specific to rollups the problem is if all value flow is going from the zero knowledge rollup to this new data availability layer who collects all the fees all the fees is going here and very because this is the only fees that scales with the amount of data the fees here does not scale with the amount of data the value already proves so all fees is going to a different layer essentially you have incentive alignment only with the holders of this other system not of this system the value alignment between the core

layer 1 and the layer 2 is broken in this model so if you wanted holders to say hey you know this is a heat friendly roll up then there's a value alignment problem okay so again back to the same question can we get near zero fees and eat security together okay so i'm going to zoom out a little bit this was a very specific discussion about data availability but the problems raised here are not specific at all they are a problem for any middleware what is the middleware

middleware is anything that has their own trust network middleware is basically anything that has their own trust network you can think of oracles you can think of bridges you can think of sidechains you can think of keepers you can think of anything that needs their own distributed validation services so if you think of systems that need their own distributed validation services and if you think of a core protocol like ethereum there is the current ecosystem is organized like this ethereum provides trust to the dapps and receives fees in exchange

right so the core value proposition of ethereum is to provide decentralized trust and the adapts consume set trust and then pay fees this is the value flow ethereum creates trust supplies it adapts consume trust and pay the fee so however ethereum is only able to provide trust in certain axes in particular in block making the ethereum stakers only certify block making they do not certify anything else that's the limited action space that they're operating inside but you want an oracle input you want some new data

availability you want some you know different execution environment you have to opt in to some other middleware so now what is the problem with the economics of this kind of a middleware ecosystem there are really three problems the first one is the middleware if you have it staked it faces the cost of capital you want a large enough staking quorum that means you better feed that quorum you have to have enough revenue to feed this quorum that's very expensive what's the problem for the dapps the dapps are paying high fees to ethereum for the high security but the security that they're getting is

the worst of all the things that they depend on so whichever rickety middleware that you depend on that's what is going to dominate your cost of corruption and your trust model so the dabs aren't happy the middleware is being very expensive they are not happy ethereum is not happy because they are the decentralized trust source but there are so many small islands of trust rather than a federated union of trust okay so having laid out the problem space in middleware i also want to say something a little bit more ideological here which is if you think about the space of middleware

these are basically the new algorithms that drive our ecosystem forward you need new oracles you need new data availability protocols you need our new execution engines we need people to iterate permissionlessly and compete on all of these things we need open innovation but to do open innovation today you need to be able to create your own decentralized trust network in the middleware or the core blockchain layers which is two different things being able to create a decentralized trust network and being able to write very good distributed systems have nothing in common they are just completely different jobs okay so an ideal world would have you build your distributed

system and deploy it on an existing large truss network that brings us to our main product called eigen layer eigen layer is the restaking collective what is restaking it's a new word a new operative word here restaking i'll try to explain what we mean by re is taking so you have the core protocol which is providing services as i mentioned before but you want to create you want the ability for anybody to create new services how do we do this

we have eigen layer eigen layer is a series of smart contracts written on top of ethereum and eigen layer allows stakers in ethereum to re-stake what is re-staking re-staking is the mechanism by which you subject your stake to additional slashing conditions if you're a heat staker you're subjecting your stake to eat slashing conditions but if you're an eigen layer if you can opt into eigen layer any staker in ethereum can opt into either layer they are basically restaking their stake they are putting

the same stake at additional jeopardy and now they can opt in to provide services on a wide variety of different middleware okay these middleware are not built by like us or eigen layer or anything anybody can build these middleware and deploy them this is the open innovation part the creativity is the limit for what these different middleware can be okay so core principle is re-staking restaking is you take your stake and subject it to a double jeopardy on eigen layer on any new distributed system that you're agreeing to provide validation

services to so why is this interesting so the first point is eigen layer allows reuse of staked capital to provide additional services you've already staked you're already receiving a certain apr from the core protocol so the marginal cost of capital to opt into eigen layer the marginal cost of capital to opt into eigen layer is zero because you already like steak than getting returns and there's nothing else you can do with this steak today

you can opt in into eigen layer and start providing additional services so the marginal cost of capital is zero so your marginal cost as a each staker to opt into a new validation service is purely operational cost you look at it and see hey they're giving me x revenue i'm spending x dollars is the equation balanced you do not have to worry about what is my capital cost this fundamentally transforms what set of middleware can be built as far as the dapps are concerned the dapps especially if they consume eigen layer services which are widely used

potentially can get the full ethereum trust if stakers restake and if if it is favorable in their interest to restake then they may restake so you could potentially get not guaranteed but potentially you can get the core ethereum trust but you always know how much trust you're getting because it's crypto economically measurable you know you have 10 billion dollar worth of security or 20 billion worth of security and finally you have alignment of incentives instead of having fragmented trust small trust networks which accrue value due to small services that they're providing you get

internalized integrated trust back into a common protocol ethereum ethereum can now basically as more and more services are provided on eigen layer the revenue capability of each taking goes up and the value of ether is basically the potential future revenue that you can get by staking ether that goes up instead of only earning revenue on ordering and execution now you earn revenue on ordering execution authentication articles whatever new thing you can think of all right so that's the paradigm of how to build new middleware on top of eigen layer eigen layer is a restaking

collective for those of you familiar with merge mining it looks like merge mining but fundamentally the crypto economics are totally different because in merge mining if you merge mine the new token with bitcoin if you if miners attack the token nothing happens to bitcoin in re-staking if you did something bad on eigen layer you lose your eat you're not losing eigen tokens you're losing your eat it's a very different value proposition okay so we're building data layer on top of eigen layer eigen layer is the restaking collective anybody can build any distributed system on top one of the most immediate and interesting things to

build is a data layer so how does it work some subset of like the ethereum stakers who join eigen layer they can opt in to data layer which is a set of smart contracts and off chain software and what you do is you know the usual thing that the sequencer does in a rollup you collect all the transactions you update the state and then [Music] instead of just dumping the data you do polynomial erasure coding along with polynomial commitments you do the aeration coding with polynomial commitments send this data over to all these data

layer stakers through a private network and so if you're registered in data layer you need to have you need to provide your ip address to which they can send your particular chunk you download the chunk sign it send it back to the sequencer the sequencer aggregates these signatures sends it back to ethereum contracts where it is transparent that a quorum has been obtained okay the principle is very simple you have e3 stakers and e3 these takers opt into data layer and in data layer what do they do they basically guarantee data availability

how do they guarantee data availability in a scalable manner you use all the best ideas that ethereum community and others have come up with use erasure codes use polynomial commitments use multi-reveal all the things that you guys you know have come up with okay so how do we enforce a crypto economic cost how do we make sure that if these restakers behave willingly they say they're providing data availability but they're just sleeping or don't download the data or don't serve the data how do we penalize these guys the core mechanism is what we call force disclosure what is force disclosure

force disclosure is a mechanism by which anybody can trigger put up a fee and trigger disclosure of data and if you disclose there if you if the stakers do not disclose the data on chain only on request right then they can get slashed their eth can be slashed okay so if a sequencer makes a claim and that the data is available and the data layers nodes agree that the data is available and it turns out not to be available that is incentives for somebody using the rollup to ask for forced disclosure and if they ask for forced disclosure you basically have to respond and if the data layer nodes don't respond their stake will be slashed by each smart contract

it's not dependent on slashing is not dependent on some side chain or some other thing slashing happens on chain through smart contracts so that that specifies also the set of use cases you can build with eigen layer anything where you can slash using on-chain smart contracts okay so that's the core idea of data layer it's a low-cost data availability data layer is hyperscale what do i mean by hyperscale if there are n nodes in data layer each of them have a bandwidth c megabytes per second that they can download data layer can reach a commitment rate

of n times c there are n nodes each of them have a bandwidth c you can the theoretical bound is n times c that's the rate at which you can transmit information to them data layer can reach one half times n times c why one half maybe half the nodes are bad who knows so just plug in some numbers you a typical node can store you know a validator can have like one megabyte per second of download rate if there are 10 000 validators like in eth2 you may have 10 gigabytes per second running through the system this is hyperscale data availability

there is no trade-off between scalability and security as more and more nodes join you get more and more secure and more and more scalable so here are some numbers the data availability rate is the rate at which you can commit data into a system ethereum is like you know few hundred kilobytes per second each two downshotting down cloud is here 1.3 megabytes per second i'll explain why we are able to achieve more even though we're using the same scheme we're not innovating on the scheme at all this is due to the opt-in mechanism of iron layer so we are able to achieve

already on our test net 10 megabytes per second on 100 node system of course the scheme is okay it's hidden due to this uh banner here but thanks to don crowdfaced uh dimitri and proto-lambda for the code basis for downshotting which we've built extensively upon and also for the core ideas that underlie the system so why are we able to do better it's not because you know we are very smart or anything like that it's because eigen layer is opt-in and users can opt in to provide their ip addresses the data layers takers can opt in to provide their ip addresses what this means is you do not need peer-to-peer networks you do not suffer through peer-to-peer overheads

you do not suffer through denial of service attacks there's a whole range of benefits if you had stakers reveal their ip addresses but that's not the design goal of eth2 so that's why they have to do a much more intricate peer-to-peer network system which achieves a different rate so different trade-offs but data layer is hyperscale we do not want to claim that this is the solution to the data availability final solution to the data availability problem but we do want to claim that on eigen layer any of you who come up with a better data availability service can build on top of it use the massive network of ethereum to underwrite your service and compete permissionlessly on which is the best

system okay i have a three more minutes uh data layer is crypto economically secure and i'll not get into this there is some intricate things about slashing uh obviously data layer enables l1 alignment because you're paying fees back to the layer one layer one grows in value when your roll-up grows in value aligns everybody in the ecosystem also data layer provides flexible tokenomics what do i mean by that why am i talking about tokenomics when i'm talking about a new distributed system because eigenlayer is opt-in you can pay fees in your own token if the other guys want to take those

fees they can opt in opt-in changes everything you can pay fees in the start net token or your zksync token or whatever other things that you want and they can take it if it's helpful why would you want to do it you would want to do it if you wanted to pay like altel wants in inflation initially before you get network effect okay so data layer is highly customizable what do i mean by customizable you are using a certain elliptic curve and a proof system and don't want to use whatever things i arbitrarily choose you

want to say hey i want to use this particular polynomial commitment i don't want it to be like non-transparent you come up with your fri or whatever and then we'll you can run your own data layer data layer is hyper customizable you can choose your field you can choose your algebraic code you can choose your commitment you can choose your signatures just reuse the trust network okay so one more thing today zero knowledge systems suffer a proof latency because of cost the verification cost is still high so you may not be able to run a verification every block on ethereum but what you can do is you can send your

proof to the data layer nodes the data layer nodes because you can build an arbitrary distributed system on eigen layer they can verify your zero knowledge proof immediately certify on a quorum that the proof is correct and then post a commitment on chain potentially slashable later and what that means is you have a high speed state bridge hyper composable with ethereum every block eigen layer will be writing commitments through a single merkle root back into ethereum now that means you can move data back and forth from your roll-up compose different roll-ups do all kinds of interesting things on eigen layer i'll stop with this just point out that

we're already on internal test net we have staking contracts you can deposit you know either liquids taking derivatives or be your own home validator and participate in eigen layer and you know this is some of the data from some of our experiments we're running currently at 12 megabytes per second in our internal data layer that's the commitment rate each node does not download 12 megabytes per second they download only 700 kilobytes per second this is on a 40 node network and we are scaling up soon i'd like to stop the talk with that thank you so much hannah [Applause] you