The Missing Layer: High-Performance Memory for the World Computer

Transcript

[Music] Okay. All righty. Okay. Great. Um, hi everyone. Uh, it's it's a pleasure to be here. I'm going to talk to you about Optimum and the high performance memory for the world computer. Uh, so um I'm Muriel Mad Muriel Mid. Uh, and uh, I'm on leave from MIT to pursue this full-time. uh this has been a very very long time in the making in the lab uh and finally ready to come out. So it's been extremely exciting uh to jump from uh from academia uh with my co-founders and one of them is here to to take this this big step to make this uh technology

uh a reality. Um I've been uh at MIT for pretty much my my entire career before that uh heading the network coding and reliable communications group. uh and I'm uh I hold the chair the NC chair in software science and engineering for the entire school of engineering. Okay. Um we have amazing backers. Um we've been really really fortunate uh to get a group of people who didn't just support us financially um but are involved, knowledgeable and supportive. Uh that's really what we were looking for and that's what we had. So um our uh first uh our seed round was uh led by 1K ax uh and uh as you can see we have really

just an amazing amazing set of backers. Um so first of all what is memory? I really liked uh the previous uh the previous presentation because you know I was making mental notes about the fact that uh they were doing a really good job sort of setting the stage for a bus and a ROM and a RAM uh and uh basically that's that's really what uh what memory is. So um when you think of data propagation particularly gossip particularly in the case uh of Ethereum uh that's kind of like the ephemeral memory that's uh really just like a bus in a computer and we'll see that later.

Uh you also have the readonly memory. Uh so if you think of any sort of ledger uh you write to it but then you you don't erase you don't update you only read from it after you write. Uh so that's basically what we're going to call our optimum DROM. And then you have state and state is memory that you can access and change. And of course there you have to be very very careful uh about things like atomicity uh and consistency and durability sort of the acid framework uh that a computer which is centralized would uh be able to provide to you. So these are the three aspects the three axes uh of our memory

uh and this is basically what uh what we're building and uh to go a little bit more into detail um how does this connect to the world computer how does this connect to the title of the presentation um so I'm going to start maybe from the middle rather than from the left you know what is a computer right if you're going to build a world computer it's a very reasonable question to say well Okay, great. What are you building? What is a computer? And we have this fantastic um uh sort of model, this abstraction that has really stood the test of time, the what's called the the Venoyman or or the or the Princeton abstraction. And what it said is um you know a computer takes an input an output

of course you know so you you ask it to do something and it gives it back to you. And in the main in the in the middle you have sort of two big things. One is a compute you know with the arithmetic logic unit again we saw that in the previous presentation and the other one is a memory unit with in between the communication the bus. Okay. So you need not just to compute but you need to have memory to store what you the results of your computation also to be able to read local inputs uh for your computation and you can actually map now going to the left hand side that very cleanly uh to what's going on in the layers uh in the layers um of of web 3.

And you know you you can discuss a little bit exactly you know maybe some people have some other layers or not but you know by and large everybody agrees that at least these layers are there. Uh and what you can see at the top in green is kind of really com you know corresponding to the computation and then the bottom in purple uh networking layer state data and transaction data really going more to the memory unit. Um so let's go back to the top layer which would be say your EVM. Uh I think in there you can really recognize um something which would look familiar if you ever took a course say in systems uh you know in your undergraduate or or

operate you know in any sort of computer systems you know the top is decentralized in web 3. It's a little different but you know it looks very very recognizable. The bottom part right now does not look so recognizable right. So you have very ad hoc solutions for read rights. Uh the bus instead of being you know a serial or uh you know very um carefully managed bus is is is gossip which is very different. Um you know the memory the ROM looks like archival nodes. Um and then you know you have this extra aspect of DA for the archival nodes which generally is very much hidden inside of your computer right there is an access

that you do check whether the the data is uh is available and and indeed uh you know hasn't been corrupted but you do that sort of behind the scenes you don't you don't usually um sort of expose that feature. Um, so okay, you can say, "All right, so maybe I don't recognize the bottom part, the the purple part, but why do I care, right? I mean, if it works, uh, why do I care whether I recognize it or not?" Well, who who cares? Uh, and we care. And because that's really the the fact that the bottom part is so ad hoc is really why you have this low bandwidth, these low transactions, these massive storage costs, and also scaling challenges, right? and scaling challenges. We'll see

that later. Um mean that you know things basically get slow things get expensive um as you get more and more nodes or alternatively as you're just using the network more and more. So when you're trying to get more throughput say bigger blobs or anything like that. Okay. So what are the optimum products you know what what what do we do? So our first one and we'll talk about that a little bit more in detail is the optimum P2P corresponding effectively to what I mentioned before which is uh which is the bus and what we do is we increase bandwidth we reduce latency how do we do that how much do we do that I'll show you I'll show you that uh in a second um

and basically we can take variety of of data types uh but we really change drastically the latency and the throughput in that domain. Our next two products uh which we've already done a lot of the work on uh but uh but haven't put on on test net yet are the DROM so the the read only memory you know the the the ledger side uh where you need to have durable and efficient data storage and then the DRAM where you also need the extra twist of consistency right so so that everything looks to you as though it was done in a central control

RAM on a single computer even though it's actually completely decentralized. Um, and that's basically keeping uh the state machine versus keeping the ledger where you're just adding but you're not changing things that you've already written. Okay, so I told you that we do better uh and you know we we now have to show you now I have to show you that indeed we do better. And here's some of the performance benchmarks. This is from our private test net. Uh I'll tell you later who some of our part who our partners are in that private test net. But I'd like to spend a little bit more time on this uh on this slide. Okay. So what do we have down here uh on the obsa is the

message size. And what I have on the left hand side is the average latency in milliseconds. And on the right hand side is the standard deviation. Okay. So basically this is the mean. This is the first moment and that's the standard deviation. That's a square root of the second moment. All right. So you want of course the mean to be as low as possible, but you also want the standard deviation to be as low as possible. Why? Well, first of all, you'd like things to be not uncertain. And actually, you know, for all of you out there who who maybe

have studied EQing, you know, there was there was actually a really interesting conversation recently uh on X uh about, you know, how uh you know, how how going to to um uh smaller message sizes was was helping. And I I I jumped in and I said, well, you know, it's it's because something called uh the polychin equation. And the polychin equation tells you that actually having this variance, the second moment be large hurts you not just you know in sort of uh a user experience but actually also contributes to higher delays. The variance actually at a system level um contributes to to higher delays. So we

want the first one to be low but we also want at a system level the second one to be to be as low as possible. And what you can see is that we get a benefit. Um this is comparing gossip sum. So effectively what is currently in ethereum with optimum P2P and you can see at 2 megabytes, 4 megabytes, 6 megabyte, 8 megabyte. So you see a steady improvement but also an improvement that's highly nonlinear. Again, why is it highly nonlinear? Uh for all of you who took queuing at some stage and for all of you who didn't take queuing, it's because you have a very very nonlinear relation between delay and utilization. So as you're trying to

put in more throughput, your utilization goes up and as utilization goes up, your delay, your latency goes up in a completely nonlinear way for that. This is something that we experience all the time particularly if you live in a place which maybe has some congestion on the roads. You know you're on the road everything looks great. You know it's it's uh okay it's just it's just going really well and then you know you have a very short period of time when it looks like it's a little congested but you're still going uh going uh you know as fast as you want and then bam you got a traffic jam. Okay. And it's because there's

actually a very short interval between being really low utilization and not congested and low delay and having high utilization being congested and having delay. So basically because we are able to accommodate a higher throughput, it's as though I was adding more lanes in your highway. Okay? because you know that if you had more lanes in your highway, you wouldn't get that uh traffic jam as quickly, right? It would take a lot lot longer to get a traffic jam and therefore you wouldn't have that delay. That's exactly what's happening there. So, you can see that very very nonlinear uh behavior and that's extremely important for us. uh and

especially when you look at Ethereum, it's extremely important as we're looking to have our message sizes get bigger and bigger for a variety uh for a variety of reasons. Okay, I just want to touch very briefly on how we do it, right? I think the important thing is more what it does but it's uh it's fun and I think it's important to just touch briefly on how we do it. So the way we do it is we use uh we use coding uh where coding is making algebraic combinations and how we make these algebraic combinations is we make them using galwa fields. So

Evaristo Galwa, you know, local local person, well actually no, he was from Bular up up near uh Paris, but um you know, basically came up with this notion which is kind of like a generalization of George Bull's binary just two. He he basically thought of going beyond two. Um and what we do is we take representations in these uh algebraic finite fields. Um and we mix them in a completely decentralized way which is why I talk here about a decentralized code. This is very different from codes you may have heard of for instance um in traditional uh DA where you use something called read Solomon codes. Um

or you may have seen things like um uh like repto codes um and things like monot and so on. Those are end to end centralized codes. This is a completely decentralized code and in particular you have this unique ability to take coded pieces and create a new coded piece without ever decoding. And this is providing that completely decentralized propagation and will later be the basis for decentralized memory both ROM and RAM by having this RLNC only composability. That's what we're

implementing. And the reason we're called optimum is because that is actually optimum. Uh if any of you are going to be tomorrow at the P2P event, I'll I'll nerd out a little bit more and show you the math of why why it's actually optimum. So if you're interested, happy to talk afterwards or happy to to catch up with you tomorrow. Uh but basically, you cannot do better. Interestingly enough, you know, there's always a lot of discussion around, well, you know, uh should are you better off being centralized and you want to be decentralized, but you're actually just um you know, having to sacrifice something. If you gave me a centralized scheduler that

knew everything that was going on in the network and could actually distribute just so versus this completely decentralized stateless system where we don't actually even care at any node what other nodes are doing. You can prove it's the same. You would not do better if you had a controller. And that's not often the case, right? So how often can you show that if somebody gave you an omnisient, you know, completely competent scheduler and instead you had this approach where you have absolutely no idea what the other nodes are doing and you actually don't

care that the two will do the same. That's basically what we can show and that's really what you're seeing. That's why you're seeing this great gains. we are getting the best performance possible provably from the resources that are available and that's what optimum does and this is basically what I've been working on in my lab now for about two decades so you know it takes a long time to go from from the math uh to the uh to the actual uh implementation uh and this is this is really what what you're seeing okay so who's going to benefit from this well you know everybody L1's and L2s validators DAPs end users

things going faster higher throughput and basically cheaper you know everybody everybody gets a gain from this so I mentioned to you our test net I showed you some results on test net we just announced very recently who our partners are so we are extremely ly fortunate to be working with a fantastic set uh of of partners. Uh some of them actually are uh are here um at ETHCC and uh looking forward to to catching up with them. Um and and they've been they've been really fantastic and and it's it's really them and you know I encourage you again to to

take a look um if you want on AXA what we've uh what uh we've been doing. It's really them who have been, you know, the the verification uh that what we're doing is really inherently inherently useful. So with that, um check us out. We'd love to chat. Um we, as I said, right now we are on private testnet. Um, you know, we'll be looking at going uh on a public um next uh to to Hoodie and then and then to to having we hope a a public test net and um really looking to for forward to any questions that you may have with that. I think we have a couple minutes.

Great. Are there any questions? Uh one and two and three. Yes. Thank you. I was wondering with RLNC like how many times are you able to encode and thank you um I was wondering with RLNC how many times are you able to encode and if you do that a number of times then does the latency also decrease and with what function? Thank you. Yeah, very good question. So um because it is unstructured uh you can actually in keep a re-encoding indefinitely.

So you basically modify um a header with the uh with the coefficients and the mo the coefficients just gets updated. So you cannot look at a network coded packet and even know how many times it's been re-encoded. Uh it you're not appending new coefficients, you're modifying the existing coefficients. So you can keep doing it for as long as you want and it's very fast. So it doesn't add to the latency. Very good question. Thank you so much. Two. Yeah. Um I'm wondering um Ethereum pretty much have like block sizes of 150 kilobytes. Like how well does it compare this new method to with existing like smaller messages? Yeah, that's a good question. Uh so we did with the smaller

messages we started with that we were getting about a 2x benefit. um the really nonlinear benefit which I was showing you know gets as the number of messages goes as the size of the message goes up or alternatively if you have small messages but a lot of demand okay so uh here I only showed in terms of the size of the messages because it's a little easier to to just uh control one item at a time but even if you have small messages but you have um but you have a high uh high utilization high demand then you still see that that nonlinear behavior. So even with small messages and high demand you will see

that uh that nonlinear behavior. Is this lito based? So it's a modification of it uh to add to the coding. That's correct. That's correct. Yeah, very much so. I think there was another question. Unfortunately out of time I'll talk to you later. Okay. Thank you all. Thank you. Thanks.