Node-reth: Flashblocks-aware Ethereum Client - Cody Wang - Base

Transcript

Uh hello everyone. My name is Cody and I'm uh one of the engineers on the base chain team. Uh today I'm going to talk about no rest which is a uh flash block flashbox aware rest client that we built when we were launching flashbox on base mainet. Uh today I'm I'm going to be giving some quick overview of flashbox and then uh going to some details of how no rest works. Uh yeah. Uh yeah. So just some quick context on flash blocks. Uh what are flash blocks?

Uh they are pre-confirmation blocks that are built every 200 milliseconds. Uh for opt L2s. uh there are uh because uh opts generally have two second block time and that means there are 10 flash blocks every two second and uses 10% of total gas limit and then transactions in one two second block equals to the sum of transactions from all 10 flash blocks. So from this diagram here uh you could see that on the left hand side is the regular two-cond block without flash blocks and then on the right hand side is with flash blocks. You could see that one block is divided into multiple flash blocks and each contains subset of transactions.

And uh second the effects of flashbox uh just some quick overview of how it worked on uh base manet. So uh we collected some data on the ETH get transaction receipt call that's flashbox aware. uh you could clearly see that there's uh pretty much 10x improvements if you send a transactions from the US or the EU uh from the Asia is a little bit slower but still like significantly faster than before. Uh so uh thanks to flashboss and also the RPC that we uh built so that people could enjoy this uh significant UX influence uh and also dive just a little bit into the architecture uh of the L2 uh before

and after flashbox. So before uh flashbox is a very standard OP stack L2 architecture. Uh this is like super simplified. So we have the sequencer on the left hand side. it's inserting to second blocks to the EVM state uh which the sequencer also ha has it uh but I'm just like showing it there for simplicity and then we have the RPC nodes that sync up to the chain uh that syncs the two second blocks so that it could serve the data to the uh to the RPCs uh to the RPC callers um and after the flash blocks uh I I'm mostly highlighting two components here that are new uh one is the builder and then The other is web soocket proxy. So we

have this builder that's communicating with the sequencer that's building the two 200 millisecond flashbox and then uh it also streams out this 200 millisecond flash box to this component called websocket proxy which is the wi-i which handles the external connections uh to stream out the 200 millisecond flashbox. Um so now the RPC nodes they still sync to the two second block uh uh they still sync the two second blocks as usual but they also set up these connections to the web socket proxy so that they could listen to the 200 millisecond flashbox in order to serve those data back to the users. Uh so uh so yeah that's like the updated architecture uh with flashbox

and uh so yeah so basically these RPC nodes after flashbox are uh are are basically using no rest uh which is the flashbox aware client that I'm going to talk about more uh and it's a ref extension uh what that means means is that it's a REST binary and it's importing REST as a library. So that uh it's still a REST node. Uh and the amount of code that's required is pretty minimal because RAT made it really easy to import RAS as a library. Um and then uh the main part is that it overrides the ETH API uh module so that it could uh so so so that you could add some

customized implementation for the RPC methods. Uh currently there are total 10 RPC methods supported that are flashbox aware. Uh those including like east get block by number east call uh east transaction receipt and etc. So uh no rest basically uh from my point of view it has like three main components uh which are highlighted here. Uh the first component is a flashbox subscriber. As I mentioned earlier, there is a websocket that's streaming out these flashbox. So we need a websocket subscriber that subscribes to the flashbox. And then because uh it's also a REST extension, so that

means it's a REST client that that that continues to sync the canical chain as usual. And then uh the next component is a state processor which is probably the most important part of this uh uh this software because it handles uh things like it would execute the flashbox uh in into some pending state and then it was it also does state management between the flashbox and the connectical box which we will see more later. Um and then the state processor uh uh computes this pending state that's that's used by the last component which is the RPC handler uh which uh uses that uh state

to what's happening here. Hello, hello, hello. Okay. Uh uh yeah, the last component the RPC handler uses this pending state to eventually uh serve the data for those flashbox aware RPCs. So the first part is the flashbox subscriber. Here I'm showing like the flashbox data format here. Uh it's uh it basically has five fields. Uh the first one is the payload ID which is the payload ID per flash block and then there's uh this index which specifies uh which flash block it is. The index

ranges from number 0 to 10. Uh and then there are three other fields base diff and metadata b uh each uh they each represents uh some part of the data of flashblock uh which I'll show a little bit more here. Uh I'm not sure if people can see it clearly but this is like uh one example of a flash block data. Uh so as you can see on the left hand side we have the payload ID, the index which is zero and then the base field which contains uh data that that are constant throughout the entire block uh throughout the entire block uh uh like it contains things like parent hash uh fee recipient

block number gas limit time stamp which are the fields that do not change. Uh and uh next there's the div field which contains the data that do change per flash block because it has things like state root, block hash, gas use which are different for individual flash blocks. Uh so they do change and then it also has some meta data field which are useful for uh the uh the pending state to serve those data back to the RPC method. So things like new account balances, receipts, uh those are necessary for some of the RPC methods.

So they're also included in the flash blocks uh format. Uh and uh next is the state processor. Uh it essentially has like two cues to receive and process flash blocks and also the connectical blocks. And it also executes a flashbox to build penny state overrides because those are necessary for methods like east call, e simulate because they require the actual s uh the actual executed EVM state. Uh and uh the state process also stores everything in this data structure called pending blocks. Uh it also performs some checks.

uh so for example uh for connectical blocks it is basically going to clear pending blocks when some conditions are met which I'll explain more later and then it also checks for uh for flashbox as well because uh for for example if uh it re if it didn't receive flashbox one but it receives flashbox two it's not going to uh update the pending uh state because it's missing some some information on the previous flashbox. uh but if the ordering is correct then it's going to update the pending state uh uh when it's valid.

So uh here's some examples of like how it's building the pending uh uh how it's processing the connected block and also the pending uh state. So let's say there's at like time zero uh this node has connect 100 and then he hasn't seen any flash blocks. So the pending block is none and then at time one it starts to see some flash blocks coming in. So let's assume that it start to see flash blocks for block number 101 and 102. So that means we now have a pending stage that's built up to uh block number 102.

And then next uh it start to receive connectable blocks again. So here uh you just received connect block 101. Then now we start to do some checks. So we're going to compare the transactions from the connectical block 101 and then the pending block that's that keeps track uh so it's going to check between the connect block 101 and the pending block 101 and then if they have the same if they don't have the same number of transactions then we think there might be some of the rework situation going on for flash blocks. So then we're going to drop the pending state 101 and then rebuild this pending state using the connectical block 101 and the pending block 102. And if they're the same then we think like the flash blocks uh do

have all the transactions then we don't do anything here. And then at time three let's assume that receives another uh connectical block on gen two that means like the connectical blocks have already caught up. Uh in this case the pending state don't really do anything for us. So we just drop everything from the pending state. And then lastly is the last scenario. Uh let's assume this node has received a bunch of uh data. uh it starts to with connectical block 100 and then start to receive flash blocks from 101 102 and then receives another connectical block 101 and then receives another uh flash blocks from 103 and then eventually

receives connectical block 102. Uh in this case it has a lot of information in its pending state. Uh so this pending state grows pretty big. So we added another variable called max depths which uh kind of restricts how long this penny state can grow. Uh so in this case we have max depths of two. Uh and then when the connectical block 102 comes it's going to be checking if we have the pending state too long. Uh if it's too long then we will drop some of the previous pending blocks that we keep track of. In this case, we will drop the pending block 101 and 102 and then we will rebuild the pending state uh based on connectical block 102 and the pending

block 103. So uh to summarize like how we build this pending state here uh we basically can build the pending state based on the canical block just before the first pending block that we keep track. Uh so like the previous example, we will eventually start to build this pending state based on the last connectical block and then the uh pending state afterwards that we've seen uh from the flash blocks. Uh and then it also uses EVM cache to kind of reduce the amount of work uh in the scenarios where we don't prune the previous pending blocks.

Uh it's al also it's also going to clear penny state if it's seeing reorgs as I mentioned earlier. uh if it sees that uh the connectable block transactions do not match with the transaction that we keep track from the flash blocks uh or it's going to clear the pending state if uh the connectical block has already caught up uh to the pending blocks and also the performance part. We have this max steps field that would help us to prune the pending state when it's necessary so that we keep the flashbox processing time uh pretty quickly. Uh because every flash ball comes every 200 milliseconds. We have to make sure that it's able to process uh this entire thing within a reasonable

reasonable amount of time. Uh if I reme remember correctly right now we process every flashbog within 30 milliseconds uh for for base mainet. So um so so it so it's still able to do pretty well. And then the last component is the RPC handler. Uh uh like I mentioned before this this data structure called pending blocks that uh that's computed by the state processor. It basically contains all the information that's needed for the RPC methods to uh to be flashbox aware. like it contains things like uh the flashbots that are used to build this view. Uh also uh account balances, transaction

receipt, transaction state, state overrides. Uh uh yeah and uh just one quick example of how a RPC method kind of works here. Uh uh so I'm using an example of block by number. Uh as we all know block number, block by number takes in two fields. uh the block number or the tag and also the boolean uh condition if it wants to show all the details about transactions. Uh so in this case it's basically just going to check if the tag that's passed in is pending. If it's pending it's going to be looking at the pending state that I showed earlier to get the pending to get the latest pending blocks and

then eventually return back to the user. If it's not pending, then it's just going to go to the original uh RPC block uh call and then it's going to just return the latest block back to the user. So, it's pretty straightforward here to uh to override the implementation once we have all the penny state ready for it to be used. And how to run no rest? Uh you could run the no rest directly from the repo but you just need to specify some of the flashbox related flats uh because uh you need to have it to listen to the websocket in order to uh get those data uh or the other way is to run from this

base/node repo that we also have uh which contains uh three different client ref and also nethermite. uh you could simply just update the M and then run the docker compose. It should also run the no rest uh pretty easily. Uh la yeah here lastly I'm just uh having uh I'm just showing some additional links if you're interested. Uh we have our docs page that shows everything about flashbox like how you will integrate with the websocket how you could run a node rest node and uh also has definitions of uh all the flashbox related stuff. Uh and we also have like blog post on flashbox if

you're interested in learning more about flashbox. Uh yeah that's pretty much it. Uh if you have any questions, I'll be here. Uh and uh feel free to come to me to uh ask more questions. Yeah. Uh that's it. [applause] Thank you very much, Cody. There are actually a number of questions in the Mircat uh app in there. We won't be able to get to all of them, but I would like to get to one of them. It was the top question that was there. Um have you had any uh pre-MP reorgs and uh how often?

>> Uh yeah, we used to have uh like we used to have uh some amount like I think few months back but after we changed some of the uh implementations in our block building site now reworks happen like very rarely. Uh, so you really shouldn't really see much reorgs that's happened. Uh, we also have a dashboard called bass. Uh, I forgot the name, but maybe it's bass.orgstats that that that we show the reorg rate for flashbox. So, if you're interested in checking out the rework rate in real time, you could go to that dashboard.

>> And then just as the followup to that, uh, how have DAPs managed them? Do you have any downstream knowledge >> uh, >> on the on the reorg? um how would uh I think if they want to be really really safe, they should listen to uh the regular blocks as well. But I but I think the but I think because right now we already have the flash block rework rate to be very very low. So they can similarly just kind of assume that there's no re rework just like how they assume base don't really have rework. Uh but really depends on how you want your applications to be. If you're really strict about rework, then

maybe you want to be more safe. Uh so I think that's app dependent in my opinion. >> All right. Again, a round of applause for Cody. [applause]