REST-SSZ Engine API for Lighthouse

Transcript

timing. Thank you so much for being here folks. Thank you so much for being here. Samir, go ahead. Hi guys. Uh my name is Samir and for the EPF cohort I was working on the rest engine API implementation lighthouse. Shout out to Mac from lighthouse for helping me out during the dur duration of the code whenever I required any help. All right. Uh why did why are we refactoring the engine API to the rest SSD to rest SZ? Uh CL already holds uh SS. So currently the current procedure is the execution client has to uh has to uh uh has to

convert JSON uh has to convert uh the sorry the consensus client has to convert the J uh JSON to SS. uh and with what happens with that is SS brings in hex inflation which essentially doubles all the hashes, transactions, blobs, the travel. So moving towards uh SS where the execution client essentially converts uh RP to SS and then transfer directly SS. So the size that travels the actual size and we don't have to worry about the hex inflation. This allows for blob scaling where essentially we can raise the per

block blob limit as well as it increases the root to scale much further since this increases the uh bandwidth for all the engine API calls. So the approach while implementing this was that the rest engine API would be an opt-in feature. So it would be enabled via command line and and uh in case rest engine API connection with the E does not work, we would fall back to the JSON RPC which is a try and tested since since the merge.

Also just as an implementation note I try to make sure that the changes done for while implementing this is confined as can is confined to the execution layer crate of the lighthouse codebase so that the changes don't spill over to various different crates and yeah shout out to Marius for creating the initial engine uh rest engine API I spec which op to Amsterdam and shout out to Barnabas as well for uh creating for adding the Bota extension to the rest spec.

Yeah, talking about implementation. So essentially this is how the endpoints uh are work from uh JSON RPC to rest endpoint where we've if you notice we have move moved from uh method versioning to essentially uh placing the fork inside the header call itself. So just for example instead of having to maintain method versionings of new payload from v1 to v5 you just it's all compressed into a single endpoint and depending on which fork uh fork the uh node is running at that particular point

or depend depending on the request required waiting on the requested fork the schema will is decided. Here is an example call. Uh like I mentioned for AM uh you mention the uh fork in the execution version header and uh and this is just an example of like having an HTTP2 connection which was picked up from the spec itself. Also yeah I forgot to mention a couple of things the previous slide that uh uh the method versioning is done for uh fork choice update get payload and get payload bodies by hash as well uh get blobs are their independent endpoints so

we don't so for them you don't have to send across the fork and for client version and exchange capabilities it's unscoped where you actually receive a JSON And yeah uh get inclusion lists v1 was just recently added where uh it is uh method versioned for now but just for simplicity sakes I've just hardcoded uh to hes since that it's only supported for hes. Yeah. So spec features just go through this pretty quickly like now the rest relies on the forkhead of over method versioning. So each each method uh call

will have its own sec containers uh that needs to be implemented in the seal clients as well to a certain extent where then you convert it to your internal types. uh since we need to support both rest SS as well as JSON RPC we need to uh mean so we create a new rest SS engine client and we create a new transport team where we give priority to rest when connecting to the EL and if that doesn't happen uh we move to the JSON RPC and that logic is part of the transport team one feature that was added in the spec is that folk choice update calls the serialization on this is now in the

CL clients and not uh the EL client. So, and we are adding HTTP2 support with HTTP 1.1 as fallback for the rest engine API client. So, it'll be like a three T system where we initially try to connect with HTTP2. If that doesn't work, we move to HTTP1 all with rest and if both fail then we fall back to JSON RPC. So current status is that I had initially created a uh the draft PR that has the complete implementation of the rest engine API but the diff is around 6,000 lines plus and to improve ability I've just divided into five different ps

where with the base PR being adding the wire containers for STC and each one of them is stacked on top of each other where the other four are created locally in repo for now. So once each of them start getting merged, I'll open the next PR and uh I created another draft PR but this is just to uh showcase that since he uh pota spec was added to the rest I was able to implement uh I was able to add that support for the heet branches.

So uh so this PR was built on top of the he definite 2 branch and I was able to run a devet for he whose metrics I'll show in the upcoming slides. I was also able to run the local devet for gloss with uh with uh let me just recall yeah with red nethermind ether and as well up to clas. So that worked pretty well. So moving up to hea was the natural progression.

So the results that were captured just to set up the prerequisites the devet fork was he I've not written he go just for technicality since we have not had a heota devet yet and the e client user was red lighthouse like I mentioned the spammer config was like 10 transactions per slot and two blocks per slot and uh yeah this was a configuration with a number of nodes and just one thing to note is that most of the ratios are in are JSON RPC divided by rest SEC. So above one basically means rest is faster or is ahead.

And the last slide is just cap it capture results between HTTP2 with HTTP1. So like I mentioned initially uh for almost all the request and response bodies we uh SSD saves half the byes except for fork choice and new payload payload responses which are actually quite the response is quite small. So the saving is almost up to 3/4 and yeah you can see that the bytes per engine call is basically for each bar denotes the uh addition of of both request as well as

response bodies. So the biggest gain you can see is when get blobs v4 and get payload where for get get blobs we move from 5.1 MB all to in JSON RPC all down to 2.57 latency this is latency over HTTP2 connection where rest SS wins over uh on every other large payload on JSON RPC But I was seeing that it sits below par with folk choice update with attributes and inclusion lists. But sort of a good part like I mentioned

earlier was that the size of the request and response is extremely small. So there can there can be room for improvement here. This doesn't mean to say that HTTP2 does is not to be uh uh considered but it's just that this was implemented with a shared HT H2C connection. So maybe one approach could be to essentially create a small pool of H2C connection so that pork choice and inclusion list can have their own connections and the rest of the other or like get blobs and uh get payload can have their own since they were showing better results.

Yeah, this is because with HTTP1 we see that all the engine methods uh are above par. So at least we were sure that it's not the we can see that it's not the encoding that's causing the costs but more in the way how we are transporting it with HTTP2. uh but in overall the block production in both HTTP2 and HTTP 1 the ratios were quite similar where essentially we were almost 40 45% faster in both uh so this is great for scaling in the future as well uh where like yeah uh like you can see HTTP1 is still slightly bit faster

but then again like I mentioned HTTP there is room for improvement with HTTP2 And lastly just as a comparison slide between HTTP2 and HTTP1. So currently in the uh current implementation seems that HTTP2 is costing just a bit more compared to HTTP uh one but block production is nearly similar. So like I've mentioned maybe one fixable is essentially on the client side we create a small pool of H2C connections future work. So currently I've got the implementation working on glance and his local dev nets and I've got PR for that

respectively. Hopefully by the end of the year I'll can improve HTTP2 support try and get the PR merged and uh uh have support up till glam slam and at the same time try and push for to be part of uh heota test nets as well to have this implementation tested and with from next year onwards essentially have the complete heota support after spec updates and eventually we replace it replace JSON RPC engine entirely with rest after enough enough uh testing and it's run on mainet for a long period of time.

Yeah, thank you. Thank you so much, Samir. Awesome. Uh do we have any questions? Any thoughts for Samir? Yeah. Um >> hi. Great talk. Uh I was just wondering uh like did you see uh you said that you used like approximately five blobs for all the curtosis testing that was the max amount that you used or did you also test for like higher blob counts and higher gas limits and I'm curious if you'd expect the latency improvements to be just linear or would like is there anything that would like if you have 21 blobs for example do you see like much

better improvements than with smaller number of blobs or something like that. >> Yeah. So I did test out with bit much more blobs. The maximum number of blobs I tested out was bit was 10. So with that uh with that what I saw was the ratio stayed the same essentially where uh the latency ratio as well as uh the bodies uh the bas size ratios remain the same but the absolute numbers show that uh rest worked far better compared to uh compared to JSON RPC. So I do see that as we scale as we increase that number with as the absolute values will also

show that difference really well. >> Okay. But like the ratio was constant is what you're saying like the >> Yeah, that's what like the ratio from what I remember fiddle between 2 to 2.5 depending on various different runs I did >> like 2x improvement on the SSE. >> Yeah, exactly. >> Yeah. Okay, great. And uh just uh curious did you use Axom on lighthouse for the uh library? >> No the library that I used for HTTP connection was the request. >> Oh sorry sorry >> that's on the client side.

>> Yeah. >> Thank you. >> Awesome. Thank you. Any more thoughts anyone? Okay. Oh we got eaten. Yeah. Yeah. Awesome. Um in li P2P they are moving now to like quick and http3 and kind of skipping the http2. Do you know like why we are using http2 here and not just skipping ahead to quick as well. >> All right to be honest I'm not I wasn't aware of that. So I'm I'm not in the space to answer that but I would actually this is interesting experiment that I I would like to do post >> I was just to test this out and yeah

I'll share it once I'm done with Sure. Uh thanks a lot.