EIP-7805 (FOCIL) Implementation in Erigon

Transcript

Thanks for being here today man. Uh the stage is yours. So hello everyone. So this Aragon in fossil in Aragon is basically my second project in Aragon because the first one which was SS query language was declined for inclusion from the next four. So I took this up during early October under my mentor Milan and so let's start. So till now there are four PRs in Argon. The first one was for implementing the fork heota fork time utils like for example we have some block timestamp checks for checking whether the current fork is the current time stamp is from

hegota or not and that is already merged. The second one was of for implementing the first engine API method which is basically the get inclusion list v1 for building an inclusion list below 8 kilobyt and the third PR was for a new payload v6. Now this is a very huge PR and it has 33 files which is currently under review but it has been approved with some nit uh suggestions which I will be fixing and then there is a fourth draft PR for the fourth choice updated V5 which is just still in draft and I am waiting on the new payload V6 because some of the work depends on that. So

yeah, so the three engine API methods, the first one, the engine get inclusion list v1 builds an inclusion list on the public meool. And the size of this list is kept at 8 kilobytes. And this has already been merged to main and is part of Aragon 3.8.0. And what the engine new payload v6 does is it takes a new parameter the inclusion list transactions and checks the block for the whether those inclusion list transactions are validly included or not and simply just uh returns a flag in the payload status which is returned whether the block satisfied that that inclusion list or

not. And as for the payload for sorry fork choice updated V5, it starts the payload builds with the IL transactions and it will also close another issue which is basically to increase the limit for increasing the IL inclusion list cache. And yeah, one more thing I mentioned that the fork choice updated V5 depends on the new payload. That's because the fork choice updated V5 needs a type of payload status. the new basically um the new field in the payload status which is the inclusion list. So that is something that we need in the fork choice. So I'm waiting for it to be for the new payload to be merged first.

Yeah. And apart from that I have also implemented the rest SSD compatibility for every every method that is that I have implemented for fossil. So that includes the inclusion list which basically returns an SSD list of transactions and this is also mentioned in the exchange capabilities so that engines like uh layers can see that this is available and use this. And as for the payloads V6 also I have introduced a new SSD codec so that the inclusion list satisfied is included in the SSD specs.

And as for a fork choice V5, it is still in draft and I have not built the SSG equivalent for that yet. Yeah. So how does get inclusion list V1 actually work? So it basically reuses that transaction pools provide TXense parameters which is in present in the Agon client itself and it is in the block builder path actually. And so what it does is basically uses 8192 bytes of RLP space and there's no gas cap and it just takes the transactions and skips blob transactions and any transactions like oversized transaction that go over the 8 kilob limits and just returns that simply and empty are basically never

null but it's just an empty array and as you can see encode and the type that we return for the traditional engine API v1 method is basically hex util.tes and as for the rests it's just an SSG encoded array. Yeah. So how do are we actually checking the uh check inclusion list in the new payload function. So what happens is we are not really checking whether the transactions could have include been included in a time before the while during the block building. What the check does is it basically tries to

append the transactions which are not there at the end of the block. But if it if it is possible to append that then it it just returns with a value of false in the inclusion list satisfied. If not then for every transaction in the inclusion list it will return true. So yeah if what are the checks that actually happen in the inclusion list. So if a transaction in the inclusion list is already in the block we just skip that transaction. If the transaction has not been included in the block but it does not have enough gloss to append to the block it just skips that transaction too. And if it's a blob transaction then it then also it is

excluded and it also does a pre uh pre pre-checks on the transaction like the nons and the balance checks. If those balance checks fail then the transaction was invalid anyways. So it is skipped and if all these checks pass and the it is deemed that the transaction could have been safely included in the block but it has been skipped. So it is like the flag is set to false for the inclusion list satisfied and it is returned and yeah how the work landed. So I mentioned that this project started in early October right. So from October 7th to 8 the four qs for the heota and the

inclusion list v1 engine API method were merged from October 17 to 22. The new payload V6 was uh the PR was opened and reviews were being made from Alex Dom and Copilot and there were lot of reviews and it is still like not approved by everyone. It is just approved by one maintainer right now and right now what I'm working on is basically the fork choice updated. I'm going through the specs and writing the tests.

Yeah. So challenges and how we solve them. So the first challenge was how the IL is not part of the block right. So how are we storing it? Because if we wanted to store it in a database, it would change the format and like with the release of a new version like every node would need to need to update it. So after speaking with my mentor the solution I came up with is that the inclusion list is for a block is stored in a cache and as for the a block can have multiple inclusion list right. So what happens is for a inclusion list that we are receiving we are hashing that and storing that as a key value pair for that hash. So for a block we can have multiple inclusion

list with each of them keyed by its hash and yeah the checks on the inclusion list whether the it could have been included in the block or not is checked on both the serial and parallel execution parts which is present in argon. Like for a parallel part it tries to basically use uh execute uh transactions in parallel for building the block and on serial it does not use parallel execution and yeah so there was a bug initially that all the inclusion list satisfied

flags were rever defaulting to true for blocks that are already known. So this was something that me my mentor and I had decided to for first go with but then later after reviews it was decided that we would just re recheck the inclusion list for every block whether it was known or not known. So let's see what the path is. So for example if the current it's a new block that we are receiving on a new payload. So we are just going through the normal process of checking the inclusion list and just return the do the checks and return what it is actually. And if we are receiving a block that is canonical and has been already received before then we check

whether the inclusion list that we received with it is in the memory. So if it is in the memory that we can just simply return the uh result that we derived earlier when we had received that inclusion list. If the inclusion list is not in the block then we just read the history of the state the post block and check it recheck the inclusion list with that state to check if that inclusion list is satisfied or not. And if it is still undecided then we just return syncing.

Yeah. So another bug that was uncovered during the reviews was that for example if a inclusion list is sent in the new payload v6. So one invalid and undecodable transaction means that the entire call fails with an error and that was something that completely missed my review. So I fix that by just when one transaction fails to decode what we are doing is we just skip it instead of returning with an error. And yeah blobs are also skipped and now that the as for the logs are missing line TX move from uh de warning to

debugs sorry like printing in the terminal. Yeah. So rechecking a block we have already executed. So as we know that the header only keeps the maximum of execution and a state gas right. So we can have actually three splits for a gas. So first of all is that suppose we have the header has kept the gas G. Let's suppose so the combination can be either both the execution and state gas is G and the second split is the execution gas is G and the state gas is zero and the third is that execution gas is zero and the state gas is G. So in either way the limit for the gas for

each of these dimensions is that gas limit minus G. So for when the execution for example suppose the execution is zero we are the gas limit is just GL and the gas limit for the state is uh gas gas limit minus the gas that we have stored. Yeah. So when a transaction is there for checking these transactions the transaction has to agree on both the dimensions. For example, when the execution and state gas both are at let's suppose G. So the gas limit minus a G, the amount of trans gas that is left, the transaction has to be part of

that gas. If that is the transaction does not fit into either one, then we return false as because the inclusion list is invalid. Yeah. So how were the reviews actually undertaken? So when we uncovered some bugs in the PR reviews. So what do we did was uh we just I added a test for each of the bugs that we uncovered specifically for that bug. And after that while testing he just used to revert the commit and use the new test that I have introduced with both the new commit and without that commit and check if that

test was returning false earlier in the earlier commit and returns true in the current commit. So that's how and he also checked the test manually. So yeah that's how we verified that the bugs that we were uncovering was fixed. Yeah. So the learnings so first of all is that one of the bugs that I mentioned was that for quick status return like for blocks that we have already known the inclusion list satisfied was defaulting to true. So actually the spec does not leave any room for shortcuts.

So the spec says that all blocks need to be checked for. So we are just checking each and every block whether it is known or not known and yeah the inclusion list contents should be treated as hostile input because any client can just send them and even if it is a malicious client it can send that. So we are not sure what we are just treating the IL contains as hostile input and checking the validity of that before actually checking if the uh it could have been included in the block or not. And as for the implementation itself, the work landed

so quickly and on to two engine API methods because we like nethermind is currently the most developed client in terms of fossil. So I went through nethermind code to see how they have implemented stuff and what they're doing different. So that helped a lot and yeah so what's next? So the next immediate task is to get new payload V6 merged and it is I think it is really close to merging right now. And after that comes the finish finish the fourth choice updated V5 PR and then the next task is testing. So run fossil deets against the other fossil clients

and the hive hive suites and then I will just document a new engine API method that I have implemented in the argon documentation website. So thank you uh thank you to Mario for conducting the fellowship and GZ to all the fellows and yeah huge thank you to Argon team too for reviewing the PRs and being patient. Thank you so much, Sagar. I appreciate all the work, all the projects, but especially this one. And thanks for being here. Yeah. Uh, do we have any thoughts, questions for Sagar? Anyone, anything? Yeah, Pavl, go ahead.

>> Yeah. Um, I have question. Uh uh if you have any feedback to the spec itself uh after all the work you've done um and yeah do you think like the spec is like what is the quality of the spec uh and if you would change anything there uh from your implementation learnings? Um, as for the spec, I would like I think it's pretty good right now, but if anything changes, I would just make a PR later on. And if there's anything I would like to change, no, I don't think so.

Like if that was your original question. >> Uh, yeah. Yeah. because um so like my my general uh question was um that I think uh the the people who who write the spec also look for some feedback from the implementers um about yeah how how easy it was to understand it and how like resilient is in in the implementation later. >> Okay. So the bug as I said the first bug that we caught initially that the inclusion list satisfied was defaulting to true for known blocks. So I think that was something uh it was more about

reading between the lines rather than explicit explicit specs. And yeah one more thing actually it's a good question because on the get inclusion list v1 we are specifying that we cannot include blob transactions but on the new payload v6 where we are checking the inclusion list it is nowhere mentioned that mentioned that we don't we can't include the blob transactions in the block. So yeah, I think that would have it would have been good to include that in the spec as well. >> I I've got a a question which is you've what you've implemented is spec. Have you thought about the RPC interface at all? cuz I'm assuming one of the

advantages of having this in Aragon is going to be if you if you've included stuff, you can do things like run a query to find out what the what the inclusion performance was. >> Uh, no, I haven't really checked the performance because I going to do that after implementing the three engine API methods. >> Yeah. Yeah. Now, I'm more thinking the decision to not put stuff in the database is going to mean that we're going to lose a history. So it's going to make it difficult to implement RPCs that effectively monitor the IL performance.

uh like the decision to not include it in the database was for from a performance perspective because if we just change the database structure what happened is in a new version >> like the database uh people would need the clients would need to change their databases too. >> Yeah. Yeah. But that's the tricky thing about writing a client node, right? It's a um >> if you just implement the uh machine spec, you make one set of decisions. Once you start thinking about the RPC requirements, you get a whole additional set of um constraints that quite often compete.

>> Yeah. So I am haven't really thought about it this way. So that is something I would have to look into. >> Okay. Thank you.