Glamsterdam Repricings #3

Transcript

Maria Silva: Hello, hello?

Carl Beekhuizen: Hell no.

Ameziane Hamlat: Hello, everyone.

Toni Wahrstätter: Hello, hello!

Maria Silva: So, I think we can maybe wait one more minute to see if more people join, and if not, we'll start. And in the meantime, I will share… The agenda… For the new people joining, just to warn that we are just waiting a bit to see if more people join, and then we'll kick off.

So I don't think more people are joining, so I think we can just, kick off. So, the first, topic is 7904. So this is still very, preliminary, so it's not the final numbers, but I just wanted to show you, the recent runs we've been doing. So I'll just… copy the link here. So this is the… I put in the link for… for the meeting, but, this is the overview of how the numbers look after the Osaka fork. So the numbers we currently have in the EIP were still being run On Prague, and so they didn't have the transaction limit EEIP. And so we would expect numbers to be a bit better, after this fork.

And, in fact, we can see, for instance, on the pre-compiles that now a lot of them show, price decreases. Another… but still, this is something we need to double-check. But, so the main changes between these numbers and the previous ones, or the current ones in the EIP, are, yes, the Osaka fork. Second, Besu did implement some optimizations on, mods and div, related operations, and so we should, we should be seeing that here as well. We also did some fixes to some of the mods tests, because they were, so the inputs they were using were sometimes leading to some degenerations that actually Students… they were not, like, really estimating the full cost of the operation, and so now we kind of fixed that. And so the… you can see one big change is that I think numbers between the various mods are not… are a bit more consistent than in the original proposal.

And also, another, change was… So we have, in the tests, we do these loops where we run the same opcode, the target opcode, many times. So, for instance, if it's a mod, we are running this mod many times. But because of how we need to set up the test, we need to provide inputs to the operations, so on and so forth, we many times have these self-scaffolding opcodes that get bundled in the runtime when we are estimating the individual runtime of a single operation. And so now we are also Excluding, excluding these kind of scaffolding, opcodes from… from the estimation, so we get a more accurate. value on the actual runtime of that target opcode. And so this was something that we included as well, and we… we still… so…

For the Osaka fork, we are still missing data from Erigon and Reth. So the tool that we are using to do this is the one that the Nethermind team has been developing, and I think there was some issues running these most recent tests on these two clients. I don't have a lot of details on this, but maybe something to check after if you are curious. Also, there were some caching issues on some precompiles, so especially for EC Recover and EC pairing, we know that because the test is set up in a way that is calling the same opcode with the same inputs multiple times, there is likely some caching happening, and actually. The clients are not recalculating things, and so this will lead to

Numbers that look better than they are in reality. And then there's also, to have the final, final numbers, we still need to measure the performance on ball-optimized, implementations. And these will likely Rethuce the cost increases as well, because we expect, the parallelization to, up here. But just to give an update, so again, this is not the final numbers on, 7904. Oh, and I realized that I was not sharing my screen, and I meant to do it, sorry. I'll share it now. So these are the… these are the… the… the latest numbers for 7904. And if you see something that jumps to you, it'll be great to have some feedback async. Or if you already have some, some feedback, also to, to also have it in the call.

But yeah, are any comments here, or should we move to the next section?

Ameziane Hamlat: So, I already shaReth in, in the chat, just quickly on keychak, so it is cheaper, right? So the new cash would be 20 instead of 30. Right. And this was one of the opcodes that is actually pretty slow currently, so it was a bit surprised to have it cheaper.

Maria Silva: Yeah, me too. I think this is one of the cases that I think we should look into the test more… more carefully. From a first analysis I did, I don't think there's any obvious Hashing happening, because the inputs are always different. But this is one that I definitely think we should double check and see if there could be any weird behavior from clients on these tests. I'm happy to share in the report, I can share the link after. I have the actual test that was being used to estimate this. And so we, we, we can probably, yeah, if, if you, if any of you have time to look into it and… and give your thoughts, it would be great. Yes, Luis?

Luis Pinto: Just one remark on the Div and S, Div. Besu has a PR app for adding a new implementation based on amounts, optimizations. Which hasn't gone in yet, so… I would expect it to equalize more with not, because the algorithm is basically the same. Young. So maybe back to finding people license. Speaking all.

Maria Silva: Yep. Yeah, makes sense. I think, again, we still need to also do the final run with the ball-optimized operation, so for sure, these are not final numbers, so we still have time to take into account all the optimizations you are still working on. Perfect. Any more comments here, or should we move to the next section?

Dragan Rakita: I have a question, on what client was this run?

Maria Silva: on all of them, except Erigon and Reth, that were having, out-of-memory issues, if I recall correctly.

Dragan Rakita: So…

Maria Silva: Yeah. Gabe? Besu… And Nethermind are the clients that were run here.

Dragan Rakita: okay, maybe we can, like, talk in async? About why… I was, like, out of memory. But yeah, okay.

Maria Silva: Yeah, I think probably we could, I… so, Rafael from IthpandaOps and Camille from NetherMind, they are the ones currently running the tools to run these benchmarks, so they would be the right people to reach out to, but I think… I'm not sure if they were already in contact with your team or not.

Dragan Rakita: Yeah, sure, but this is on the state load network.

Maria Silva: Yeah, yeah, yeah. Okay.

Dragan Rakita: Oh, okay.

Maria Silva: Perfect. Just comment. It would be really nice if this can be reproducible.

Dragan Rakita: Because this doesn't depend on the state or anything like that. Maybe having something like the state test or something like that, that can be run and just, like, checked. A diss runs and times slower, faster, whatever it is. So you have… every client, like, has a slightly different implementation, and it depends on… You know, like. implementation depends how fast or slow it is. So, we need to be aware of that.

Maria Silva: Right, so I think… so my part of the analysis, the code is open source, I'm happy to share it after, so you could run these By yourself. I think for the data collection. I think it's a bit more complicated, but my understanding is that both NetherMind's Gas Benchmark Tool and Benchmark Core which is the one being developed by Fail from Itpendops. They are also open source, so you could also set them up and run it yourself. I'm… I'm not sure if someone in the call has more information on these that can chime in, but I… all of this should be reproducible.

Jochem Brouwer: Yes, maybe also to mention, that these, tests, they are the compute tests, so they are run on empty state. And I think… that you can… yeah, you can also run these benchmarks yourself, so, and, like, on Benchmark or on GasBenchmark to do, like, these local tests. These tests are also reported on both Benchmarker and on, via the guest benchmarks, so you can also see, the results of different runs, except Yeah, okay, we… I don't think they are targeted at a performance bench currently, so that's something we should maybe check, yeah.

Maria Silva: I actually… I think for Benchmark Core, at least, I think they are. I remember seeing that they are tracking better performance bench. And, just to clarify, actually, for these numbers, they were run on top of maintenance states, just because we were, at the same time, also collecting some state benchmarks that needed at, but it shouldn't make a huge difference, because the… these compute ones shouldn't have, like, a huge… Like, having it run on my net wouldn't make a lot of difference. Perfect. Any more comments here? So, Lucas, to answer your question, this doesn't take into account wall-optimized branches. I'm assuming there's still temporalization going on, but I think the main difference was actually the Osaka fork and the 60 million.

Gas limit transaction. And I think there might still be some… Issues with some of the pre-compiled tests, and we should look into them as well.

Łukasz Rozmej: Okay, so… Oh, those numbers actually puzzle me, because I expect that we will be increasing a lot of precompiles, and here we are kind of decreasing most of them.

Maria Silva: Yeah, yeah, yeah. So I think at least I know for EC pairing and EC Recover, these numbers are bogus, because essentially we are doing the same inputs, so there's definitely caching going on. For the other ones, I think we need to double-check and see if the issue is in the test definition, or how they are being run. But yeah, we need to… to… to double check on that.

DanielVF: Yeah, as I said in the comments, Monte had instrumented these and found that, these EC methods were much more expensive. And drastically raise the prices on them.

Maria Silva: Yep.

Łukasz Rozmej: Yeah, so this… again, this is completely not what we got from our benchmarking some time ago, from our benchmarks. I don't… I'm not sure what the current state is, Camille is… Leading that, is he here? Why isn't he here?

Maria Silva: I don't think it's…

Łukasz Rozmej: Maybe I should ping him. But that's, like I said, completely the other way, that's one thing. And second thing, I think we shouldn't, for example, decrease pre-compile costs now if we, for example, will be increasing them very soon due to ZZK cost, because that's kind of… Doesn't make any sense.

Maria Silva: So, yeah, just a quick comment on that. I completely agree. So, anything that is here with a cost decrease, if there actually were, at the end, with the final numbers and everything, a cost decrease, the proposal would be not to change it. So this is just the preliminary numbers for you to get a sense, and I think we'll likely Find some issues with these tests, or how they are being run, that will… Tell us why these numbers are having a decrease rather than increase.

Łukasz Rozmej: Okay, okay, that's fine, that's fine, but again, the two are completely… numbers don't agree with… my previous knowledge. Right.

Maria Silva: Yeah, agreed, yeah. Joram, do you want to go next?

Jochem Brouwer: Yeah, so Daniel and Lukas, you said that, yeah, your benchmarks say otherwise. I also think that These numbers shall be, well, at least be increased, but could you maybe share these benchmarks, so we can also check? And I think, like, this, this caching thing, it's interesting because this cache thing keeps, hitting us, because we don't want the best case here, we want the worst case, and this is, well, very hard to, to achieve using, like, the EVM level benchmark. So you can't directly go into the client, you have to make EVM tests, and then this cache keeps… hitting us, but yeah, okay. If you could share, like, the, these benchmarks, that would be great.

Łukasz Rozmej: Yeah, those were the, you know, the old benchmarks… well, I don't remember, again, I pinged Camille to… if he can… if he can join. I don't remember, I don't know the current state, I don't follow. I remember that we had those benchmarks when we created the block. And we pushed, you know, the block with, I probably are aware of it, right? That those are the earliest benchmarks that we used in Berlin on the Interop.

Maria Silva: And we pushed the block full of.

Łukasz Rozmej: opcode. Of course, you know, it needed, for example, also opcode to manipulate the stack, because those were kind… kind of real blocks. So it wasn't just isolated opcode, but the opcode sequence that allowed us to use this opcode. And we used, Data that isn't repeating, so it can… couldn't be cached, right? So, each opcode should have, each user.opcode should have a different data, right? So, slightly changed, so the cache couldn't be used. Is that how it worked? I don't know… I know those, like, those tests were trying to migrate it to eels, etc. I have no idea what the current state. I know that, this migration, I heard that there were problems, for example, with, with this, non-repitability, this, this, caching…

prevention strategy, I know that there was a problem. I have no idea what the current state, Camel, Camel knows that. I checked out a few months ago on exact details.

Maria Silva: Right, and I… I think on that, I definitely know that for, again, EC Perry and EC Recover, they are using the same inputs in the current tests. So that's definitely something we need to change. But I would also say that right now, with Osaka, you can't fill a block with just one transaction, right? So you would at least get, like, around, like, 4 transactions, right? So you need to split it by the 60. So you would already expect to see Some performance gain there, because now you can… That was my…

Łukasz Rozmej: Question, do you… those… does those tests, take into account parallelization?

Maria Silva: That was my question, and… I would assume that you would use the default for the client, right? Like, now you have a block with, like, a certain number of transactions that have the 60 million Limit, and then it depends on how the client's, place.

Łukasz Rozmej: But, okay, but for example, you can make a block this way that, because of previous transaction, the new transaction will have a slightly different data into that, going into that. Into that, pre-compile, which makes it more or less non-parallelizable, right? Until you get balls, right? With balls, it will make it not probable, so… you can still attack it, right?

Maria Silva: That makes sense, yeah. I don't know how this… what is the status on that, on these tests, I don't know if someone in the call knows, but.

Jochem Brouwer: Yeah, this, this is… Yeah, this is definitely something to, to, check, very, soon. But I also realized that NetherMind has a decent strategy to do, like, the parallel execution, and then there's also, like, a pre-compiled cache, I think, at the transaction level. And I realized that, for the Osaka test, where you actually spit up your transaction into multiple, this will actually make it perform, like, very good if, like, each transaction does not change their inputs Depending on the result of the previous transaction, because now you can use Parallelize all these transactions, cache them, and then when the transaction actually hits in your sequential run, then you can just read it from cache.

Łukasz Rozmej: Yeah, so we actually have a setting, I think, we added to disable this cache, if you want that, because we just wanted to have it for test, and Camille should join in a few minutes, so we can actually ask him more questions about…

Jochem Brouwer: If you could post this flag, and then I will check if this is already activated on the benchmark, because this is something we definitely need. But also, I think we should integrate this in the test anyways, because Yeah, like… We should generate these worst-case blocks, and also think about, like, client optimizations like this, and to ensure that this kind of caching cannot happen, and… I… I'm not… I don't think we have this yet. That should be added, but thanks. If you could post this flag, that would be great.

Maria Silva: Dragon, do you want to go next?

Dragan Rakita: Yeah, I would like to propose to remove changes to precompiles. precompiles… There are a lot of nuisances there, depending on the cache, depending on the library that's used. Depending on the implementation that's used. It feels… I feel that you… A lot more time to be sure that they are okay. So I think removing them, it's… at least I'm proposing it to remove them from this EIP.

Maria Silva: Yeah, I'm not 100% sure we can do that. Like, if you just… if we just… Like, blankly remove them, then we may not be able to increase the gas limit, right? But…

Dragan Rakita: What do you mean by that?

Maria Silva: I, I mean that if… So, for instance, I… again, I know EC pairing and EC Recover, they are being cached, for sure, because the inputs are… are… are… are repeated. So, if they actually require a cost increase, like. two times cost increase. This means that they will be bottlenecks that we are not repricing, and so they will be blocking us to raise the cost limit. Or were you meaning that if the numbers lead to a gas decrease, then we shouldn't include them? And if that's the case, then I agree. So, as I was saying.

Dragan Rakita: I'm also…

Maria Silva: As I was saying before, this is just the numbers that I'm showing, but the rationale would be, if for some reason, with the final numbers, we've still figuReth out that some precompile was more efficient than we thought, and it would require a gas decrease, then we won't change it. Like, here we are just making gas increases in this EEIP.

Dragan Rakita: Yeah, to be honest, make the worst case be coveReth by gas is the, like, the best case for us. If we can manage that, that would be great. And yeah, I agree that we need to do that if we want to increase the block gas limit. So yeah, we should include those. for example, the Blake… pre-compile, you have, like, instruction, AVX instruction that you can use to speed it up, or you can use, like, default implementation that's slower. It's, like, very… I have both of those inside the REVM, but depending on what platform is used, it could be one case or another case. So we want to, like, for precompiles, we want to, like, match them in the… for the worst case.

So there are a lot, like, nuisances around that. We should be careful. But yeah, I agree with you, we should definitely increase it, because we need it for the increase on the blog, yes, I mean.

Maria Silva: Right, and I think this is exactly the reason why I think it would be helpful to have more of you looking into the tests and see if there are, like, any particularities in your client implementations that might mean that we are not really capturing the worst case. Like, any help here at this point, I think would be much appreciated. Conscious of time, I think we should move to the next topic, so maybe if there are any more comments, we can continue these. async. I think the next update is around 1838, and Joram was going to chat a bit about the benchmarking of state operations, and, the update there. Joram, do you want to go next? I'll stop sharing.

Jochem Brouwer: Yes, let me share my screen. Wait, this… boom, boom. Interesting. It's horrible. Alright, can you see my screen?

Ameziane Hamlat: Yes.

Jochem Brouwer: Yes. Cool, okay, great. So this is about ERP 8038, about the benchmarking state operations. So, a quick recap, it's about the, about the change of the state access operations. These are the parameters which it is targeting, so this is anything which has to do with changing things to the state, so this has to do with contact, storage. And accounts. And the nasty thing here is… again, the cache. So we just talked about the compute, caches, which are, well, they can be nasty, or, like, they, they, they optimize the, the benchmarks. And that's not what we want, because we are actually targeting the worst cases here, so we have to take care of these caches. And in the EVM, we have the cache accounting, so we have cold storage and barn storage, both for the accounts and the storage.

And this is what I wanted to mainly go into, because this is what we have been tackling in the last, well, week, two weeks. And the EVM cache, this is from ERP2929, And what it does is it tracks which accounts and which storage has been accessed, and if it has already been accessed. then… The cost to change this value is, decreased. So we get, like, a discount. And this happens on the transaction level. So there's the EVM cache. For the benchmarks, we had the initial setup. That, if we want to target existing accounts or storage. We have this setup where we Create these accounts, or these, the storage. Before running the benchmark. And the reason for this is that in the accounts, but also in the stores, you have to pay extra if you start creating accounts, so this is one of the reasons from the EVM. But also, from the clients, we see that in some clients, reading

The storage or the accounts, which do not exist is actually the worst case for them. So we need to have tests for both accounts or storage for which does not exist, and for which do exist, for both for the EVM constant or the non-EVM. content, so like the, the, like, how the database operates, like that, and, also about the caching strategy of this client. So, what we have seen, and I will call it a cash monster, because we have just seen that this is not nice in the, in the computer test, but it is also very not nice, Not to… yeah. On the state level. So normally, if we have, like, cache hits on the mainnet, we are, oh, we are, we are, like, it's, like, very nice, because this increases our performance, but if we hit it on the benchmark, this is not very nice.

So, Maria, made these graphs, and you can see on the left the cache hits and the misses without preset keys. And on the right, you can see it with the preset keys. So, with the preset keys, we actually started to create these keys in, like, blocks right before the Benchmarks actually being run. And what we see is, like, the difference between these two is, like, not so nice. So there's a problem. With, like, this cache, because it looks like, well, most values are actually in this cache, which is not what we want, because we want to, We want to… to get, like, the worst case, analysis. Yeah, and just to recap it, I think most of you know, like, the tests I run on top of a mainnet or a BloatNet snapshot, and BlowNet is the network which is a fork of mainnet, and we just spam, like, a lot of state on top of this.

So, what we have to do is we have to battle this cash monster, and the new strategy is to ensure that the tests are actually… they are forced to go to disk. And this means that the snapshot which we use to run this benchmark on have these keys, so these storage keys and these accounts keys already preset. And, Oh yeah, and we have to skip in the test itself, so we don't want to have the set of this in the test that, like, in the previous blocks, before the benchmark, we, the benchmark block, that we actually set up these keys or these accounts, because this already puts them into the cache, and that's not what we want.

So what did you do on bloatnets? And I want to know that this bloating is, like. Well, very, slow, like, it takes a long time to create, like, a reasonable amount of accounts, or, like, mainly storage, that's, that's, that's the main thing. So what we do is we build ESC20-like contracts with storage, and we also upload a range of accounts to query EOAs, so accounts without code. And we also want to target contracts. The reason for this is that we want to target also ERP2780. So 2780 wants to reprice, intrinsic gas costs, and also gas costs if you query accounts with code or without code. Like, now there is a different… this introduces different costs if you

target accounts. With code or without code, so we want to directly integrate this. And, well, as a nice thing, like, for the target contracts, we do not need to upload this, because we can use an old contract on mainnet, which has created many accounts in the past, like, it's almost 430,000 contracts, which is the old ENS PTS3, so that's, like, very nice to resurrect one of these, old contracts, so we don't have to bloat on BloatNet, like these extra accounts. And the reason that we target the create opcode is because this generates a pattern of contracts, sorry, of addresses into the EVM, which we can directly, generate in the EVM without, like, a lot of cost, because otherwise we have to hard code the addresses into the EVM, and this is, like, both complex and,

We also have to figure out, then, a way to actually hard-code these things, because we have the contract size limit and all kinds of challenges. So, on BloatNet, we have the… these new steps of different sizes. There's one missing that's already there. There's also 30GB of addresses, and we also want to see, like, okay, what Does this contact size of the source itself have an impact on the benchmark? For the benchmark targets, we have 3 levels of cache, and to also see, like, if there's, like, a difference. We have the no cache, so this is the part where we directly read from the snapshot state, so from disk. the caching at the THX level, that's where we target the state where it is both warm in the EVM and it is warm in the cache, so we actually target the storage slots, or the accounts, into the EVM where we first query these accounts before actually doing the opcodes which we are targeting for this benchmark.

And there's also the caching strategy at the previous block. What we then do is, like, in the previous block, we query these accounts on the storage, and then in the next block, we do the actual benchmarks. For the accounts, we target EOAs and, contracts. And we also call these with value and without value, to also see the impact of changing our counts here. And for the storage bytes, we target these ESC20 targets, the existing and the non-existing keys, because, well, reading non-existent keys in some clients is the worst case. And also, reading keys is always necessary for storage writes because of this gas cost calculations. So if you create, like, new keys, you have to know if the key already exists or not, and a key which has zero in it, so it's non-existent, changing it to a value is actually expensive.

And, we also check, like, okay, what if we write exactly the same value, or what if we change the value, what is the impact? For the storage reads, we target different sizes EC20 contracts, so this list which I showed before, and also, like, the existing and the non-existent keys. So, what are the next steps? So, right now, the broad net step is being taken. What we then do is we run these benchmarks on top of the snapshot. And then we analyze these benchmarks, well, if there is a problem with the test, we fix it, and, well, the main question is, and I likely… I would be very surprised if this, result would be a no, but did we fight this cache monster?

Correctly. Alright, that's my, those are my points. Any questions, points, remarks?

Maria Silva: You can go, Daniel.

DanielVF: Yes, do you have a rough idea at this point? how much increase you're thinking? I know the EEIP says TBD. On, like, read access.

Maria Silva: I say it's still TBD, and it's hard to say, because on one hand, we do expect a pri… like, based on previous, benchmarks and how Zen We do expect there to… the fact that we need to increase the price, but at the same time, with ball-optimized state reads and, state route calculations, it's hard to say how much more performant clients will get. So I think, in my mind now, it's very hard to say what will be the numbers. I would expect An increase, but how big it is, it will depend on the ball optimizations, And how… Clients will behave after that.

DanielVF: Okay. Yeah, coming from, Monad and, doing a lot of performance testing there. State, you know, as you get into thousands of transactions a second, or 10,000 transactions a second, just the speed that things can be read from the disk very much becomes a bottleneck, and is probably the primary bottleneck, at that… as you get up toward that speed. So, we increased the cost on cold reads, just because it is very much a bottleneck. But, again, if you're running slower transactions, that may or may not be the case, or it may not matter quite as much.

Maria Silva: Right, I think it will be great. Once we have the… more numbers on these, it will be great to compare and see how things differ, but I think at this point, it's really hard, because we are still… I think the numbers we have right now are not reliable enough for us to have a good view on it.

DanielVF: Yeah, that makes sense. I guess just the one thing I want to throw out there is that, Disk read speed can be another access of, you know, the hard balancing act of getting, all the different dimensions of gas pricing together, but it's definitely a fundamental Fundamental constraint. performance.

Maria Silva: Right, thank you. Amisienne, do you want to go next?

Ameziane Hamlat: Yeah, thanks. So, yeah, thanks, Yukam, for the presentation. I think it makes a lot of sense to have, like, these different scenarios. My question is related on how will our product sort of productable are these tests? So I guess the idea is to take, like, the snapshot of the blood network. then run on top of it, like, all these tests, and then we can delete, you know, like, everything that was generated, and then run again, run again. Is that, like, the idea? And the other question is, like, you said bloating is very slow. And I was wondering if it is related to the network itself, or to the, like, the benchmarks. Are we benchmarking the state, like, state creation in a way that we are creating, like, a lot of, basically, state as well?

Jochem Brouwer: Okay, so for the, for the first question is, like, how, how, how reproducible are these, these benchmarks? So, these are East, benchmark tests. They are specifically targeted for BloatNet. But, if you, like, create, like, an EC20 contract with the functionality which the benchmark needs, then you can also point these benchmarks to these contracts on any network. So you can also generate, like, a pre-state yourself, or a set state yourself, from, like, empty state. And run it on top of that. maybe… yeah, yeah, to answer your question, like, how are these benchmarks being run? Like, does it, like, delete all the states when this benchmark happens? Yes, that's exactly the case. So, how these benchmarks are being ran?

Camille could explain better, but I think I… well, if I explain it like this, that this would be, Correct? Like, these are, like, clients, you start them up, you run this benchmark, and then you kill them, and then you do, like, the next benchmark. So this should get rid of any caching or, yeah, whatever. I think that answers your first question. The second question was… No, I don't. Now I don't remember. But…

Ameziane Hamlat: was related to blue tin. You said gluten was slow, and I was wondering…

Jochem Brouwer: Yes. So, the problem with bloating is, like, it's not really about the network or, like, if you want to, like, stress test the network there. The problem is that we are really being constrained there by the EVM. So, the bloating speed's essentially, like, you have to go… you have to do, like, a lot of overhead operations all the time. So you have to run the transaction, you have to win the blocks, but that also means you have to do, like, the state rules calculation and these kind of things. So this is, like, very slow. If you have to upload, like, 50 gigabytes of state, well, you can already calculate it yourself. So for 50GB of state, you would need 50GB divided by 32 storage slots, and creating storage slots is 20,000 GAS.

So you can calculate how many guests you would need to actually build this thing, and that's, like. the hard low limit of this, and of course you have, like, some extra overhead, like, intrinsic transaction cost, the Osaka thing where you have to split the transactions. Well, you can imagine that this is not very fast to do this. Okay, thank you.

Maria Silva: Thank you. So, conscious of time, I think we'll move to the next, section, and if you still have any comments or questions, please let's do it a sync. I think now we have Spencer. He'll, talk a little bit about 8037, so the State Creation Gas Costing for CEIP. And his update on the specs and tests for Definite 3, and then also we'll chat a bit about The… changes they are doing to the entire testing framework to then accumulate all the repricings for DevNet 4. Spencer, you want to go?

Spencer Taylor-Brown: Hey, thanks. Thanks, Maria. Yeah, so… last… last week on… I think it was Saturday, we… released, or DevNet Free release, with… Our, like, initial set of tests for… 807, and Yeah, it seems… seems to… seems to have gone well, like, I'm glad, clients are passing, I'm glad Besu's passing most of those tests, which makes me happy. With this release, we have… Hard-coded the cost per state byte to, 1174, which is what it equates to when you use a block gas limit of 100 million. And… So the reason we did that was because of, complexity in the framework issues, and we wouldn't be able to get a release out, on time before the definite.

So… For the next EVNET, at least over the coming weeks, we plan to… Yeah, update our framework to handle the EEIP correctly. So the issue being, that we've never had this, I guess, extra dimension of the block gas limit affecting gas costs within our framework, and it affecting a lot of tests, and I'd say the biggest difficulty has been the, static tests. We call them static tests, basically the… tests from Ethereum tests, they're YAML files that we are filling to JSON, so we have, like, a separate way of doing that. And a lot of those tests were failing. The hard part is that, yeah, changing those tests, to… past the spec for 8037 is not really possible in YAML, so, we've made good progress on porting these tests to Python, and then once they're in Python, we're going to update the logic for these tests for 8037.

I'd say the overarching goal for me, and, I guess for DevNet4 is to, integrate, 8037, 78… no, 2780, 8038, and 7904, all into the framework with all the tests passing. Really just… Hardening everything so that… Yeah, the framework's more robust. And I think we should be in good shape for DevNet 4. Yeah, that's, that's my update. I can keep it brief. If there are any questions, please ask. I guess, I also have a question. M… Regarding, benchmark tests for 8037, Do we want any specific tests to target the… Reservoir mechanism, or should we… Or are we okay with the existing, State operation…

tests that we have, for a blood net. I'm not super deep with those, so that's, what I'm asking. Thanks.

Maria Silva: Yeah, I think that's a great question. So, the tests on BloodNet are really just… benchmarks, they are not really tests, so they are just trying to assess the execution time of all state operations, so I don't think they are good to test that the clients are implementing 8037 the way they are supposed to. And so for that, I do feel like we need some sort of tests to to… to test. I don't have a better word for it, but we need tests that, kind of allows us to see these new code paths, using the reservoir. So I think it would be… it would make sense for us to have something around that.

Spencer Taylor-Brown: Sure, sure. Just to clarify, so we have… we have tests to make sure that clients are up-to-date with the spec, but for… benchmarking the reservoir specifically, we don't, so if… if we… if we need some of those, I can, take a note and take a look at that as well.

Maria Silva: So then, can you… can you explain what you mean by benchmarking the reservoir? What do you think is missing from the tests you already implemented?

Spencer Taylor-Brown: Well, the… the tests so far are just, like, spec compliance tests, so there's no… Like, I assume… I'm assuming the existing tests that benchmark, I guess what I'm trying to say is, because there's this kind of new code path. With the, the second dimension of gas. Should we have an extra… should we benchmark that specifically? Or are the existing tests that we have enough? To cover that. My understanding is it's the latter.

Maria Silva: Right, I think my understanding is also the latter, so I would assume that Adding this second, dimension, would it increase significantly execution time of things, but if… if it does increase, you'd still see it in the test that we are running right now, because we are considering the variance where the, for instance, SSTOR creates new slots, and so we would be able to observe on that test, like, an increase in in execution time. So, definitely once… and it's something we can definitely do, like, once we have a DevNet with all the repricings. can Retho all the benchmarks and see if there are any new bottlenecks, but I wouldn't expect us to need… new tests, Because with the tests we already have, we should be able to… to see it.

yeah. I don't know, Jocham, you're going to say something as well, right?

Jochem Brouwer: Yeah, yeah, yeah, yeah, yeah. So, so, okay, so I'm not… Deep into, 8037. like the current spec. But what we should do is we should… so we have, like, the spec test, and we have the benchmarks, and for the benchmarks, what we target there is, like, we want to target the worst case. What we should check. is if this ERP introduces, well, new code paths, or, like, new ways to, abuse, export, or, like, are there, like, new scenarios Where we have, like, these worst cases happening. And if there are new scenarios which are then just not coveReth, or likely not coveReth by the benchmark tests, or the benchmarks, then these should be added. So there should be, like, an analysis of this.

The question is. who is going to do this? Because I don't think I have the bandwidth to do that right now.

Maria Silva: Right, and I, I do think there's… first, on, on, on, on, on one sense, there is… The question of, now, state creation gas is not included in the transaction limit of $60 million. Gas units, so we could actually have New worst cases that we should think about. So yeah, maybe it's something to put as a to-do, and then try to dedicate some time to think about this.

Spencer Taylor-Brown: I can, I can take a look over the next week.

Jochem Brouwer: Cool, cool.

Maria Silva: Thank you, Spencer. Great. Any more comments, or can we move to the next section and lasts?

DanielVF: Yeah, one question I have, given that Block times are scheduled to… are desiReth to go down, and probably will go down. How does this interact with that? Because it seems like if you just follow the formula, we're actually making state, cheaper as block times go down.

Maria Silva: Yeah, so it would adjust with the… With a block limit. I think once we do shorter slots, we would have to refactor that in. Because at the end of the day, what we want is to keep a steady… like, we want to enforce, like, a long-term state growth. So, now if we are doing… more… So that the… the sort of the… the block space should, like, the… the cost in relation to the aggregate block space should become constant, shouldn't… shouldn't change. So once we are doing the zigzag or the short… if we end up doing the short, Slots, we need to… Do the analysis and likely update the cost per byte. But I think we'll… we'll… we'll hit that when… we will hit that… we'll cross that… that bridge when… when we get there.

DanielVF: Green.

Maria Silva: Perfect. So, conscious of time, any more comments, I'll, ask them to be async, and I'll now call Bhuta. He has an important update on the community outreach for repricings. Bhuta, the floor is yours.

Butta: Yes, thank you. Now, I feel bad for not having a nice presentation, just like you have. But, yeah, for the survey itself, right now we have, 19 different entities who filled it out, mostly application developers, around 70% of them are application developers. Other ones are, yeah, wallets, validator operators, or node operators in general. In summary, I would say, so for… I will, I will go through each EEIP, so it's a bit more, clearer. But for the first 5 EEIPs, which is 79.81, 7976, 7904, and 7778. There were no controversial opinions, so most of them, if not all of them, were fine with it. With 2780, there was one question, I would say, which also led to a significant remark from them, asking whether

Deploying contract is also affected by it, because if it is, then also smart wallets will be affected, so maybe that's a question. From Maria. Something we also did is, or I did is, everyone who had a high concern with any of the EEIPs, I discussed it with Maria and then reached out to them to clarify. How they're affected, and if they're affected at all. So, maybe this is something we can discuss afterwards, Maria. For the last two EEIPs, 8037 and 8038, these were more the controversial ones, because, I think… so out of the… 19 people, 6 were very concerned that it's extremely complex, to integrate, and also to use. Maybe also worth noting that, the survey was filled out by larger entities like Lido, Yearn, Arvware,

handle, like, more of the DeFi applications. So, yeah, I think 8037 is the most controversial one. But yeah, we reached out to them, so depending on whether this EEIP actually makes it in, this is the one that, yeah, I would say is a bit questionable. Right now. As for the next step, so this is me reaching out in general, we will see which of the entities are actually affected. So some of the data was already deliveReth by Tony, where we reached out to some of the Layer 2s. I think it was, One was Polygon, and one I would… I'm not allowed to mention, I think. I also reached out to Arbitrum and Optimism, and see what they will…

think about, generally, the gas repricing. So yeah, the Layer 2s will also be involved in… in their opinions, and the big… the big stakeholders we're missing right now are wallets, which are a bit harder to reach out, it seems, but I will try my best to get their opinion during EFCC. And then we will have the big picture. And in the final step, this will happen… End of this month, ideally. is when Carl provides his in-depth data, and to see who's actually affected, we will be able to reach out to affected entities specifically. Yeah, I think that summarizes. Ideally, next time I have also a nice presentation.

Maria Silva: Yeah, I would just add, and thank you, Bhuta. So, my understanding was that the two biggest concerns around the state's operation EEIP, so 8037 and AT38, Was, on one hand, the actual increase, so a worry that making things more expensive could… Hurt some applications, either because things get too expensive, or because there's some, like, cold pots that need to be updated. And the second one was on the cost per byte, so the dynamic… the fact that we have a dynamic cost per byte now leads to some more complexity when Optimizing, apps, and so I think on the final point, it's, to me, the one that we should, and I think it's something we'll do in the next steps, as we follow through with these entities, is trying to understand, like.

With the… so we do have now a quantization trick that makes the… dynamic cost per byte a bit smoother, so only changing when there's big, increases, so if this already makes things better, or, Or not, and then, actually, how much worse does it get in their overall experience? Because sometimes things, just because they are complex, they seem More complicated, but then, like, the final… Day-to-day impact is… is not… that's, big, so I think it's, it's, it's understanding, the actual, the actual concern by getting more more context, But yeah, so those… those… I just wanted to add those two points. And, Tony, do you have, something to… to add?

Toni Wahrstätter: Yeah, just to… just to quickly out, to add, the one entity that Buda left out is Starkware, and what I was meaning with you don't need to reach out to them was more because I already reached out to them, so not, like. they don't want to be mentioned or something. It's very obvious that Starkware, who is not using blobs, is affected there.

Maria Silva: Thank you. Are there any more comments or topics? Great, thank you so much, then, for attending, and we'll see each other in two weeks.

Jochem Brouwer: Thanks, Harold. Bye-bye.

Louis Tsai: Thank you.

Transcript adapted from ethereum/pm, CC BY-SA 3.0.