All Core Devs - Testing #98

Transcript

Justin Traglia: Yo, Barnabas, are you doing the, devnet updates, or is PK doing them?

Barnabas Busa: Pooja is the boy, you know.

Justin Traglia: He's away, so you'll be doing him.

Barnabas Busa: Yep.

Justin Traglia: Okay. Thank you.

Barnabas Busa: I mean, I can try, but…

Justin Traglia: Oh, would there be someone better then?

Barnabas Busa: you Stefan should be around, I think.

Justin Traglia: Okay. I'll update the agenda. Just say, like by Devops or something.

Barnabas Busa: Sure.

Justin Traglia: Sorry, not DevOps, PandaOps.

Barnabas Busa: Official DevOps, right?

Justin Traglia: I can never remember, is it? Protocol DevOps?

Barnabas Busa: Oops.

Justin Traglia: Dumb, dumb, dumb.

Barnabas Busa: I'll go adjust it in version 2.

Justin Florentine: ELLE.

Justin Traglia: Good morning, Justin. Good morning, Pooja.

Pooja Ranjan: Good morning.

Justin Traglia: Barnabas and DevOps, is the O capitalized or not capitalized? Yeah.

Good morning, everyone. Let's wait a few more minutes before getting started.

Alright, let's get started. Welcome, everyone, to ACDT98. Let's start with devnet updates. Is there someone here from Protocol DevOps who can give us an update on… The devnets.

Stefan Starflinger: Yeah, sure. So, in general, over the weekend. The network has been running decently stable, considering that there were some attacks, different, three different variations. Maybe Paweł can talk about them if he's on the call today. But overall, it seemed that the network stayed finalizing. They didn't seem… To impact nodes too much, and that optimizations have continually improved. The… basically… Robustness of the clients, to make sure that They can, even in these very edge-case scenarios, manage to process blocks, in at least around 6 seconds. Some Clients might take a tiny bit longer still in some scenarios, but on average, most clients are doing pretty well.

And just in general, wanted to remind clients, that the releases, hopefully are out pretty soon. So tomorrow is the deadline, so that we can prepare in time for the Shadow Fork. And one thing that happened also worth mentioning that the security team found bug in ethrex that would basically explode the memory of the ethrex clients. And that has been fixed already. And all of the nodes should be running smoothly again on devnet 8 as well. I think that's most of the main, issues. 4 does not add…

Justin Traglia: Great, thank you. I don't believe Paweł is here to give us an update on those other things. But maybe In a few moments, Maria will be able to discuss it. Okay, any other updates regarding the devnets, maybe from clients or anyone else. Okay. And yep, like Stefan said. Release this for Sepolia, should be coming out today, tomorrow, and then we'll make the blog post. I saw that Teku made a release, like. 9 hours ago, or 10 hours ago. When do other clients expect to make releases? If you want to talk about it. Yeah, would any client not be ready to make a release by end of day tomorrow?

Lodestar has a release as well. Oh, I didn't notice that. Great. Reth released this morning. I saw that. Nimbus who has a release. Nimbus is releasing tomorrow. Prysm says they should be out. Hopefully tomorrow, but we'll check with Preston. Grandine should be out tomorrow as well. Okay, amazing. Sounds like we're all pretty well on track. Okay, any other updates on devnets or Sepolia?

Barnabas Busa: Yeah. So just mentioned that we're gonna be rolling out the releases on devnet 8 to making sure that everything is still working. And then, once we fork the Sepolia successfully, then we're gonna be reverting devnet 8 images to use latest trunk branches.

Justin Traglia: Okay, cool. And, client says that fine. There's no… I don't think there'd be any issues with that. Correct. Alright, Maria, are you here to give us an update regarding the 200 million gas limit follow-up from, like. Last week.

Maria Silva: Sure, I just wanted to clarify, I'm happy to bring this up on another ACD if you guys feel important, but I think it should be… we… I really want to clarify, because on… the last ACDE, there was, this question around, is 200 million still safe, based on the slow blocks we were observing on devnet 8? And. Since then, we did a deep dive, and the takeaways are, yes, we are still safe, and the reason is. So I think Potuz was mixing up two different attacks that happened at two different times. So there was some attacks that Paweł did. And those actually… were not producing slow blocks. They were at worse taking 2.5 seconds to execute. And we verified these by

looking at the actual payloads during the devnet, but also, in the meantime, we also deployed that same attack in Benchmark Core. We are benchmarking it, and the worst client is also taking, at most, 2.5 seconds on that attack. So that's… all that… Set of attacks that Paweł was testing. They are safe. Now, there was blocks that were slower, and that happened before the attack that Paweł Eitan, it was something that Jochem was, deploying. And essentially, it was also a JUMPDEST analysis, attack vector. And the main issue was that clients were going out of memory. Once we looked into that, and it… we kind of, like, realized that the issue was actually because we were both running these

heavy, memory-intensive thing at the same time as we had the debug traces, on, which is not something we would expect to happen in a normal validator. And so, we also consider, based on the results that we were already seeing also from benchmarks, that that attack is also safe. It's just, it was, compounding of having debug traces on at the same time as we were pounding the node with this heavy attack. But without the debug traces, it would be fine. So, yeah, so our conclusion right now, based both on benchmarks and what we've observed on devnet, is that 200 millions is safe, at least from, yeah, from an execution time perspective, it is safe.

Justin Traglia: Great. Oh. Would anyone like to reply to that, or… Justin is asking, did we resimulate without the tracing on?

Stefan Starflinger: On the devnet, we haven't…

Justin Traglia: someone.

Maria Silva: Yeah, I was going to say that I'm not sure. I don't know, Stefan, you were saying that in the devnet? No, right? But…

Stefan Starflinger: Yeah, on the devnet, we haven't, turned tracing off.

Maria Silva: Right. The only thing we know is that doing a similar test on Benchmark Core, which doesn't have that, doesn't create slower blocks, but we haven't really tested it in a devnet without the traces to… Pre-simulate as far as I know.

Justin Traglia: Should we? I don't know either. Łukasz, thumbs up. Okay. I mean, let's look into that.

Maria Silva: So, just… just to be sure, like, the only difference here that we would be observing is that we would have the EL plus the CL. So this is… this is the main difference between running it on a devnet and running it on Benchmark Core, because on Benchmark Core, you only have the EL, and the CL is not doing additional work, per se. But we are also limiting the cores there, so… Yeah, we also take into account that not all the cores are assigned for the EL, but I think this is the… the main difference between the two environments.

Justin Traglia: Got it. That makes sense. Okay, anything else regarding this whole topic and the 200 million gas limit before moving on?

Stefan Starflinger: Yeah, I think it would just generally be good to understand all of the differences that there are between running it in the devnet and running it on Benchmarker.

Justin Traglia: Okay, yeah, I guess we can discuss that, like, on Discord or something, I guess. Csaba?

Csaba Kiraly: Yeah, I just wanted to mention that That will be much more sapphic. It's just good to be aware it's not an execution difference. So you don't see it on the benchmarks, you only see it on the network. Okay. We will see the effect of it on the network.

Justin Traglia: Yeah, Maria.

Maria Silva: Oh, sorry, it's, going back to the… to the previous point on should we test it or not. So I guess one question is, like, where should we test it? Should we… already tried to deploy something on devnet 8, like, have a few nodes that don't have debug traces, and then restart Jochem's attack. Or should we just do it on the main network? Yeah, I'm I'm I'm not sure what the best approach is there.

Stefan Starflinger: I don't think we have to do either or. I think we can do devnet 8 now, and the main fork in the… One and a half, two weeks.

Justin Traglia: Okay. Let's, try to do that. Sounds good to me. Okay, Moving on, I guess. Julian, are you here to discuss the new FAST, or… Considering adding a new FAST tag instead of repurposing the SAFE one for FCR?

Julian Ma: Yep. Yeah, so I wanted to share some learnings from trying to get adoption for FCR and propose a change, namely to create a new FAST block tag. So today, FCR, or the FAST Confirmation Rule, or the FAST Confirmed Blocks, are served behind the SAFE block tag, with a flag, usually. And the proposal is to create a new FastBlock tag for FastConfirmBlocks. The reason is that, we're seeing backwards compatibility issues. So previously, it was thought that justification, which is served behind SAFE today, wasn't used much, and that FCR has a very, comparable set of assumptions, and so just users that would use justification would be able to move over to FCR, and there wouldn't be many users in general. We're seeing now that while people aren't necessarily arguing against that.

It is hindering adoption that RPCs want to be backwards compatible for legal reasons, for many other reasons. And that's why they want to continue serving JustifyBlocks to their customers as well, because apparently there is a cohort of customers that relies on justification. And these backwards compatibility concerns are really hindering adoption, which otherwise is going quite well. So Chainlink just announced that it went live with FCR today. Other bridges and L2s want to use it as well. So really, the blocker is RPC support here, and RPCs in turn are blocked by backwards compatibility. And so the ask is to create a new FAST block tag. Mikhail already made a draft implementation, which I'm linking in the chat here. So the ask would be to look into that implementation and see how complicated it would be for you guys.

To Barnabas's point, the tag could also be FCR. I'm not very opinionated on the name of the tag.

Justin Traglia: I'd be interested to hear what clients have to say here. Like, just internal. Dustin, would you like to speak up about, your comment.

Dustin Brody: Sure. Yeah. So. Justification could always go forward, backwards, sideways, and any which way it was, as it were, not final. It was sometimes sort of propagandized as being, I was pretty sure that's going to be like this, but Okay, not backwards. But… That was… It was both slow. Although not as slow as finalization, and also not actually final. It was a middle ground which didn't make sense to begin with, and it also was changed… actually, if one looks at the history on the CL specs, maybe this is more from a spec-based argument rather than just my, you know, subjective view. It actually the definition changed of exactly what CL sent in forktories updated at partway through the history of the CL specs to

In terms of what Justified was. or what SAFE was. And SAFE became… sort of… the definition has already changed, basically. So, it was not consistent to begin with, and changing it again doesn't really break some kind of long-standing compatibility point. From that perspective.

Julian Ma: Yeah, I hear your points, and this is also basically the starting point that we started out with, which is why FastConfirmBlocks were served behind the safe tag, for pretty much the same reason that you're saying. What we're seeing now is that you can use the safe tag with some additional, additional logic to create some confirmations that are meaningfully different from fast-confirmed confirmations. And for specifics here, I would call on Francesco, but yeah, so there is a way to use justification to be, protect against different threat model than fast confirmation rule could protect you against. But I think the more important point here is just that empirically, we are seeing, apparently large financial institutions relying on the safe tag, and they have, larger governance processes of upgrading as well. So if we…

if we don't want to create a new FAST tag, that would just hinder adoption. And so I think that's more of an empirical point and less of a first principles point, but from a first principles point as well, you could use justification in a way to be safer than the fast confirmation rule can be.

Justin Traglia: Just to mention, Potuz had a good question in the chat, which was answered. He asked if RPC providers are claiming that they need to serve both, which Francesco said yes. That makes sense. Any other comments from clients? Questions? Anything? Do do we have a preference on fast versus FCR versus confirmed? versus some other alternative. It doesn't really matter. Justin Florentine says confirmed. Is it possible to have a… Or, sorry, is that a… possible option. It doesn't have to be 4 letters, does it?

I guess not. Kev is asking, would we eventually deprecate SAFE, or will SAFE be served in perpetuity? Julian.

Julian Ma: So what we're seeing RPC providers interested in doing now is that they would spin up dedicated nodes with the flag for FCR and for justification. I think as we do that and we see adoption going, maybe eventually there could be a path of deprecating safe. I think here we should just like. yeah, let's let users speak and see what they want to use. If people are moving away from safe. And now we'll be measuring who is using safe for these RPC providers. Maybe there's a way to deprecate it eventually. If people continue using it, then obviously it shouldn't be deprecated.

Justin Traglia: Okay, thank you. And, where is a good place for us to discuss this? Async.

Julian Ma: So maybe it could be on the draft that Mikhail made and otherwise I could open a thread on EF Magician somewhere.

Justin Traglia: Okay. I might also suggest, somewhere on the ETH R&D Discord.

Julian Ma: Right. That makes sense. Yeah, that works.

Justin Traglia: Okay. Okay, great, thank you so much. That was the last shared agenda item. Is there anything else someone wants to bring up before we split into the breakouts?

Mikhail Kalinin: Sorry, I have a quick question on the, this fast confirmation stuff. Before we jump into breakout rooms.

Justin Traglia: That's fine.

Mikhail Kalinin: Just, yeah, if there is… please share if there is any opinion. If we do this thing, like, let's suppose we decide to do this, what would be the best way to do it? Like, using a hard fork as a… you know, as some coordination point, or just do it after the hard fork, or… I don't know, if we want to do it in Gloas. If we can do it in Gloas, or if we do it… if we decide to do this, and use the hard fork as a, you know, coordination point, then… Yeah, would we have to wait for the next hard fork after Gloas to make this happen?

Justin Traglia: Dustin is saying that Gloas is very, very late. Do other clients agree with that? Or, like. Are we sure that we can't do the single loss? And Kev is asking, does this actually require a hard fork? Maybe Mikhail. You can answer that one.

Mikhail Kalinin: It does not require a hard fork in principle, but it requires coordination between CL and EL, but they will, yeah, they can use the capability, the capabilities endpoint of Engine API, I guess. So… Yeah, I don't know. Maybe hard fork would be helpful, but yeah, not necessary. Because, we are, we are, we will add a new, a new hash to FCU, so we will have to update the FCU. And, if we don't do it as a part of the hard fork, like, the CL will have to To know if, like, EL supports and you… Hash in FCU.

Potuz: Why do you want to change the normal node behavior? I thought we were just talking about this FCR as something that's optional, very few nodes would be running it. Why do we need to change the default node behavior?

Mikhail Kalinin: Yeah, just, because we want this, this new tag in, in, in JSON RPC.

Potuz: But why don't you just make it optional? If the EL supports it, it's fine. The node operator that is running this node should know this, that it's supporting it. And the rest majority of mortals will not care about this, neither this fork nor the next fork.

Mikhail Kalinin: Yeah, but then CL will need to somehow communicate that, what's passed on the FCU. it's not justified anymore. I mean, like, if we… if you safe block hash for this, is this what you mean?

Potuz: I'm just saying that we have now, some nodes that are able to serve fast. And this is going to be the very few nodes on Mainnet that are going to be using this. We can just keep having this as an optional thing, that very few nodes… node operators set up in versions that know that they support this, and we just don't affect FCU at all. Why would we need to, like, change FCU on nodes?

Mikhail Kalinin: Yeah, that's… that's one of the options, but yeah, maybe, Julian. And share his opinion on that. Why do we want this in the… I don't know, India. RPC, like in the JSON-RPC.

Julian Ma: Yeah, I mean, I'm not quite sure how exactly it has to be implemented. Truthfully, why we want it in a JSON-RPC. Is that it's to make it as easy as possible for RPC providers to serve it to their customers in a backwards, compatible way. Basically. If that makes sense.

Enrico Del Fante: Is this… is the… there is a… the only concern here is that if, for some reason, NEL go crazy because there is a new field set to null in the JSON? just those kind of incompatibilities, because otherwise there is just a new field in a JSON object that should be just simply ignored by ELs, if the CL serve it, or if it doesn't serve it, it's just the backward compatibility from the EL.

Julian Ma: Yeah I I say. I'm. Yeah, I'm not quite sure how exactly it should be implemented. My idea here was to speak with most of you here and try to get a bit of a survey of basically how complicated it would be and what would be the right way to implement it. So I could take it like that and give an update here, or continue conversation here, depending on what Justin thinks.

Enrico Del Fante: I mean, the easiest way is just not… not… Put this field into it by default, and then just add it when the fast confirmation is enabled on the… on the CL side, and when you do that, you want to make sure that your counterparty on the EL side also supports that. So it's a completely manual and optional To enable, otherwise it would be completely transparent for normal users.

Justin Traglia: Potuz, you had your hand up for a second. Is there anything you'd like to say?

Potuz: No, Enrico's just saying exactly the same thing that I was saying. We can just rely on node operators that require this to be smart enough to run compatible ELs and CLs that support this. And it's fine if we just add it on the engine API, but without changing the engine API of the vast majority of nodes.

Julian Ma: Yeah, I'd have to look into this a bit more, but yeah, thanks for the context, and it sounds logical to me, I guess.

Justin Traglia: Yeah, thanks for the input, everyone. Julian, is there anything else you'd like to say about this, or do you think the topic is done?

Julian Ma: Yeah, done for me. Thanks.

Justin Traglia: Yep, okay, so that was the last agenda item. Let's move into the breakout call so that we have enough time to do those. Oh! I've just shared the CL… or sorry, the link to the CL breakout. I'll be joining that one in just a second. Who here will Be able to take over the EL breakout. moderation. I saw Dan unmute. Dan, do you want to? Oh, if you're talking, I can't hear you.

Stefan Starflinger: I think it's just one point, by Barnabas. I think we can handle it somehow.

Justin Traglia: Okay, I'll let you guys handle it then. Alright, bye everyone.

Barnabas Busa: Hey guys. So, yeah, I just have a very small issue that I would like to bring up, that the… JSON file has the alloc keys without the 0x prefix. I'm just curious if we could, patch this in the eth-clients repo. I noticed this bug when, we were trying to do a mini shadow fork, and, Nethermind failed without, the 0x prefix. Using the JSON file because we stopped using the chainspec JSONs. This requires, approval from each client, I think, before we can merge it in, so it would be nice if you guys could take a look. Yep, that's it. Marius, do you mean the new bootnodes?

For a minute.

Marius van der Wijden: Yes. I don't think this was ever… Talked about in the call.

Barnabas Busa: Sure. So EF has a new bunch of, boot nodes. I think the PR was already mentioned, too.

Parithosh Jayanthi: It's done in almost all the clients. I think it's already been merged in.

Barnabas Busa: Yeah, the PR has been merged.

Parithosh Jayanthi: Not on Geth. I just flagged it today for Geth, that's why. But yeah, maybe just a brief announcement. The… so far, the bootnodes have basically been managed by the PandaOps team. We're handing that over to another team within the EF. So we're just migrating infrastructure from one set of machines to another set of machines, and there was a few reasons to rotate your IDE and things like that. So… We've made PRs. He's approved. Merge release. The existing boot nodes will continue on at least until like mainnet Glamsterdam, when we're sure that everyone's updated, and then we'll deprecate the old machines.

Barnabas Busa: Dan, we can still not hear you.

Mario Vega: Is there? Is there any other topic that we should discuss.

Parithosh Jayanthi: I guess maybe the only other one is, Paweł again sent a message about the, the blocks. I guess this is just a result of debug logs being turned on, and once we turn off, sorry, of debug being turned on. And once we remove that, it should be okay. Or is there anything else worth looking into with this block?

Danceratopz: Hey, hey, is Pari talking? Because I can't hear anything now. Maybe you can hear me.

Barnabas Busa: Now we can hear you.

Mario Vega: We can hear you.

Parithosh Jayanthi: Yep. Wait, did you guys get the question?

Stefan Starflinger: That's good.

Danceratopz: Good that you can hear me. Let me see if I can hear you. Otherwise, someone else can please step up. Let's see.

Mario Vega: Maybe I can take over. Pari, do you want to discuss this block, or do you want Paweł to…

Parithosh Jayanthi: I'm.

Mario Vega: far.

Parithosh Jayanthi: Make. I just wanna make sure that there's nothing funky happening in this block, and as long as it's in line with our current expectations, then this is fine. So, the, the message that Paweł put is that he's done JUMPDEST of 64KB and, slow execution sporadically. once the code caches are empty. So I'm assuming we're seeing 5 ish second blocks at at the p90 value, because we still have debug stuff turned on right? So once we turn it off, it should be probably faster.

Mario Vega: Does anybody else have context? I think this is related to the topic that we discussed earlier in the call. Is that correct? Jochem, you were mentioned about this?

Parithosh Jayanthi: Yeah, I think Marius just answered on chat. He said, yeah, it should be much faster without, so… Okay.

Maria Silva: Yeah, but just to make sure, like, again, I'm not sure what this blocking specific is about. I'm not sure if it's Paweł's attack or Jochem's attack. For Jochem's attack…

Parithosh Jayanthi: balance. He… it's Paweł's attack. He flagged the block, and he, put a message in R&D saying what the block does.

Maria Silva: Right, yeah, I think from what we are observing on Benchmark Core. This shouldn't be taking so long, so yeah, maybe doing the test without the debug traces and then observing again is a good idea.

Mario Vega: I think what I can ask from the spec side is perhaps we can take a look if we can reproduce this in the benchmarking test that we have. I do not have context, but maybe we can take a look async, yeah.

Maria Silva: I can say that JUMPDEST analysis is already implemented and running on Benchmark Core.

Ben Adams: Yeah, we're seeing like a massive difference between Benchmarker and the devnet. Not sure why. Assume it's the, the debug tracing, but it'd be good to… Do a check on the devnet without the tracing.

Stefan Starflinger: I'm still a little bit confused about the difference, because you created a compute test. And how is that, comparable to what is on the devnet? Because, from what I can see, there is Difference between the tests that you're running? This JUMPDEST and calling a 64… Kilobytes, contract. Or could someone explain how the compute test is comparable?

Louis Tsai: a So, this is a compute test, but it is still running on your home net. And it is named compute because in the past, we don't run it on a stateful environment. But right now, we are… But for both computing stateful, they're all running on your home net. Yeah, that is the difference. Not sure if this is clear to you.

Stefan Starflinger: We can talk about it more async. I mean, I get the general distinction, but I don't quite understand yet, how it's representative, the JUMPDEST, compute test.

Marius van der Wijden: It's not a compute test, it is just named Compute Test. It's… it should be named stateful.

Ben Adams: So the stateful tests and benchmark are tear down between and restart between each test and the compute tests keep running. It's like a big difference.

Maria Silva: And also, if I understand correctly, the… the way in… so the path that this test is stressing is not really the state access component, but really the JUMPDEST analysis component, and so in that sense, it's not the stateful test, in a sense that it's It… it doesn't… it shouldn't make too much of a difference the size of the state that you are testing it on. While other tests, like the test account access, or the test test store bloated, those ones, we expect the performance to actually change significantly under different state sizes.

Daniel Lehrner: But, I mean, the test that Paweł did, we needed to load around 4.4GB of contracts from disk, so I mean, that's definitely a factor. It's just you need to load the contract and then do the JUMPDEST. It's like both…

Stefan Starflinger: Yeah, I think that's what's confusing me a bit.

Maria Silva: Yeah, but just… just to comment on that, but still, code loading shouldn't… like, code loading shouldn't be impacted by, for instance, the size of the account trie, right? So I think it's in that sense that I'm saying that. The state size is not… I mean, it's still… It is still a difference, like, the amount of code you are loading, but in that case, the test is already deploying that code, so the code is there. It's just, yeah, having larger or smaller storage and account tries isn't really… shouldn't impact this. That's significantly from what I can see.

Daniel Lehrner: In Besu's case, it was also because, The existing state, and how many contracts are loaded during this block. completely… Gaozi, So basically, we have the code… we have a code cache. This test is loading so many contracts, basically, that we have evicted Everything that is not our cache. So, I mean… In a way, it does depend a bit how much of the contract… you have already in your cache or not. So, if the state is bigger, you have a smaller percentage.

Ben Adams: Yeah, but if it's also doing tracing, then you're outputting gigabytes of JSON at the same time.

Daniel Lehrner: Yeah, yeah, yeah, I mean, that's, that, that, that on top of it. This is for sure.

Mario Vega: I have a question. Do we have a way of reproducing this somehow?

Maria Silva: What do you mean on the devnet or on Benchmark Core?

Mario Vega: So just for example, if you sync and you hit this block, does the same, Effect appear again. Or. Or was it just one time?

Daniel Lehrner: If the BALs are still available, then I think you can do a performance check. If the BALs are not available, you run slower than during the normal… Block import. So, I mean, if you do a full sync and the BALs are not available anymore, it is blocked, and then you're definitely slower. What would have been otherwise.

Dragan Rakita: I'm looking at the block that Paweł shared. It seems that we have consistent around 5 seconds of execution time. Four six, four seven to like five. 5, 5, 5, 6. And this is one block that has 100 million gas. So if you dub. Double that… maybe with BAL it's not going to be double time, but it's going to… Strain your cores. Either way, There is like two paths. This seems okay to go with it. This is a TAG, but it seems that we are 5 seconds should be okay to propagate everything. The question is, do we want to increase the gas for the JUMPDEST analysis, or… push every client to make optimization the same as that Nethermind did, basically with some specific instructions.

Maria Silva: Yeah, I'm still not convinced about the repricing, to be honest, because again, like, we are running this on Benchmark Core, and the worst case is 2.5 seconds with a 200 million block. So, until we actually measure this without the debug tracer on, I don't think we should make any repricing decisions, to be honest.

Dragan Rakita: From reth's side, we are going to optimize it, so it should be a lot lower in a few days, maybe a week. But either way, it's up to the group.

Stefan Starflinger: But from what I understand now, the test JUMPDEST analysis It loads the contract straight into memory, and that's why that JUMP is faster. Versus Paweł's attack. Has to load it from memory. And that's why it's much slower, and that loading into memory is the main difference. Between the compute test and what we're seeing on mainnet.

Mario Vega: Alright, it seems to me that we have to analyze this further, is that correct? Do we have other opinion on this, or… How should we proceed?

Parithosh Jayanthi: There's maybe only… there's, like, one other data point here, especially when you're trying to do a lot of database reads, since we're running on, like, cloud droplets, they would perform differently than what Benchmarker has. Benchmarker has more dedicated machinery that That's sort of closer to what people run on Mainnet. So it might also just be a false positive that we see on devnets, because it involves reading from disk.

Stefan Starflinger: And those are network attached disks, so they're not local NVMe disks.

Parithosh Jayanthi: Exactly. So the latency would also be different. Yeah.

Mario Vega: Ben?

Ben Adams: I mean, before we did optimization, some of the benchmarker tests were like, if you ran it on one core, we were. 2 seconds, and now… We're, like.3. So, I mean, I don't think it's just loading it from disk, it's the actual analysis.

Stefan Starflinger: Since there's no more topics, should we… So… Close the call for today.

Mario Vega: Just one question, where can we follow this up? It seems like, it would be nice if we can keep on discussing this, too.

Stefan Starflinger: I think the benchmarking channel. on steel. Or interrupt, maybe?

Mario Vega: Sounds good. Go for it, Ben.

Ben Adams: I mean, can we switch off the debug trace on devnet and then retry it? I don't know if… There was a conclusion on that.

Stefan Starflinger: Yeah, sure. I can turn it off. And, reach out to Paweł, so he can run it again.

Daniel Lehrner: So, at least for Besu, when we switch to the release version… For Sepolia, it will be slower than what is currently in our main branch, because we have some optimizations in main that are not in the Sepolia release. Just, we might see worse. Most, most blocking parts than we otherwise have.

Ben Adams: Okay, that's a good point, because, Barnabas said they just… Just switched to our 2.0 release this morning, so that doesn't have the optimizations in.

Parithosh Jayanthi: Would you guys prefer that we swap back to trunk branches or some other version before we do this?

Ben Adams: All right.

Parithosh Jayanthi: We can swap back and forth, it's not a big deal.

Ben Adams: Yeah, if you can swap back to our main branch and then before doing the test. Because we'll have that out for, Hoodi. But we don't have it for Sepolia.

Daniel Lehrner: Yeah, same, same focus.

Mario Vega: All right, shall we wrap up? All right. Thank you, everyone. We can close the call here. Bye.

Transcript adapted from ethereum/pm, CC BY-SA 3.0.