Live State Fuzzing: Testing Live Deployments Proactively

Transcript

Hello. Okay, there we go. Uh hi, I'm John. I am the CEO of Veridise, and today I'm going to talk to you about something we've been working on lately, which we call live state fuzzing, which is basically a method of testing deployments after, you know, the smart contract is deployed, obviously, in order to identify bugs proactively. So, let's start by taking a look at the security workflow that we see nowadays. So, basically, there's a phase right now where someone evaluates their project locally. And the the thing that we want to examine is kind of the trust assumptions that someone is making when they are evaluating the security of their project. So, with these types of local deployments, what we see is

dependencies are often mocked. We also see limited interactions because usually you have developers who are like designing the types of interactions that are being performed. And you also see a closed environment in that it's usually only a set group of users that are interacting with the smart contracts itself. It's not open widely to the public, and so that kind of biases again the types of interactions, but also kind of what happens or is modeled on the blockchain in general. After this local phase, some projects will go through a testnet phase where things, you know, open up a little bit more, where at this point they might work with more realistic dependencies.

They might not work on their mocked contracts, but still you have limited interactions because it's usually just the developers who are interacting with your contracts, and still it's a closed environment in that you're only modeling certain types of interactions with the blockchain at large. And then finally, once things hit mainnet, we see a more realistic environment being or I mean, you're in the most realistic environment where dependencies are real. You have arbitrary interactions from any user, and anything can happen on the blockchain. And so, the reason why I'm bringing this up is because we want to look at the techniques that are used in each particular phase. So, in the local and testnet phase, you use proactive methods in order to try to identify bugs early,

but developers in the mainnet phase rely primarily on monitoring. And monitoring is reactive because it essentially waits for an issue to be flagged so that they can basically go and contain the fallout. So, essentially, the thing that we're pointing out here is that proactive security ends at deployment, which is what we want to look at a little bit closer. So, let's look a little bit at the issues that can pop up. So, here, basically, we want to look at the implicit environmental assumptions that can pop up. So, basically, you've got a contract, it's interacting with a set of testing users, and those testing users might interact with the contract or dependencies. This testing phase occurs for a limited amount of time, and so you've got, you know, N interactions

which are being modeled. Once you hit a production environment, though, you've got arbitrary users who are interacting with your contract, with the blockchain at large, they're making changes, and your contract can be live for many, many transactions, which means that things can change pretty dramatically from when you were testing. And so, essentially, in that testing phase, you have implicit assumptions which are introduced by, you know, mocked dependencies. You also have these limited user interactions, so the types of user interactions which might identify a bug might not occur.

And then you've got a controlled user set, and so that controlled user set means that uh certain types of interactions which are common on the blockchain, like changing configurations to dependencies, changing your configurations and stuff like that, might not be modeled. And so, this is what some of the things that we see breaking once things hit production because in production, you can have if your mocked contract was not really modeled correctly, unanticipated dependency behaviors, you can also have unanticipated user interactions, and finally, the evolving blockchain state can cause errors that you do not anticipate because they didn't they they didn't appear at the time that you were actually designing the smart contract

itself. So, why is this a problem? Well, let's take a look at post-deployment security. So, basically, currently, you deploy your contract onto a blockchain, and then anyone can interact with it by submitting a transaction. So, the way monitoring generally works at a very high level is monitoring is going to inspect those transactions, and it's going to classify them into bug or not bug. If bug was identified, then it flags an error or an alarm, and then the developers have to go and they have to try to uh contain the fallout. So, basically, the attack is already in progress, and so the developer's main job at that responsibility is to try to figure out how to mitigate the attack as much as possible, but typically, the

attack has already occurred, its funds have already been lost, and so they have to figure out what to deal with after that. So, essentially, what I'm talking about here is we want a black box. That black box takes in the current state of the blockchain, and it tries to identify bugs. If a bug is found, an alarm is still raised, and then developers can make changes, but the important thing is those bugs didn't come from users, and so that gives developers time to actually go and fix the issue rather than contain fallout. So, why do we want to do this? Well, at least $348 million was hacked in 2024.

Quite a few came from audited smart contracts, but one key thing is a lot of issues also came from smart contracts that were audited, but the bug was out of scope of that audit, and so while audits are clearly having an impact, there are issues that, you know, pop up with audits. And as bad as 2024 was, 2025 is worse. We had I mean, these four bugs already sum to more than 2024, and we've seen a number of different errors which have a number of different kind of root causes. So, for example, Setus, which happened at the beginning of 2025, was caused because of an out-of-scope arithmetic error. You also saw like Abracadabra, which happened, I think, closer to the

end where there was an issue with the dependency modeling, and other more traditional errors also popped up as well. So, how are we going to create this black box? Well, there's a couple of things that we could do. We could perform more manual reviews, where every so often, maybe you get someone to go and inspect the state of the the blockchain, maybe after certain events happen, but we'll look a little bit more into that in a minute. You could perform AI reviews, where rather than having a manual reviewer go and do that, you have an AI statically inspect the blockchain state as well as contracts. You could use fuzzing. So, fuzzing is a dynamic technique that randomly interacts with a

a contract in order to try to find issues. And then finally, you can use formal verification, where if, you know, there are implicit assumptions you have on dependencies, you could go and verify that the implicit assumptions hold within those dependencies. So, if these are the techniques that we're going to look at, we can evaluate them on these particular criteria. So, we want something that is scalable because blockchains are actually very large software. While individual contracts are small, the blockchain at large is big, and there is quite a lot of software that interacts with each other.

We want something that has low false positive rates because developers are occupied with other things. They don't have time to go and sort through false positives. And finally, we want something low effort on behalf of the developers because, well, they're already used to monitoring, which is very low effort. And so, basically, the like the closer we have something that plugs into what's already being done, the more likely it is to be used. So, if we take those particular techniques and we classify them, this is the classification I came up with. So, manual review, not very scalable, doesn't really perform well on any of the metrics. Formal verification is great, but formal verification can take time, especially if you have to perform formal verification often because dependencies change or anything like that. AI review

can possibly be scalable and low effort. It's unclear to me how scalable it would end up being, but at the very least, one thing that would likely pop up is false positives, and so determining whether something is a false positive or not would still be a challenge for developers. And finally, there's fuzzing. Fuzzing is something that we think would satisfy all of these, and we're going to take a look at how. So, in order to use fuzzing in this way, we created something which we're calling live state fuzzing. So, basically, what you're going to do is you're going to take the transaction history, you're going to take a specification which classifies something as a bug, and you're going to take the blockchain state. That all is going to be fed into

a live state fuzzer, and the live state fuzzer is basically going to process the transaction history, it's going to fork the current state of the blockchain, so you're operating on realistic state, and then it is going to create a fuzzing campaign over those specifications. It's then going to run for a certain amount of time, and it's either going to say, "Yes, there's a bug, and here's the bug." or "I couldn't find a bug." That doesn't guarantee that no bug exists, it just means that one could not be found. So, like fuzzing, this does rely on randomness, but the nice thing is live state fuzzing can make use of some information that already exists, and I'll go into that a little bit later.

So, basically, this is very similar to fuzzing, but we're going to pull the live state frequently and fuzz for a certain amount of time. We are going to use the transaction history in order to improve the efficiency of fuzzing by mining information from that transaction history, and because we are starting from different live states at different moments in time, we can penetrate deeper into the smart contract logic than traditional fuzzers can. Because typical fuzzers, you know, they run for a certain amount of time, they can only make it K transactions deep, And so, by continuing to use uh state or interactions that users have made with contracts, you can go farther in that contract state.

So, how does this work? Well, basically what we're going to do is we're first going to fork uh mainnet locally. Then, we're going to analyze the transaction history in order to learn certain information about the interactions that were being performed and use that in order to help guide the fuzzer. [snorts] Then, we're going to initialize a fuzzing campaign. So, this basically is just taking specifications which say exactly what a bug is defined to be, and then initializing a fuzzing campaign with those specifications. Then, we simply run the campaign. This is going to run for a certain amount of time, try to identify bugs.

If one's found, we raise an alarm. Very simple. And so, the important thing here is we're using the evolving state of the blockchain in order to identify concrete counter examples, and we're doing so without human interaction, without relying on other people finding attacks. And so, we are proactively identifying vulnerabilities before attackers. So, what can this do? Well, uh we've used our fuzzer Orca in order to find a number of different vulnerabilities, but I'm not going to go into that. Instead, I'm going to talk a little bit about when fuzzers struggle because some people here might be familiar with fuzzing. So, fuzzers in particular struggle when you have uh an exploit that requires very specific inputs or an exploit that requires complex

interactions. So, maybe it requires several transactions in order to execute. And uh basically, what I'm here to say is that live state fuzzing can help with both of these. So, let's take a look at how. So, first let's take a look at the Beanstalk hack. For those of you who aren't familiar with Beanstalk, basically what happened was uh at some point someone submitted a malicious proposal to the governance, and then Beanstalk had a restriction that proposals could only be executed 24 hours later. So, after 24 hours, the uh malicious user then went and executed the proposal because Beanstalk had this corner case in their logic, which was an emergency commit, which said, "If someone with enough tokens executes emergency commit, we don't even have to go to voting. We can just execute this proposal immediately."

So, how would live state fuzzing help with this? Well, I told you that these complex interactions can be difficult for fuzzers. And so, the important thing here is that uh fuzzers can be used in order to kind of interrupt attacks that are in progress. So, first of all, if within those 24 hours, we used a live state fuzzer and a fairly simple specification which basically says, "The contract must have some balance," then uh you can actually get a fuzzer to spit out a counter example at in those blocks, which basically says, "Okay, if we had a power user, so we collected bean under one particular user and then called emergency commit on this proposal, then bad things happen."

So, the important thing here is if an exploit's in progress, we can use a fuzzer, a live state fuzzer, in order to make use of already executed steps in order to basically complete the attack and flag early before the attack is actually executed. But before I was talking about finding attacks that attackers don't know, so let's take a look at that. So here, let's just take a look at the Nomad hack. Uh the Nomad hack happened a couple of years ago. Basically, they updated their contract. After that update, 41 days later, someone discovered that there was a bug in their configuration, which allowed uh messages to be processed that weren't actually committed to. And so, within those 41 days though, you have ample

opportunity to go and find that attack ahead [laughter] of time. And know, even though uh audits did flag that they should make sure that things were configured correctly by adding extra requires uh you know, auditing the post configuration or the post-deployment configuration is not the job that most auditors have. So, what do we do in this case? Well, again, we can have a very simple specification that states, "Never process a message that hasn't been proven." And then, Orca can basically, you know, randomly interact, and Orca finds this particular counter example.

Uh the Nomad hack is fairly simple. The complex parts is the bytes that I left out because you have to create very specific byte values. And so, Orca can create a transaction that uh uses a message not proven, not processed, and therefore uh flags an exploit. So, the important thing here is before an attack is even known because if we did this anytime within those 41 days, we can start finding vulnerabilities rather than waiting for them to come to us.

So, let's see where this fits in uh the current security workflow. So, right now, we have the pre-deployment phase where we use tools, perform security reviews. Then, in the post-deployment phase, we perform monitoring. So, I already said that it would go basically right under monitoring. But the thing that we don't have at this point is you guys might have flagged [snorts] that we require specifications. That's something that, you know, you don't need if you're provide performing monitoring. So, where do these specifications come from?

Well, there are three particular places where you can get these specs from. So, one, they can be heuristics. And so, that means that you can just create some sort of heuristic that says, "In a single transaction, the value of my contract will never fall by 50%." Uh this is a heuristic because it could be triggered by uh basically Orca finding a whale and transferring all of those whale's funds out. But uh it can also flag quite a few attacks. Uh the other thing that we can do is we can have a generic specification which requires a bit more information from the user. So, this might be something like uh the vault's balance should uh exceed the total supply of the vault.

So, that's pretty obvious. That's something that we would want to have, but it requires information from the user about how to get that information. So, it is uh generic in that it would hold in a number of different contracts, but it requires the user to specify some additional information. And then finally, specs can be project specific. These can come from a number of different places, including things like security reviews before deployment and stuff like that. And so, uh these specifications can ensure that any logical errors could also be flagged by using a technique like this.

And so, basically, if we did employ live state fuzzing, our workflow would go from the top to the bottom, where we would have live state fuzzing, and you could take security properties from pre-deployment and make use of them post-deployment in order to try to find bugs before they actually occur. So, uh that's a bit about something that we've been working on. Uh we're currently uh working on It's currently in beta. We were, you know, performing some further work on it. And so, if anyone's interested in learning more or wants to try this out, happy to talk and uh see what we can do.

Thanks. >> [applause] >> So, thank you. Um anyone have a question? Yes? Yes, thank you. Um how would you filter out the contracts you're uh fuzzing? Uh because if even if a contract was deployed 41 days earlier to the hack, how would you know that it would target the uh contract of your client? [sighs and gasps] Yeah, uh that's a good question. So, this obviously wouldn't be able to solve everything. Uh and in some cases, like let's say, you're basically asking about

something like the Beanstalk hack, but you don't know about that malicious proposal yet. So, uh yeah, that is something that can occur, and so this wouldn't find everything. That's why I'm not saying this is a replacement for monitoring, but rather a way of identifying uh a fairly large class of vulnerabilities without requiring uh Yes, so you're not continuously fuzzing each deployed contracts on Ethereum mainnet. >> [laughter] >> No. So, basically, the way that we've been doing it so far is for a particular protocol that someone wanted to monitor, you can essentially give us a list of addresses and ABIs.

Uh you start by fuzzing those. Uh if we wanted to, we can also mine other addresses that are interacting with uh the particular contract. But in the case that you mentioned where um you know, a new contract was popped up, uh that wouldn't work out. Uh I guess the thing that you could do is uh you could always try just executing a contract randomly that, you know, uh pops onto mainnet because once it pops onto mainnet, we can, you know, decompile it, try to find the uh functions and then just see what happens when you execute them.

Uh that's not something that we're talking about doing because it impacts scalability a little bit. Um but yeah. Thanks. Another question? No? So, thanks all. Thank you. >> [applause] [music] [music] [music]