Workshop: Security Agents, Not Alerts:A New Metalanguage for Live Threat Detection in DeFi

Transcript

I'm gonna start. Yeah. Hello, I am Neville Gre. I'm a technical co-founder at DOB and today I'll be will be presenting our monitoring solution and I'll be teaching you how to use it and how to write real time monitoring routines uh for your projects. Um they're saying that the sound doesn't work now. It works. All right, cool. All right, a little bit about DOB before I start. Um, so we're a security uh company. We do security audits, but we also develop uh tools, especially static analysis tools and a and a decompiler.

Um, we started off being mostly whitehat hackers in the very beginning and we were responsible for quite a few uh vulnerabilities that we found in the wild and large protocols. Um at this point we are mostly associated as security auditors but we do a lot of tooling and a lot of other services as well. We're around 23 people mostly uh based in the Mediterranean region. We also have a product integrates a number of applications app.dub.com dub.com and the monitoring um suite and the monitoring ecosystem that I will be showing you and I'll be teaching you how to use and how to writing queries for is at app.dub.com. Okay. So um at this

point we've made monitoring free for a limited time period to increase our adoption. So you can try out our tutorial as I go along. Um and most of the stuff that I'll be showing you works for the free tier. Um and yeah and most of our most of the people use app.dub.com to for our decompiler. At this point we have like 10,000 users and growing. So when we set out to build monitoring, we talked to a number of projects. Um and we we got a little bit confused at the beginning because every project had their own different requirements and at then eventually we understood why they would have different requirements. But

I'll just give you a few examples of of different requirements. Let's say you are working on a on a decentralized exchange and um each decentralized exchange has its own API and in addition to that the centralized exchange might be interested in something like say for instance uh if there's a large trade on the centralized exchange they want to be notified okay on the other hand if you're working with an L2 they might be interested more about for instance uh statistics about gas or statistics about um the large withdrawals on their canonical bridge. So every single project has their own uh requirements and this means that you either do all of

these things one by one and you maintain this for every single version of the protocol or on the other hand you take a step back and try to understand fundamentally what all of these projects are trying to do. And when it comes to observational monitoring, which is what I'll be showing you guys, um really we have a few things that we need to do. We need to react to new transactions with very very little delay.

But at the same time, we also need to derive facts that are true from the beginning of time. such as for instance sometimes we we want to talk about things like all unis swap uh pools or all past proposers that have ever use governance bravo style DAO contracts for instance and we need to maintain these facts current uh in real time as well okay so it's a very difficult problem to solve obviously we need to be EVM unsolidity native and easy to modify and tailor each particular monitoring agent to this specific protocol.

And so uh we looked to design a solution and what eventually turned out to be uh DDQL. So DWQL is both a query language and it's also an execution environment and DWQL uh connects three different things. So first of all it is a variant of classic SQL. Under the hood we use Postgress and the reason why we chose SQL is that you have a lot of algorithms that um the query planner will choose for you. Um so you don't need to write a lot of code to to be doing like very complex things.

Okay you do need fast algorithms if you're going to reason about things that have happened since the beginning of time. Um incremental computation is another thing that we added to DOPQL. Okay. And materialization. And then the other thing is that DQL has um is very opinionated when it comes to the EVM and specifically solidity and its conventions. So it can model stack frames, EVM stack frames pretty well. And I will show you how how this is done. So today we'll learn how to write a DWQL query. Um we'll see how the EVM execution uh affects your query and how your query will uh issue alerts in real

time. But in order to do so I'll need to um you know teach you or like remind you a few things. So we'll need to first understand some classic SQL. We need to understand um the EVM a little bit and how um stack frames are created and the relationship between one stack frame and its parent for instance and also some things about um DWQL such as these things that we call gadgets which turn DWQL into a highly opinionated language for monitoring uh EVM uh execution.

Okay, so just a little bit of a reminder. Um, I know that all of you probably know Solidity, but maybe SQL is something that we don't use as much in the blockchain world. So when you have a SQL query, um, there's the following few clauses. So you've got from, this is the first thing that gets executed from. So from A join B means that you take two tables, one table on the left, one table on the right and you do a cross productduct of these two tables. This is sometimes called a cross join. Um if you add a condition uh that becomes an inner join and there are algorithms that um

with and as long as you have indexes that will make the join pretty fast. This is a wear clause. It's basically a filter. It filters your result set. The group by clause um puts your result set into different buckets depending on what you're grouping on. Having is like a wear clause, but it only works on the buckets and the aggregates that work on the buckets. So it's like a work clause, but it works after the group by. Uh then you have select which projects from one um kind of result. So from specified number of columns to another

specified number of columns as well. And you can do operations on these columns. And most of the time you are dropping columns. Okay. And order by and limit sort and truncate respectively. Okay. And in terms of like the execution sequence um in a nutshell like you the way you think about a query you have to first look at the from clause and the from clause you know pulls um rows from a table.

The join takes rows from the other table and adds the columns from the other table as well. Then you've got the where filters okay and if you notice these things form a pipeline and that's exactly the way the uh you know modern uh SQL implementation will will execute this. So sometimes this pipeline is called the volcano model. It's basically an iterator model um and so on and so forth. But one crucial thing about this pipeline is that it's a pullbased process and uh the data that you are getting out of this query is finite and

once you exhaust the iterator the query terminates and that's it. You get your result. However, that is not the execution model that we want for a monitoring agent. We want our monitoring agent to be run 24/7 and it's we want to have the impression that our monitoring agent is running on an infinite stream of data. So that's where we kind of like t tune the execution semantics and we added some things um so that we give the impression that these monitoring agents are running uh continuously.

Okay. case of the monitoring agents are declared in DubQL which is a variant of PostgresQL and you need to follow certain rules in order to make these effective. So first of all the queries in DWQL they are triggered by new blockchain data which means that as new blockchain data comes in your query is triggered on this new blockchain data and you get results [snorts] and your query then needs to be written in a way that is monotonic. Meaning that the more blockchain data arrives the more alerts will happen. Okay, not less otherwise you interfere with the execution

semantics um that we designed for for DWQL. Now, uh blockchain data in our schema is represented as time series tables. So, here's an example of a uh blockchain table that we have. This is called transaction detail. And we have for instance three time series columns over here. We've got the block number, the index within the block number, and the VM step, which I'll show you what VM step is in the in the next slide. And then there's the rest of the data associated in this case with a particular AVM stack frame. So, call data, return data, and various other things.

Okay, so just a little bit of a reminder here of um you know how EVM frames relate to each other. So if you if you use Forge for instance, if you trace transactions, this this will be a little bit uh familiar to you or if you use for instance our transaction tracer. But basically each frame will have the caller and colleague. We call these message sender and uh and this call data return data and then various other things. Um so each frame really corresponds to a particular call delegate call or create context. So when you create a smart contract for instance that is also a call. So if you look at

this for instance, we have two um two counters. We have the VM step and we have a program counter. And the VM step is a global counter that we increment uh when the execution of a transaction starts. So it starts at zero and with every single instruction that um that we execute, we increment this counter. And then when we get to the beginning of a of a new frame, so after a call, we look at the VM step and that uniquely identifies that particular EVM frame from the rest of the EVM frames. And we can also

represent these VM steps in a tree for instance as well. And there there is a way for instance to uh join your frame for instance with your parent frame and things like that. Okay. Another interesting thing about this is that uh the program counter which is a counter which is incremented and like you know changes when whenever there's a jump the program counter in a frame we can also record that and that becomes the call site of the current frame as well. Another thing you should note is that arguments um are AI encoded when you are calling a function. Um and

they are passed through call data and the return values are passed through a buffer called return data and again everything is AI encoded. Okay. So now that we know this, so we represent all of that information. We represent uh frames, logs, uh and everything else as new blockchain datas, uh new new blockchain um time series tables. And all of these tables are actually indexed in temporal order. And typically they're indexed by block number. And in cases that it's an actual transaction

transaction index and the VM step start reminder the VM step start is the VM step at the at the beginning of the particular EVM frame that we are operating in. So for instance if we take transaction detail this will contain definitely the VM step start because transaction detail contains all the EVM frames whereas outer transaction for instance contains one entry per every single transaction. Logs for instance contains all the events and within that event you can also know which frame you are in. Okay.

So you can then do things such as join um your event for instance with transaction detail to also you know reason about the call data that was passed uh when that event was was ex was uh issued. An interesting table also is is token ledger. This is also a time series table. But in token ledger, this represents all sorts of transfers that happen, including token transfers and like eat transfers, even things such as self-destruct.

But it records two entries for every single transfer in a double um in a double entry bookkeeping style. And this gets quite handy if you want to reason about for instance net transfers uh for a particular period of time. Okay. So now we're going to try our first monitoring agent. So you can you can test this along. So you can go to app.dub.com and you can go to monitoring and What I'll do is I'll create a monitoring agent for

which triggers every time there's a big transfer that happens on chain. In this case, I'm going to do it on the Ethereum network. And um yeah, whenever there's a big transfer, I want to receive an alert. And the way I'll do it, typically when you write SQL, you first start with the uh with the from clause. But since I already know sort of what I I'm going to I'm going to write here, I'll just write the whole thing like so. Um so I need I need the transaction hash.

Okay. And that will be the um what's what the monitoring agent is going to return. Now we don't store the transaction hash in our time series tables because the transaction hash is not easily indexed. It's basically random data and but all we really need is the block number and the transaction index because those like uniquely identify um the transaction. So we need to calculate it using this function transaction hash. So block number Okay.

>> Yeah, let me do that. Can you see it better now? Okay. So, so now I'm going to use a special gadget. Okay. I'll explain a little bit what the gadget is, but um when you define when you use a gadget, you need to put it between these double curler braces. Okay, so I'm using the token ledger uh gadget and I'm going to add a condition and the condition is basically that the transfer is bigger than uh $10 million.

I'm going to start with $1 million since uh so that we can see the this in real time. Okay. And since the prices of cryptocurrencies change all the time, we need a way to uh turn the token amount into a US dollar price for this to be relevant. And there's a function which does this to USD value which takes the amount of value that has been transferred and if you know the token address, it will do the conversion for you.

Okay. So, I'm going to save the query and I'm going to run it just to test it. So, when you run it, it's basically just doing uh back testing. Um, so I'm going to back test it on the last hour of data and hopefully it returns something. So, we've got unable to parse duration. Okay. Yeah, try this. Okay, this one is an easy one. Uh, basically these gadgets are represented as sub queries. So, they need an alias.

Okay. And here we need a condition uh greater than $1 million. Okay. So, these are some transactions that there are transfers greater than $1 million. Let's see one of them. Um, yeah. So, we can see there's there's been a big transfer of blur token in this transaction. Um, but really we don't want to run this and and that's it. We want to turn this into a monitoring agent. And the way we do that is we assign a specific schedule for it.

So I went to the query settings over here and I'm going to set this to be and now I need to shrink the font because otherwise I will not be able to see. So I set this as an alert. Now when I set this to be an alert, this creates a materializ this sets a materialization strategy. The reason why it does so is because I don't want to receive the same alert more than once. Okay, but in order to do that, I need to define a unique key. And it took the transaction hash to be the unique key.

And I'm going to set a frequency. And I'm going to set 30 seconds to be a frequency. And this is going to run now every 30 seconds on the new blockchain data that comes in. Okay, now we will get back to this uh and and see the logs and see whether it it has successfully run. But let's do it now because it might have already run. I'll refresh again. Okay, so this has run 5 seconds ago. we

can see, you know, some details about its execution and we can see whether it has actually found any any transfers that are this big. And it did indeed return some alerts. So obviously like $1 million is a very low um low amount. So now I set it to 10 and hopefully it would return less alerts. But let's get back a little bit over here and I'll explain a little bit token ledger a little bit more. So token ledger we have two entries per transfer. We have one entry from

beef to coffee and one entry from coffee to beef. And if you notice the value deltas they are in different directions. And this is useful um again to to compute things such as net transfers. Now let's see this in more detail how the monitoring agents are executed. So to turn your query into an agent, you have to add a schedule and also materialization strategy and also an alert sync. So at each cycle of the schedule, new blocks arrive, okay, and there is going to be abs like really no delay between new blocks

arriving to the node and like being pushed to the database because like the the node and the database are essentially on the same machine. Okay. Then if your agent has other dependent agents, those need to be executed first and then each gadget uh that you have in your query that needs to be expanded at compiled query optimizations and then you run the query and then any resulting rows those are effectively alerts. So here's an example. In run number one, you know, two new blocks came in, query executed and the query only executed on the data of block one and two in this case. Then

there was another run with block three. And the system at every stage ensures that the your query is not going to run on the same data twice. Okay. So, so the query that we saw was not a particularly good one because uh basically on every transfer that is greater than a million is going to trigger. Uh in reality, every swap is going to contain you know um one transfer in one direction and one transfer in another direction of different tokens. So it's more interesting to reason about net transfers and this is a query that does

exactly that. It reasons about net transfers in a single transaction for a particular protocol. And the way it works is still we have token ledger but we join the token ledger to a table called protocol contract which defines a lot of protocols in our system and you can include more and more protocols if you want to. Okay. And it is joined by the address that is in the token ledger. We group by the transaction and we also group by the protocol because we're reasoning about one protocol at a time and then we aggregate

those results over here. We use the same function here to use the value but we aggregate and then we check uh whether the net amount reigned is greater than a particular value in that transaction for that particular protocol. Okay. Now here's something else that you can do using uh the DWQL gadgets. So this is a gadget. This is the logs gadget. Okay. and it corresponds to the logs table.

But in the logs gadget, you can also provide a condition here is uh the address that I want to filter by and also the API of the particular event that I want to decode and this decoding is done in real time. So we never store the decoded information in our database because that's like three times larger than the original information. All the information is like stored in binary but it is decoded in real time according to the API that you provide. Okay. In this particular case with events under the hood um events are stored as a

series of columns. You've got topic zero, which is the um the hash of the AI of the event. And then you've got the other indexed columns are stored between topic one and topic four. And then you've got a a data field in events which stores the unindexed uh the unindexed column uh fields of the of the event. Okay. In other tables like transaction data, you've got call data and return data. They can also like decode errors and things like that.

Okay. So now let's see another example of another monitoring agent. This time we're going to uh look at DAO proposals and governance. So over here we're going to be writing an agent which looks at recent DAO proposals and will trigger a new alert whenever there's a new proposer that has never proposed something on that DAO. Okay. And the way we do it is like so I want to alert whenever I get a new proposal. So that means um and a proposer is basically someone that calls

into my DAO contract uh with propose. So it's basically the message sender. So that's from a and then I also need the transaction hash which basically means I need the block number and the index. And in this case I'm going to use the transaction detail gadget. And actually in this case I need to use the arbitum. Okay. Arbitum transaction detail. And I need to know the u address of of the DAO which is this.

And notice that when I press dot, the system will know the AI of that particular contract. So, propose. So, I'm going to uh hook it up to the propose um function. And that's that's it. Okay. So, now I'm going to back test it. And I'm going to back test it. Um I know that this month there weren't a lot of proposals. So I'm going to back test it on like September. See whether this returns anything. So this is now

running on a month worth of data. And okay, let's run it. on two months of data. Okay, so this has returned a result and I can actually if I wanted to I can look at this transaction hash and see the that particular proposal. Okay, so I'm going to do the same thing as before. Um you know mainly go to settings, set it as an alert. Um, you can put an alert

message. I can put a unique key. In this case, the system has identified the proposer as the unique key and I can set a frequency and in this case it's not urgent. I'm going to set it to be 10 minutes. Okay. So, what else can I do? What else can I do? [snorts] So one thing that I don't need to deal with is compiling different versions of the query depending on the mode that I want to do. So for instance, this is the compiled version of the query that I just wrote. This is the query that you

write. This is the compiled version. and the compiled version. It takes care of inserting code that real time decodes the AI um that I'm currently looking at. And it also inserts code um that makes sure that I'm only querying that interval that I need to query in that particular execution context. Okay, these are the settings. But actually I want to go beyond that.

Uh I don't just care about um new proposers uh from arbitum proposers but I don't want to get alerted if an existing proposer from like a year ago does another proposal. Okay. In that case I don't want to look at it for instance. So what I can do is I can go back into the settings over here and I can select backfilling but to backfill immediately what I'll do is I'll do materialize immediately I'll click on save and now the system is going to run this query from the beginning of time and

find me all arbitrum prop arbitum dowo proposers and after it does that which takes around 30 seconds. I can then set it up to only alert me on um new proposers after that. So by doing so, not only is the system giving me the impression that the query is running continuously on continuous data, but it's also giving me the impression that this has run from the beginning of time. Okay, so this has run from the beginning of time and it only started issuing alerts to me uh from this point on.

I'm just going to check whether everything is good over here. Okay, so this has actually now been materialized. Now I'm gonna set it back uh to the normal schedule every 10 minutes. Okay. So the idea behind a monitoring agent like this is that maybe as a security researcher, I want to check out new proposals, but only from pe from addresses that have never sent a proposal before. And this is exactly what this query gives you. And it's

literally two lines of code. Okay, let's see in detail what the system will do in in this particular case. So let's say this is the Ethereum blockchain which right now is at 24 million blocks. In this case, when I set the back filling strategy, I set it to backfill. In the first execution, it's going to run from the beginning of time. And let's say it's going to return these three results, A, B, and A. Now, since proposer is a unique key, it's only going to take the first two results.

Then in the second run the strategy changes. Now it's transitions into incremental computation. Now in the second run if it it finds for instance A and C. But since A is already here during the back filling it's not going to alert me on A, it's only going to alert me on C. And then let's say in the third run uh it finds D and C but it has already alerted me on C since it's already there. It's only going to alert me now on D. So these are all the alerts actually these two are all the alerts I'm going to receive because everything which you receive during backfilling is

not is not u alerted. One more thing that I can show you is uh dependencies. So obviously in programming languages we we like to have abstraction. Okay. And we allow abstraction also in in DubQL. So DWQL queries can depend on each other and the system needs to make sure that when queries now depend on each other that the dependent query is executed first. Okay. So the dependencies are executed first. Sorry.

Okay. So in this case, let's say I want a query which uh finds me new unis swap proposers. So unis swap dow proposals but only but I only want to be alerted if that proposer has never issued a proposal on any other DAO that follows the same API. In this case the governance bravo. Okay. And the way I'll do it is by first creating governance bravo proposer in this query and then make sure that the proposer is not in there.

So let me show you how that's done. I'll go I'll go over here and I'll actually copy and paste the arbitrum code and just change it to Ethereum and to unis swap. Uh I'll copy the uh unis swap dowo address from here.

I'll put this here and I'll change this to Ethereum since I know that the DAO is on Ethereum. And then I'll add another condition. I will say where proposer not in. Oops. And basically here you just do uh select proposer And the way you do ref is you need to

get the query ID and you get the query ID from the URL over here. So I had already created a query uh with the governance bravo proposers which is this one and I can get the query ID from here. But just to give you an idea here, what I did is I just uh let me ex enlarge my screen a little bit. I basically just union all proposals from unis swap and arbitum.

So copy this and I'll go to this over here and that's it. And then I'll do the same thing where I where I set the settings and the materialization and the alerting settings as as in the previous examples. So now let me show you something something else. So one of the things that DDO is known for is the decompiler and also the uh vulnerability scanning tools.

And it turns out that all of that information, all the decompiled code and all the static analysis results, we make them available also in our database. So when you get access to monitoring, you can join to these things. So for instance in this case let's say I wanted to scan every single contract that the governance proposal is going to interact with. I can simply do it using database operations. So basically using join and things like that.

So let's see how this can be done in monitoring again. So over here I can look at the schema of my tables and for the particular chain that I'm interested in I can see for instance here there's a list and there's a vulnerability table and the vulnerability table has is indexed by a unique identifier of the smart contract and it contains information about the vulnerability. such as the vulnerability type that it has.

Uh and so if I wanted to join the governance dow proposals to that, what I need to do is I need to decode the proposal itself. And luckily the targets of the proposal are actually in an array as part of transaction detail. and I can then join that to the vulnerability table. So I'm going to do exactly that. Um it's it's it's a slightly longer query, but I think we have time to do it. So I'll just copy this and I'll go back to files over here.

I actually already had prepared the query in advance. Let's go through it step by step. So the first thing is I need to somehow from the DAO proposal I need to take the AI. we need to take the targets which is the first argument and this is an array and if it's an array since there's no way to represent an array as a value in in SQL uh we represent it as JSON B and to unravel that array we use a feature of SQL

called a lateral join okay and with lateral join you can join um on the right hand side to a value that you have taken from the left hand side in this case an array. So this lateral join will take every single [clears throat] uh EVM stack frame that matches this AI. It will then decode the targets array and for every single target it will join to that and it will create an entry in the result set for every single target.

Now to turn that target from JSON B to uh byte a which is what we use we need to decode it. We also keep here the transaction hash and then we put this into a CTE. So that's common table expression. So in this case I put it into a common table expression as an abstraction mechanism. Okay. So that you know I can reason about this thing separately. But this also works as an optimization block.

Meaning that the query optimizer instead of trying to optimize the query globally, it will now try to just optimize this separately. Okay. And by simplifying this and by making this optimization block, the optimizer will work obviously better because if you have like six or seven joins, uh the optimizer might not find the best order of join operations. But in this case, it's going to work pretty well because I only have one join.

Now that I have the the result, I now need to join with the other tables in our database. So the first thing is I need to join to contracts. And if I show you contracts over here, contracts contains every single smart contract that has been deployed from Brock zero up till this particular point. It contains information such as for instance the block number in which it was created but al and the transaction index but also other things. For instance, in contracts, we also put a unique identifier of that smart contract. And since for you can deploy

many smart contracts with the same bite code, that is a as a way by which we can make this a little bit tractable, meaning that we can make the analysis of all smart contracts a little bit more tractable. Okay. So we join that to get the unique identifier which is the MD5 byte code and using that unique identifier then we join with vulnerability and vulnerability will give us one entry per vulnerability type that there is in that contract and if there is no vulnerability in that contract it will not give us an entry.

So now um if the DAO proposal does not contain does not interact with any smart contracts. We will not get any entries. If it contains you know one entry but that entry the uh the the smart contract that interacts with is not vulnerable according to our database. You will not get entries. Um but if it does you will get one alert. Now how do we get one alert per transaction? Again, in this case, we can go to our settings and we can set

the unique key to be something like target or we can refresh this and we can set for instance the unique key to be the transaction hash. So this means we get at most one vulnerability per one alert per transaction hash. Now if you get one alert then you will investigate further. If you want multiple alerts per, you know, depending on how many vulnerabilities you find, then you add more things to the key, you can add the vulnerability type if you wanted and the target. Okay, so in this case, if you have like four vulnerabilities in one contract,

then you will get four alerts, which is kind of like not ideal. Okay. In this case, since it's a more heavy query, I will set the frequency to 10 minutes, which is kind of like enough because Dows operate in human time. And um the system actually if you set it up with a higher frequency than 10 5 minutes, it will give you more guarantees about the correctness of the data that it has. Whereas if it's like smaller or like 30 seconds or less you know it will try to you know keep up but

it might if your query is complex and depends on many other queries it might not have executed all the dependencies by the by the point that the 30 seconds elapses. Okay. So the 30 seconds are there to make um you know smaller alerts but be faster. Okay. [snorts] So this is it. So uh this is doing you know vulnerability scanning. It is like looking at every single element in the in the targets. You know if we wanted to reason about the call datas for instance we could also unravel call datas.

Um and you know if you have for instance an external tool and you also want to analyze the call datas for instance you you can simulate them then you would you could perhaps unravel the call datas and then send those as part of a web hook. And there is a way to do it from here settings uh you can you know notify via web hook and do it like that. So, so in conclusion, um, so DubQL is this execution and also, um, query language

that allows you to write monitoring agents and it is integrated into our ecosystem and you can also make use of static analysis data and it takes care of executing your monitoring agents. So so that you don't need to write very complex ones in um in ways that [snorts] require a lot of code and algorithms to be implemented manually. So that concludes my presentation and tutorial. I hope that you learned how our system works, how to write um queries using our system, but

also more generally how you can use um these analytical techniques to you know like query blockchain data in general. [applause]