The Butterfly Effect: How Simple Oversights Turn Into Smart Contract Nightmares

Transcript

okay guys uh thank you from for having me here my name is Emilio frangela I work in Ave I work as a community contributor I worked before for other companies as a smart contract lead engineer and I'm currently contributing to the community I co-founded with other core other developers bgd Labs which focuses on contributing to the our ecosystem through core smart contract developer developing so what I want to talk about today is a quite interesting topic that for me it has been uh is something that is very

humbling when we write smart contract code and uh it's about like how very very simple mistakes can actually have uh even devastating consequence consequences when we write smart contract software in general uh what we what we call Mission critical software right so let's start why smart contracts are mission critical software well here we can see like this Twitter this tweet by token terminal actually shows how in the last two years more or less more than four billions worth of assets haven't been installing

through vulnerabilities abuse and smart contract hacks right so like four billions is uh actually a lot of money right and this is a very very important topic for our space and it's something that of course we need to solve in the proper way because users impacted by hacks are users that in general many of them like uh they put they use this Technologies these smart contracts daily many of them with important parts of their net worth and losing an important part of the network we can have disrupting impact on your life right so all of these all that we

do in defy has impact on people's lives so this is very very important for to scale our industry and also to provide reliable uh software Reliable decentralized Software right so uh why are smart contracts uh Mission critical software well one of the reasons I just explained the fact that they impact people lives and they can disrupt people lives uh one another another aspect is that like in inherent to the technology itself and is because mostly the nature of smart contracts the immutability or smart

contracts or the fact that they are governance controlled which means that it's really really hard to fix bugs right so if a smart contract is immutable you are not going to be able to fix any bug and if they are governance controlled there is usually a delay to any smart contract upgrade that is usually uh enough for attackers to still attack the system even if we know where the bug is and what the potential fixes are and not only that but the nature of the environment we work in like the nature of the ethereum blockchain and the nature of the evm and the actors that play in defy it's very adversial

adversarial right so all of these kind of exacerbates the problem of uh making uh smart contracts as safe as possible so how do we make them as safe as possible well I would say that uh the key here is the processes that we follow when we were when we write smart contracts right and we when we develop this kind of Technologies so uh let's make a parallel with a real world example right if someone follows me on Twitter they probably know that I am huge space nerd a lot of space I love astrophysics and like one example that for me is pretty pretty explanatory of what I mean

with uh Mission critical software is this particular case that happened to NASA in 1999 so essentially NASA lost a 125 Million worth probe an Orbiter because part of the team was using the magic system to measure distances while part or other part of the team was using the imperial system right so essentially this was a human oversight that made the proper ultimately useless it was impossible to drive it so the whole developing process and the whole budget allocated for the process was ultimately lost what I find particularly

interesting are like the quotes of Edward Whaler that is one of the engineers of uh NASA like people sometimes make mistakes like they make errors this is unavoidable unfortunately it's part of the human nature so the real problem here is not the error itself is the inability or the failure to capture this kind of Errors right so uh essentially a failure in our processes to detect the error which is I think what is really important here so we have just seen um real life case of um like you know failure in processes to

detect uh even trivial sometimes error now when I say trivial uh people could argue that they are actually not trivial that are really bad mistakes I don't disagree uh when I say trivial I I mean that they can be human oversights that are unfortunately for what the new manager is they are pretty easy to manifest but they are really hard to catch so in hindsight they might look like really uh bad mistakes but in reality uh they are oversights that sometimes they are quite easy to to sleep in in your processes right so I wanted to choose a parallel with what

happens with um you know smart contract development in D5 usually I took two examples there are many honestly and unfortunately like many occasions in this space where trivial errors actually cause like the loss of hundreds of millions of dollars I choose two that in my opinion are particularly meaningful one is specifically uh um for Ave so it was a failure of our process of my process I was working on the project so it was um a lesson for us on how to improve our project our processes another one uh for the nature of it and the way it actually

um ended up being an uh production code is the um probably some of you guys will remember is the an accounting error that affected one of compound proposals which cause like the system to lose about around 80 million worth of comp tokens so let's dive quickly into them uh like this is like the compound liquidity mining accounting error unfortunately this is a very very small mistake the mistake here is in the condition above the first line the uh a greater symbol should have been greater or equal right so uh unfortunately because of

this very very small oversight there were some markets on the compound protocol in which liquidity mining was not actually active which made essentially the accounting index for the liquidity mining being equal to compound index which means this whole block of code was skipped and when the proposal was executed and the supplier index was and not initialized it properly which means that people started getting way more comp rewards that they should have got and this caused essentially the loss of 80 million worth welcome tokens if I remember correctly but that is not the topic the topic is like how small the error actually was how like um narrow was the particular case of the

edge case or that the the code didn't handle because of this mistake and despite that like the consequences were kind of devastating unfortunately and this is also interesting because this was added through a governance proposal so a governance proposal was executed after uh the protocol was actually live introduced the mistakes a mistake which I will essentially reconnect after when I will I will talk about processes and uh how to handle like upgrade upgrades of code uh production um actual production code like in this case the other mistake is actually related to other specifically it was it's

interesting because it was this uh vulnerability was live for one year and a half before it was discovered by one of um the trail of bits engineers and security Engineers so thankfully nobody abused it but this would not have caused a lot of funds but would have of course like um disruptive denial of service essentially what could have happened is that the one of the key smart contracts of the other protocol could have been by manipulating the way the the contracts were deployed and initialized or non-initialized in this particular case um an attacker could force one of the

main uh protocol contracts to be uh to self-destruct which essentially would have uh disrupted the system for four days which is the uh time delay of the governance which requires three days of voting plus one day of time lock so in this particular case this was our failure of um a reviewing processes of deployment of uh production ready code so there was an oversight when we deployed this code in production which essentially could have uh caused a massive disruption of the system even though it was there was technically no loss of funds involved but

uh the system could have essentially been disrupted for four days uh and then some uh like the honestly the um the way that the vulnerability was unveiled was similar to the party hack I don't know if some of you guys uh will remember where essentially a a key a smart contract of the party wallet was forced to be uh destroyed through the same kind of attack so uh here yeah I put a bit of the code that shows how the vulnerability actually worked essentially one of the smart contracts of the logic smart contracts was not properly initialized so an attacker could have deployed the malicious provider contract which uh

here would have been called through delegate call and through the legit through delegate call you can essentially force uh self-destruction of the calling contract which would have ultimately disrupt the system uh so like one of the things that interested me when I started uh working in the smart content development was okay how does NASA handle this kind of um uh um you know Mission critical software and um the interesting thing is that the process they Define they start not from like I would when I started first reading I would have guessed they would

start from the testing but they actually start much earlier they start in when they find when starting by defining coding standards and best practices so you start catching bugs not when you test but when now you actually write the code so it's the way you write the code that helps you catching the bugs so here I put like this is the table of com of uh contents of one of the main pages of the engineering Handbook of NASA uh some of these steps are honestly can be directly translated to Smart content development some of those like uh I don't know results over subscription thread stuff safety some of those that

cannot be applied because of the nature of the technology but some of these can be directly translated to Smart contract development I put I put the link of the handbook here um I put some like Keys key points here that can be actually translated for in a smart contract development so identify and Define clear software development practices uh fault detection which is super important like detect when your smart contracts are and when your code is supposed to fail and properly handling these particular cases in-flight software modification which is kind of Records previous bug of compound right the way you handle code that is

actually in production already and how you should handle um you know upgrades for improvements of the lodging of these smart contracts and safe transitions is also super important most of the smart contracts can be identified as state machines um like and these trade machines usually have some pre and post conditions evaluating pre-imposed condition usually helps catching uh like bad designs in your uh in your process flow and in your in your model that you're trying to to develop and also uh the testing which is super important here we we see a kind of

Standard Process flow you need test integration testing regression testing and so on and so forth some of the key highlights a careful definition of the unit test starting so how you define this the software units and how you test them integration by testing like the software units working together um formal testing using automatic tools and formally defined models and NASA requires 100 cover coverage on critical paths so this is quite interesting actually because like this is a very debated Point like some say that 100 test coverage is usually not useful uh it's interesting because like another defines

these four points here um like the most of the problem they say 100 as coverage is usually not so useful is because like some cold paths are really really hard to test and some of them might look trivial but if you look at the way NASA classified uh classifies their coverage in none of them like they say you can enure because the code path is Trivial and it's hard to test and absolutely not like if you are not hitting some lines of code in your coverage it means that either the code is a either is a dead path or the activated code or or there is a

requirement that is missing but like they absolutely enforce 100 as coverage even for trivial parts that are hard to test so uh how do I translate I will try to be as quick as possible how do I translate these two concept to Smart contract development Define internal code guidelines in other way we have an mbgd as well we have like an internal document that everyone should follows that describes how the code needs to be written this can be as opinionated as you want as long as the whole team agrees with it and it's super important because this kind of brings a makes the code more homogeneous easily

easily more easier to review and to to test afterwards um clearly defined your failure paths in your contracts and handling them explicitly so failing early and loudly uh plan for software modification according accordingly so validating the storage change clearly and Define invalidated State transitions as mentioned before and in testing as well uh I really push when possible for test drive it test driven development which is uh not so easy to apply all the time but is something that we strive for uh 100 call coverage

plan or post the deployment and defining invariance which is super super important when you need to planify your uh testing strategy and also uh through automated tool like the software approver or through tools like Foundry and so on and so forth um I think I am out of time right okay I'm okay so I was faster than I thought okay so uh here are the conclusions so as I mentioned before we should consider Smart Control development as Mission critical uh software unfortunately this didn't happen often in our industry

especially uh during the last couple of years when we had defy summer and everyone was deploying code left and right and where we most of the teams like uh to be as fast as possible to get as much as fast as fast as possible in the market like they were essentially uh rushing the code to production or maybe forking smart contracts without having deep understanding or what these smart contracts were supposed to do which has ultimately led to unpleasant smart contract hacks um preventing bugs is not only testing but it starts even before you start writing code it starts it's big it begins with

proper planning a proper definition or viewer specification proper definition of your invariance and uh which ultimately help as well when you're gonna planify your testing strategies and you're essentially your approach to find education find bugs uh Team culture is fundamental so um everyone needs to be to stay focused on this particular topic so this is something that is very very important to promote like Security Forces very very important to promote within the team and

everyone should try to follow this particular approach of writing code uh so as I mentioned the plan for security and plan early start planning when even before you start writing the code uh to avoid unpleasant surprise later and then uh ultimately we need to learn uh to live with the fact that we make mistakes right that is unavoidable so the goal here is not is not avoiding making mistakes because that is unavoidable the goal here is to try and catch mistakes as early as possible in your development chain right so whenever you are able to catch bugs

earlier and they don't hit production you essentially average your goal uh okay I think I'm done I don't know yes if anyone asks questions like I would gladly take them thank you everyone [Applause]