Risk Management Approach to Security
DeFi Security Summit 2022 · 27 Aug 2022 · 22:29
storm0x, a security and core development contributor at Yearn, presents a risk management approach to security for a yield aggregator that was shipping several new strategies each week. He contrasts the heavy up-front process used for the immutable vaults with the faster cadence strategies need, and describes the operational side, including incidents and war rooms, documented procedures and on-chain risk scoring. At DeFi Security Summit 2022.
Speakers
Topics
From DeFi Security Summit 2022, hosted by DeFi Security Summit.
Transcript
all right hello everyone Thanks for anybody uh today I'll be talking about uh how your approaches security operational perspective uh I'm gonna ask for a lot of discussions that have happened today around the processors or or what I see the supermarketing and just um see how that that works for us in our special use case so a bit of info on myself but you know security and political development for smart contacts
so why does urine like a weak background so parent is basically uh yield uh aggregator which means uh you can think about it like on chain problems we're looking for people possibility into this contract and those on chain robots are looking for you somehow right um so there's magic happening in the back end I'll talk a bit of how that works um but that's basically it you've deposit money in in one of the vaults and that auto compounds and and gives you some x amount of u a bit of your uh uh stats that we have uh that I think are relevant for for the discussion
um I picked up this slide um and I noticed that uh the stats that we have since our B2 release which is our current on-chain uh generational vaults um since the first released version of that um generation we deployed 153 bulbs uh some of this are deprecated and not used and uh a lot of strategies um you can see the numbers 325 strategies so that means that at some point um we we manage uh in the in the highest uh time on tvl I think 6 billion and we were deploying around three four contracts a week
um how the how does that work and I want to show you how do we manage that we've seen other protocols say okay uh every time you're deploying something new or making a change that's immediate risk and a lot of stuff needs to happen around making sure that that doesn't have a fatal impact or or or or a critical impact within uh your your uh protocol so uh continuing with the background like the way the Vault works is uh you have a user facing a smart contract which is default it's it's very much like a near C20 token and you serve LP into it and in the background there are several strategies uh working and you
can think about them as this uh Roblox on on chain trying to look for yield doing different stuff uh interacting with different protocols and getting yield to how to compound whatever your Capital was that you deposit into the ball uh So within the volt score system um it's immutable uh we we did follow a bit of history of how B2 came about we did follow the same best practices that have been discussed here around uh very strong upfront design a clear spec testing uh even some type of staging live with with small amounts before we
actually uh got into audits three audits uh or four at this point and uh that product was very heavy uh waterfalling if you want to call it that like it's it needs every requirement heavy upfront investment heavy implementation and uh that's usually the case for most protocols uh or smart contracts right we that's the current life cycle uh in the industry at this point and the artist goes somewhere between the verification part um and we we sort of see like this seal of approval well the the contract has an audit it means that it's safe uh which
is clearly not the case uh by far um we need to to readjust things overall as a necklace system but for urine specifically you see the Cadence of strategies the way it works we just couldn't do it in a wonderful way for the strategies uh well the ball is very stable and immutable and has being done through this design and process the strategies need to we need something different we needed something different so uh why is that different because uh the yield opportunities and strategies come and go their FM role they're they're they you can have something working uh in two weeks and then after there's no yield in that
strategy so you can see it like a conveyor belt within the ball is very stable it's it's solid but the strategies uh are are changing a lot and um I talked about like in the highest of our tvl we were launching five six strategies a week um so that means a lot of changes a lot of potential risk and we we actually experience a lot of issues because of that like we we had several incidents uh we had never lost any money in the in the generation of YouTube uh for our users but uh it it was a big overhead of working out uh incidents there were War
rooms issues because it's just the the matter of the way we uh deployed the code really fast because the yield opportunities were were scarce or or if you didn't act on them quickly the yield was gone so um that's sort of like the problem that we needed to fix we realized uh this is we're we're we're uh putting out too many fires and uh there's we need to sort of switch around that or think a different way to do this so um how do we come about doing uh that so our uh the first thing you need to know about the strategy is that it's really sandboxed in in a sense that it's a you can see it like a small script that does
that's not to doing too many things it's just very narrow in scope and it's really uh just doing specific things with a specific protocol and uh so we we had some inspiration from other protocols like um and and other areas of site smart contracts that we've been discussed here like the primarily the the Aerospace industry and the health industry which have all this type of processes around risk which which we adapted here so what we came up with is okay we have our our operational risk which what we call it uh is one that you can sort of think of it as
there there's just several steps that need to happen before a strategy your contract goes to production and uh we can sort of put a number between all of these dimensions and uh for example as it hasn't has the Contour been audited probably if a strategy is coming out in two weeks uh it probably is is not going to be audited because the the audit queues are really long and you cannot get an auditor in short notice it just it's just the way the the system works at this point so okay so it goes this doesn't have an audit that means that it has a high risk score so we we score it it has code reviews okay that's something that we can control so
we can put as many code reviews as we can and we just sort of need to put a number on that amount of code reviews how thorough was it how many people saw it where the Specialists involved uh when I mean Specialists like strategies that are more um uh in that have more knowledge on that specific protocol that's interacting with that new strategy or or even the we sometimes invite the the depth protocols to to have a look um so that that impact stats for and and the lower the score on that Dimension the B is the less risk it's just a way to quantify how are we covering our bases like uh reducing the odds um
um of of something bad happening because like we can score each dimension of bit we do this if we do this and out of that you get a simple formula that that's uh it's likelihood of of what um of what that score would be so if you join all this stuff like all the dimensions audit reviews complexity team knowledge protocol safety those are very opinionated things that we came up with like uh how uh for example the protocol safety I'll I'll show more into that but it's really like an opinion on yearns um assessment of a steal of the given program we've just scored that so all of that together gives you a formula that's for like the odds that this contract is
going to have an issue okay so now you have the odds and if you're knowledgeable in gambling Theory and all that there's Equity or EV and you can sort of like out of that um put another access towards how much Capital uh can this stock contract hold and so if you put in the perspective of a strategy we came up with this um table that that sort of like it shows that the different um tiers of impact and we we say okay five is the big extreme impact is like uh urine cannot like uh potentially not even pay that if it happens and one is
something that is bad it's not something that we want but we can still uh continue to be uh operational with it if we have that impact if that impact materializes so you join the two together likelihood and impact and that's just give you like a heat map on the axis so you see in the in the right side we have the likelihood oh sorry the impact and on the horizontal axis we have um um the likely and so this gives you an overall map of your operational procedures and and this sort of now um switches the coin and flips it to now I'm working towards prevention because
I'm seeing okay this vertical is clear the rest Zone The Red Zone we don't want it to be that way so what do we need to do to mitigate it okay it doesn't have an audit and it's getting it's growing in tvl so we first of all we tap it to to to to to keep it there and the things that we control are trying to get more reviews get more alerts those are parts of the dimensions that we control or get more audits so that doesn't mean that the strategy or smart contract needs to wait for an audit to be able to have some money allocated um so and the way that this works is informs our audit external audit queue to say what is the priority that we need
to handle in the audits because we have like 50 contacts which are going to be first and we usually this this sort of like informs that that QA says okay this has the biggest impact has the best fpr or or or some other factors and and it's growing and can't sell more but it's limited at this point because uh the risk for is not what we want it so we do we move on to preventive actions to try to mitigate those and those are the dimensions that we sort of uh control in our app so if this is not new like we we've seen this um this is uh we're seeing this and several of our Industries uh um I I Echo the sentiment here that
we're looking for we're serving in an early stage of the tech but uh just uh think uh stopping and think about like you now uh buy a flight ticket and you never think twice about something bad happened um this is not the case for my contracts at this point right every time you're putting money in a smart contract uh the risk is really high um but that's not that the air industry didn't arrive to this point um just in one in one uh Swift uh jump right it went through Cycles so every time there's uh at any given point in time there's millions of flights happening and there's incidents but that
doesn't mean that there's an accident and a plane crashes there's just something happened that was inspected that went through the procedures uh even though there were defenses uh it materialized it and something needs to be done to improve that so to have more layers or more defenses and like I said this is not new I've seen other talks uh and change stuff that we we should bring all this procedures and and best practices in other industries that are mission critical to a Smart Contours because in the sense it sort of is right it's a I I I agree it's sort of like launching a satellite uh you don't you have very small flexibility and uh to to get it right
um so um that's that is whole uh hope that explains how we approach the strategies the strategies are are put in production with a certain score based on what are the operational uh mitigations that were put in that we can control and it doesn't mean that it has an audit or not all this is uh transparent and then we cue the audits because uh and then and and the operational procedures give us that uh priority of which are the the contracts that we that that need to go first and at the same time we cap them to the risk that we that we tolerate right um so the way that the code review that
it's very important for us to process and the dimension because that's something that we control that that's internal that the process usually takes to face uh two phases to to go to production and if you're a strategist uh you need two internal peers that means that other uh developers there are strategies that have done strategies or or are familiar with the urine Pro so they review and they check if there's uh more on the accounting side or more of the um or the the things about the strategy and interaction with the protocol specifically how the strategy makes money or is it Financial uh Is it feasible at all and um and there's a
phase two once that's done usually you desperate you can go to eight tax which is or like a staging live staging um environment where we put very little money and and and and and we do all the cycles so it's basically integration testing in a staging environment in maintenance on the rail network but with with small amounts of money and this test integration with the overall bold code with the core uh stable part and um the idea here is for for it to have it uh go through a few cycles of all the importing operations shut down um Harvest triggers everything and the phase two once that happens uh
there's internal security reviewers that will check the code and that phase is more towards how can something bad happen uh what are it is more focused on checking the code to attacks and and exploits and things like that um so out of base of what happened during this process we get a score uh and that's sort of like the score initial score uh that that the audit that the strategy sorry goes out um and that's one example for example the five is the works means nobody review this code and or it was or we start this recur uh something that we we started doing is recruiting reviews uh at some
point this what was the last time that this code was reviewed was more than six months ago I think that lowers the score and it merits somebody looking at it again uh if that rarely our strategy is so long-lived these days but uh it happened so we usually just because something went went to production he had Audits and and it has been live for a while and it doesn't mean that it doesn't have an exploit so we we found out like if I experienced that so uh we sort of like start lowering the scores and try to uh enter what we call recurrent reviews somebody with fresh eyes comes back and sees the code and we found many many stuff that that's interesting went that way uh so
the way you want this is the lower your number the better um so that way you can move to mitigation say okay we need to put allocate more funds into this but the score the Gap is really not where we like it okay what can we do we add a second reviewer or we hire an external reviewer or something that that sort of helps mitigate or lower the odds of something bad happening we also do for new protocols we do uh due diligence so this entails basically a mini audit from yearn if this is a protocol that we never used before we need all this needs to happen somebody the strategies usually writes this down and checks I'm not going to go through
all of this because of time but checks does the broker have Bounty programs uh what are the risks of upgradable uh are we because when you check the protocol the code says something and the rules be changed underneath you and what does the the changing entails and things like that so it's it's better like a requirement for even uh um considering the strategist the strategy to go to production um protocol safety is like I said it's a it's a assessment that we have ideally we would use uh a consensus or or a score that's given by an independent uh I don't know independent organization
outside of us right now is very subjective to what we think um we've seen we we put like the the best verticals are are one our reference point those are the best protocols like uni swap curve out of the protocols that we see before uh how they handle their security is really things that we want to look into other protocols where we decide to put funds and the audit Dimension uh very clear like a five is the worst means nobody audited this even if you have all the return on reviews but if there's no audience that impacts your sport and uh again uh that this is the like the switch switch
model that you see in like airplanes and Air Traffic Control Systems where you put all those defenses and what happens when an incident happens it's basically uh something got through all your defenses uh when when a problem happens we try to focus on the process that means not necessarily somebody missed it uh why did you did you review this when it went to probe where there's no blaming doesn't help it's just uh getting together doing a postmortem and trying to see okay the procedure from the focus standpoint that the procedure fade fail the people that were involved are in this uh something is missing and we write it down we we do root cause
analysis this is an example for example strategy went in production with a potential exploit they we use like you could use Wi-Fi wise just one tool or or different other analysis um and um in this particular case this um the expert reviewers were out where we're in PTO or wherever equivalent in the dialysis of a PTO as they weren't available so somebody else reviewed that I didn't have as much experience Okay so so we that what the solution came out of that is compile a checklist for reviewers because the the contributors come and go really in your Dow uh what do do your procedures ensure that the
experience and and the blood that those contributors had have put up in war rooms and things are documented somewhere for the Next Generation Um so that's how we look at it um it's it's implemented then you you follow through this is normal like I said all things we we didn't invent these are things that are done heavily in other Industries and um we see that this is implemented somehow and not procedure so the future enrollment for this uh we're improving and refining scores this is something that's very accustomed to yarn I don't think uh uh it works for everybody but I hope this ideas give these ideas to all the protocols how how maybe something similar can work for
them um thinking about on chain risk scoring putting all this data on chain and trying to see if that putting the rules there and and decentralizing that and and get a score from the boat this is all on the operational risk not necessarily the Financial Risk uh that's sort of like a different aspect but I think this is useful uh for us and and maybe it's for other um I have a tweet storm on this uh you can check it here and these are the links that are relevant and talk more about the docs and things uh that I discussed here um so that that's that's my talk [Applause]