Testing Best Practices in Foundry Projects

Transcript

foreign thanks for coming um who here is a Foundry developer I'll raise your hand very well so pretty much everybody 80 of the audience and who here has seen um our savior V2 core repository which has influenced this talk well one one two three four okay so like 10 people not too bad um we're going to talk about like testing and best practices in Foundry um which is this development toolkit for ethereum applications um my background is um yeah I've been a ethereum developer

for five years um now I'm the co-founder I started to lead at save their Labs um we're building a money streaming protocol um on ethereum and um with uh we've worked with Foundry for like 15 months um to ship our V2 protocol um which is pretty cool have a look uh we have stream nfts non-linear streaming and many cool features um and yeah we just announced that last week and um part the part of the protocol was um our development and research process was invested in like trying to figure out what is are there any objectively good and like you know good ways to write your founder tests and I worked

very closely with the founder team to come up with like a set of uh you know guides and and uh rules of thumb for how we go about like you have your protocol and then how do you use Foundry to like scale it like test scale basically um now let's talk a bit about Foundry you know they made their debut in 2002 um and before before voluntary Solita was rarely used for testing for my contracts if you I mean truffle and hard hat I think everybody uh no like has heard about these Frameworks they were using JavaScript and typescript but the same thing basically um to like test so we have solidity which is like this financial

um uh you know programming language like handles money uh we were using this you know 25 year old all language used for uh writing web apps to test like finance apps right so um the the the problem is that many founder tests because of like Foundry super new are like not organized um and the way it goes is that you typically start with your um simple example tasks like this um uh you know it's nice you have like a simple unit test and um you know for for like simpler cases this is enough um however you know contracts have a lot of you know paths and like functions and and uh state so what happens

um is that you end up with like monolith test contracts where you have uh test function name one two three four and uh there's like no structure at all because again it's like solidity has just been you know a solution is still very new as like a testing framework language um right so how do we solve this uh or like what our problem is that has you know like that the lead people there is I mean firstly is undirected development so there are there are um uh there are no English specifications for your smart contract protocol um there you have no fault structure you I've seen these files of like thousands of of

um uh you know tests and they test different contracts and they test the same function but like the tests are like not ordered and uh there's General like generally a lack of categories um between for example um you know like simple unit tests and like Fork tests which run against ethereum mainnet um so how do we break free um well the first and foremost thing go to the founder book there is um a tutorial called best practices written by uh Matt Solomon and uh other sales and various Foundry community members which is absolutely beautiful um they have like this two-page collection of like tips and tracks um uh that should be like I think the first top three things you read about

Foundry when you get going um but then also there are a couple of recommendations I'll give you in this talk and I have classified them under two um uh categories it's like high level structures and low level design in spec um at a very high level the idea is like to split your founder testing into clear-cut categories um because this helps with like filtering tests with your CI um also like to hone in on you know various aspects of your protocol like for example if you want to focus on your like reverts um then use contract inheritance like have like a base test contract which um you uh put your shared logic in there and then you inherit from that in all of your other tests and uh the good old

principle of like using the you treat the contract name as a describe block uh if you are familiar with mocha like in JavaScript they have like these describe um like function with blog contexts and the idea is like you create a test file called you know Foo which is your contract under test and you use that as like a pseudo describe block um okay so what are the infamous categories I've been talking about um we uh have structured them under four um uh categories and they're a unit integration invariant and Fork um they each serve a different purpose and they're like they each attack your protocol in various ways um but the idea is units are um tests involving a single contract

integration or like tests involving multiple contracts and this means including if you your function includes an erc20 contract that's an integration because in years to 20 like there are many rc20s and you want to like I like draw a distinction between your you know Simple Pure functions calculating math uh you know like logic and your complicated Integrations and that that includes the rc20 then you have your invariance which is um um uh like expression that should always hold true for example in the rc20 the total Supply should equal the sum of all the token balances and finally you have four tests where you want to like make sure that your tests actually run um against the real ethereum mainnet contracts and you use like Alchemy or

something to like Fork minute then each of each category from before um you can split it under two other categories um which is concrete and fuzz concrete are your standard deterministic test that I have no inputs um they're just like hard-coded values uh to to check that you know the output is the same as the input but then founder gives you like fuzz like like fuzzing which is super powerful um but because the fuzzing logic is typically different like your tasks will look very differently when you have fuzzing arguments you want to draw a distinction between them um again it helps with CI filtering and generally just like explaining your

protocol to somebody who's just like picking up your um testing suit now uh let's get to the the heart of this presentation which is the low level design is back this is our Innovation um is the branching three technique I will go but I'll present in the next few slides this is a very minimal um specification language that we basically invented which is super cheap like I mean there's there's no API or something it's just like using ASCII characters um I'll show you in a bit but just very very briefly to mention this naming convention here again path tips of The Foundry best practices team um uh you know like use this because

um if it's useful for like filtering tasks for example if you want to like generate a gas report you don't want to look at the reverts you don't care about like when somebody you know uh makes an error and they get like a like a revert you want to get like a a an average cost gas cost for the successful execution paths um right so the ratchet 3 technique um the idea is to like be before you write your solidity test um write an English specification um which uh defines the contract state the functional parameters and

um builds up this hierarchy of um of uh basically the all the possible execution paths in your in your function um you map them out with English like this and uh this is super useful not just for yourself like you know map how the function logic but also like you can share this with your non-solidated team members and they can get like a feel of like what's going on in the function it's like super cheap I mean you just install like a vs code extension to get highlighting um and it helps you with like structuring your your I mean just for yourself like like think deeply about all the possible States or function can be in and this is an example uh this is how it looks in solidity the trick is to

use empty modifiers which just give you the um hierarchy of like how what is the contract state that should occur for this task to pass because in the real world there's never just like three lines where the contract stage you know is just there in the background the contracts that massively influences whether your tasks should you know pass or not so this gives you like a a like an English um like a natural language um specification for what should happen for this assertion to pass um very well so what are the benefits as I said it's plain English easy to learn and teach um it also doubles as a test structuring

approach um because you like have the tree files in your actual version control and it can be shared with non-technical team and the best thing is that this is potentially automatable so imagine um you know building a tool which starts from this tree and then it generates like a skeleton solidity file with the empty modifiers and all the test function names and I mean that I mean I'm I'm envisioning building something for like like using GPT to like you know convert the English specification into uh test function names and modifiers and that's like like next level testing in solidity because you like you know a third of your job will be done by this tool um if you just have the three then everything falls into place

um right and just a a quick comparison here um like the reason why B to T is is cool is that it's both entry level and it's moderately effective um you know BTT was I was inspired by cucumber gherkin um if you guys are from North Vet it's a super duper cool language it's just that it doesn't have support for solidity and it's way more formal than what I showed you and the barriers for answer is higher because you have like dozens of keywords and they also have like they enforce like a matching between the tree and the solidity code I'm not enforcing anything I'm just like the ability approach is like to start with the tree not not the solidity test like first Define spec out

your of your function before writing the test that's the core idea and anyway and finally Satora TLA you know uh impressive languages and products but the variance reentry is super high senior level costs money uh it's highly effective but uh there we go so like and by the way these are not mutually exclusive so I I could Envision a repo using BDT for like you know early designs pack and then as time goes on and you raise money whatever you go to Sarasota um anyway um that was it thanks for coming uh you can find us uh at sailor.com uh saved her on Twitter and I'm personally at these two handles on on tutoring GitHub and the QR code here is um pointing to our GitHub

repository where you can see uh hundreds of three files and how we implemented them in solidity with again hundreds of modifiers so if you want to have a look yeah that's how we built thanks for for coming [Applause] if you want to ask yeah if anyone has any questions so you should the modifiers that you apply to the functions and you said did you say those are empty modifiers do they do anything are they actually doing setup or empty good question let me go back to the example um

uh so um you don't have to have the modifiers be empty um it's just that you you want to have them so that you mirror every single node in a tree so if you go here all of these branches uh the idea is you want them to occur um for the test to pass but in some cases the maybe the state is defined in some setup function outside of your test contract so in that case you still want the modifier to give you the English specification for like what's what's going on in that particular test um but you don't want to add like you know Define any state in there however there are cases when you do want

to do that and this example here shows you that and they when not null modifier I'm creating a the default stream and that will be used the default stream ID will be used in the actual test when I call the status of function um yeah I've actually done something similar like the modifiers but I've used contract inheritance instead and then basically called the contracts when something I don't know if you've experimented with this approach as well and if you have like a comparison to using modifiers

um the the stronger answer is that it's not mutually exclusive so um we're actually using a lot of contract inheritance in our own Corner base as well it's just that for um for complicated trees this is a simple example uh in our code base we have trees with like 10 uh levels of depthness uh uh and and for those like imagine having you know like 10 contracts that inherit one another um it gets really clunky when you have complicated trees so modifiers are like much more slim um but you know as I said like they're not mutually exclusive like you can have um for example um you can have a big shared contract that sets up you know 10 streams you

know I mean like say those case and reuse them in multiple uh test contracts which have other modifiers um but yeah it's a question of like how slim your code is yeah I wanted to ask um so the modifier that you show is it like similar to setup and tear down when we write tests in in JavaScript and hard hat uh sorry the oh the question was if this is similar to this describe Block in JavaScript you know Nova you know in in JavaScript would often you do tests where you write setups into in teardowns like fixtures you know you set up

something um you like setting up can be deploying of a contracts setting all the access control and then you run all the tests and then you have a tear down at the end of the describe block you clear everything like you clear over contract State um do you have something similar like that to recommended in Foundry or oh sure I mean that's handled by founder itself uh it's it's called a setup function and Foundry um after every test it like flushes out the entire blockchain State and you don't have to think about it at all there's like a vm.snapshot cheat code where you can do like more advanced setups or like you manually clear the blockchain state during testing but you

typically don't need that like we're we're I don't think we're using that in our on-call base and we have like hundreds of files so um yeah okay cool and yeah what's the tool that you use for scrolling in the slide because this is also a man oh yeah so you mean this yes the like the presentation software yeah like how do you oh it's called reveal.js okay thanks I will upload my presentation on my Twitter profile and you can like download it from there question anymore this one uh what are you most excited about in in Foundry and variant testing kind of moving forward is there anything that you think like an evolution that's coming down the lane that you're

particularly excited about well I mean voluntary invariants are um I think the the the coolest type of tests here if you if you go if you're invariants are Bar None the most powerful um uh like testing framework here um what am I excited about I there isn't anything in particular like like invariants are pretty nice as they are today there's only some hiccups here and there like some missing features that I reported on GitHub on Foundry but uh I would say that I mean just found it as it is today is is a major Evolution for for ethereum development so sorry for the boring answer but like Foundry is is

the exciting thing now and invariance if you don't have invariance tests like that should be your number one priority tomorrow or like when you go back to your development uh uh uh back at home like invariants are a way to um so if you don't know them it's like Expressions that should always hold true no matter what and it's like Fast differently like call your protocol in like hundreds of calls sequences and whatever it's like super powerful thank you okay one last question uh thank you um so there's a new EIP for transient storage so I was curious maybe do you guys have any ideas what to do with that or how it could benefit testing or something else with the development tool

um that that way IP will be useful for um table tests there's a GitHub issue about this um Foundry doesn't have to yet have support for um uh declarative um uh array so you specify an array of inputs and each input in the array gets tested founder doesn't support that natively and you have like to use these these complicated storage structures with like storage arrays to um have like a modifier that Loops through all those uh uh you know uh Define array so you know trans and storage will basically speed up that that testing Paradigm but um I wouldn't say it directly affects what I presented on this

um uh keynote here because you know the the tree is just uh uh let's see this is just like a non-solidity file it's like you just keep an inversion control it's it's more like a a an emerging human phenomenon what I'm talking about here not not so much EIP uh implementations thank you very much thank you for having me and just one quick mention at the end we have saved your swag like 20 T-shirts if you guys want to grab one here they are thank you for for having me again [Applause] [Music]