Enhancing Large Language Models for Smart Contract Security - DeFi Security Summit 2025

Transcript

This one. >> Hello everyone. I'm Shashank and um glad to be here. Um every year uh we are losing almost two billion dollars to malicious hackers and uh if you if you if you see the trend uh most of these you know are handful vulnerabilities access control you know like proxy upgrades business logic etc. And to scale security you know it's like we need to do automation and and it's uh it's pretty you know it's like uh and and to do automation initially we used to do you know like algorithm detectors but which is very outdated right now to find more business logic vulnerabilities we have to use AI because it is more

context aware but how and that's what my talk is about. So AI is often treated as a buzzword out of box LLM new u uh large language models neural networks trained on massive text and code bases which are very great at quering you know like code snippets but then when it comes to finding vulnerabilities we have two problems one is the hallucination of the vulnerabilities which means it's a time waste and then the second thing is missed exploits because it could not understand the deeper context of the uh of the business logic. So what is the solution? So the solution you know it's like is what if we build an LLM which acts more like a maybe a junior you know

it's like smart contract auditor and how do we do that is in my next slide. So uh if if we have to do that we have to do majorly three things. The first thing is you know it's like uh the uh the context uh the the context of uh uh the dependency aware context. So when a human audits they don't read the code line by line they uh they follow a call graph you know the inheritance you know hierarchy and you know the storage slots etc the how the value is passing through and and that's the same thing which we need to implement with the LLMs. So we create you know a trees call graphs and call only those functions which are relevant to the questions which we are

trying to ask like who is calling whom. What this does is it narrows down the context rather than feeding the entire codebase to the LLM which will cause hallucination. We are just giving a narrow context and narrow context means you know more clear understanding of the bug to the LLM and hence better results. Second point is auditor labeled corpora. So what we do is like you know LLMs will learn what we teach them. So and and how we do we do we do that? So there are audit contest, postmortems, you know, public audit reports etc. If we label them correctly and then pass it to then LLM, it will learn better and hence it will give better results. But then it is

it should also not be limited to your your rag based architecture whatever you are building in the vector databases. It should also know what is not just a bug, what is also not a bug and hence this also helps in reducing false positives and it has worked very well for us. The third thing is training. So training you know it's like uh again training LLM trainings is is an expensive thing. So how do we approach that in a cost effective and still you know like efficient manner. So the first thing we start with is like supervised uh supervised fine-tuning where we you know like we do the supervised finetuning and then we do the uh parametric optimization using you know uh uh using the Lora and Q Lura method because in

that you just have to train the one to 3% of the parameters not the entire uh entire LLM and hence it is very cost effective and then on top of that we you do preference optimization methods like DPO and ORDPO and when we combine all these methods it becomes a better you know like a better vulnerability uh you know like assessment tool which uh which believes in uh which which follows the context and hence gives the better results. So uh thank you so much. [snorts] [applause] >> Awesome. Thank you Shashank.

Uh, up next we have Alejandro Munos McDonald, security researcher and triage lead at Immunifi.