Decentralized Confidential AI - Dr. Christian Killer

Transcript

[Music] thanks for clapping in advance you don't know what I'm talking about nice so this feels like coming home I also did my PhD here so it's super nice to be back in this building although I have to admit most of the time I spent not here but in the informatics Department which is not that bougie so but still it's nice to be here so I'm head of research with accurist and I'll talk about of course what everybody's talking about which is AI right um but I'll give it a bit of a Twist as our previous um introducers mentioned we want privacy and we want security we don't just want AI for AI

sake right we still want to keep it safe somehow so compute and power demand search and this is mainly driven by AI I would say so of course cloud computing in general digitization everything you uses more power more compute but as we look at the data we can see that these models also use huge amounts of data and also huge amounts of power so we need to be considering that when we think about our next J PT rapper startup and you can see on the right hand side on this box that not just the training compute is increasing heavily but also um just the sheer amount of models that are being deployed and trained is increasing

heavily so where are these things deployed as of now would you think of course in the cloud but the cloud is just an idea that doesn't really exist it's just other people's computers right so I always like to remind people of that that we don't forget that at the end of the day cloud is just a marketing boss word and you interact with some infrastructure from some big Corp and at the end of the day we don't know how they're configured we can verify nothing more or less uh we just have a UI that says very secure and we trust that right but that's today's world we cannot develop everything from the bottom up unless we want to spend all our time configuring our servers and or

firewalls cloud computing in general is very monopolized uh you have a few players they own your data enable mass surveillance um they POS a supply chain risk as we have seen every week every month but like the headlines are not that strong anymore everybody just cares about meme coins nowadays but that's fine I think it will pass over um and also they're not censorship resistant uh as we can see now with politics changing all over the world this might become more relevant over time and they're very hard to localize so the benefit they have compared to an Asian state is that they can just move around and say no we are actually in Ireland we have some data

centers there but we're actually there so they're really fluid um and really hard to grasp so this is like cloud computing in a nutshell and now I'd like to um introduce to you crowd strike which happened last year with a few numbers um there were 8.5 million devices affected worldwide and this was just one security incident we have outage costs between 1 billion to5 billion US dollar and this is an rough estimate that was like after the second week after it happened so probably it was much more um and the root cause was just one file that was updated multiple times a day and there was a bug in this file so and this is mainly due to a cloud being perpetrated in every OS so

nowadays if you set up an OS um yeah good luck because you will have to use some sort of cloud somewhere like it's really hard nowadays to um get out of the grasp of these Giants and best of all then I'll start start my stop my rent and just start to be more productive and more constructive is that other people's computers also can access again other so there's these sub party third party subprocessors with Google Cloud that actually have access to all your data or a subset of your data I don't know if people are aware of this but so that means not just Google Cloud but also other subcontractors have access to your

data um that means it's just an exponential growth of people having access to all sorts of other things they shouldn't have access to so what do we need to do I know this is like the pitch of uh ethereum 2017 all over again but we can actually build it with the hardware and the security software that we have nowadays we need a global scale Cloud Network founded on open source principles um I know it might be a bit of a hard example but it's very recent with deep seek uh we can see that open sourcing things really proves certain things that we can verify although all the political spectrum of this thing uh is a different story but simply open

sourcing things really proves a lot um and open sourcing means that we actually can verify things and not just that we can um you know uh say open wash it uh if you don't know the term that just means that you open source things that actually don't really uh give others your competitive Advantage just to say you're open source but at the end of today uh you still keep most of the things that are valuable proprietary right so um verifiability is the topic here and also this is a quote that was adapted by by Edward Snowden from Thomas Jefferson essentially saying that we need

cryptography um to make things happen in the long run otherwise we'll just have to trust as we do right now so our approach um is to actually move compute not just AI compute but uh to a large part to the edge with a global scale network with heterogeneous devices what it means is that we use phones so these are actual pictures uh from few months ago from few weeks ago collected in our community where people run um our software on their phones at home how exactly that works in detail um I will show you and if you have more

doubts about security of this thing please come talk to us at the at the boo because we like to be also on that end since um our team is really um security and mobile first why mobile devices you have much less cacks but also Opex which means Capital expenses are lower because the server costs couple of thousand dollars let's not go to the GPU side or h100 side which is like a small house uh but the operation expenses are also much lower which brings us back to the power Demand right because if you use less power you can run much more compute per watt so this is the second thing performance per watt is much much better

with mobile devices because they're optimized to actually run a lot of things with not too much power right otherwise we wouldn't have them in our pocket also there are a based Hardware which makes them like um prone to be very specifically optimized for certain tasks with all these GPU accelerators Etc and it brings us to to the topic of upcycling as well it's like a cheat code that we can use um like if you recycle phones uh or broken screens if you have broken screens on phones you can still use them for their compute you just have you don't have to throw them away right also we support IOS and Android which um is quite nice because you can

either decide to fully dedicate an Android phone which is then fully locked down and cannot do any other computation um or um use this light version which is simply an app that is running in the background which of course is not that secure if you think about sandboxing from a software perspective right but at the end of the day the hardware is probably the most secure that that you can get for the cost um because like for for example Apple spends billions of dollars every year same on the Google site on the pixel site to make these devices imp impenetrable of course this assumes there's no back door on purpose but that's another discussion for um after

the conference with a beer um and of course one interesting thing uh from a network perspective especially for me is that we have residential IPS and we have 5G uplinks which are much more resilient than using fiber cables as some of them tend to get cut in the middle of the ocean uh which happened just a few weeks ago again so this is my last slide because I just have 10 minutes I have one minute left and I'd like to outline one example that is very recent um on the right hand side you can see a terminal output which is not fake this was actually done by our co-founder alesandro he downloaded deeps R1 deployed it to one of our

processors as how we call these these entities that run our software which are phones um and then actually interacted with the processor directly and ran I think think the prompt was built a website and it actually ran quite fast and this was one single processor um so imagine like chaining them together with USBC or over ethernet then you can really uh build a Powerhouse with a couple of thousand of dollars that can run llms uh either training them or uh inference side and then actually use it in the real world for decentralized confidential AI unlike running it in the cloud or buying like

millions of dollars of Hardware yes um thanks for listening that was it from my side uh we also have a so-called Rebellion that we run where you can onboard devices and then get incentivized uh through our test net token thanks and have a great Conference of the deep VAR I think R1 r distilled thanks thanks [Music]