How Blockchains Can Make AI User-Owned

Transcript

[Music] And we're back. Uh thank everyone for coming after lunch. Hope you don't fall asleep uh during this presentation. Uh today I want to talk about how blockchains can make AI user owned. For those who not familiar with me, I am a co-founder of Near Ilia Suhen. My background is in machine learning and AI research. I worked at Google research um until 2017.

I was working across infrastructure like TensorFlow as well as natural language understanding back in the day. you know before Gemini you could ask questions and there was an AI model trying to respond uh reading different articles now one of the challenges we faced there was that uh models we're using were highly not performant they couldn't read all the context in the time that Google needs to respond and so that actually what gave birth to the transformer model which is a TGPT and so I'm one of the co-authors working with a team to actually bring this uh to life. Now I left Google to start near as an AI startup. It originally was about using

transformers to uh teach machines to code which now is called VIPE coding. Back then we had uh way less uh compute capacity. There was no uh kind of even A100s yet. And so we're trying to get a lot more training data for us to train our models. And so we had a lot of users um kind of students uh around the world who would give us um kind of contribute data in various ways write some uh small amounts of code write some descriptions for code and we had challenge paying them right in China students don't have bank accounts they have which pay in Ukraine you need to sell half of your dollars in arrival back in the day uh in Russia you know PayPal didn't work and

so like all of the systems had some kind of challenges and so we ended uh looking at blockchain as a solution to our problem as AI company. How do we actually facilitate this contributions uh from different people around the world to our model and we didn't find anything that would scale and would be cheap enough for the for our need and that's how we started Near Protocol in 2018. Now fast forwarding we're now here with AI evolving faster than ever. we have effectively AGI that is coming closer and closer where most of the digital work will get automated and physical work is not far behind with robotics.

Now the challenge is whoever controls this AI, whoever decides how you can use it, whoever defines what bias the training data has, whoever defines what safety rules can be applied controls everything. Right? We already have this. You know, I was just checking two Nobel prizes last year were won by effectively AI. Uh the, you know, a lot more code has been written by AI. A lot more decisions are going to be delegated to AI as well. And so whoever controls how this AIS are trained will effectively affect all the decisions, all this code, and all this inventions.

There's also a paradigm shift on kind of application and user side. We're going from kind of web two where we had uh kind of converged into this aggregator model right we had Google for aggregating all the websites we had Amazon for getting all products we had you know social media for aggregating all content um where we now seen this transition where um you starting to talk to you know kind of instead of asking these models you're talking to your AI assistant and and it kind of routes you maybe to this content but actually tries to answer things directly. Now the next step is it actually goes and does even actions on your behalf on those

different websites right it goes and registers for you uh and you know books tickets and figures out how to uh do minion tasks. Now the interesting step be after that going to be where you can trust this AI with more decisions because it has more context and now it's able to go in directly for example talk to AI of the pharma company and figure out exact medicines exact proteins that need to be synthesized for you specifically based on your medical data right this bypasses five or you know 10 different middleman decision makers everything from FDA to you know pharma to kind uh all of the different entities including pharmacies that are right now

facilitating how you you know transacting with uh kind of medicine. So all of this is the transformation that kind of we see happening with agent network right this is this is going to happen with or without web 3 but as I said if AI is controlled by a single party it's effectively this whole system will be controlled by them now there's also just general challenges in AI one of them is monetization right now if you're building kind of a normal AI company uh you effectively have very scarce resource of compute and uh you build a model or you build some agent.

You offer this to the uh to the users. If this is a good model, you get lots of usage and now you run out of compute. You also don't have now capacity to train new model because you're out of compute and you know getting more compute uh at a good price is actually pretty long process you know takes time because you only want to get it from you know few of the providers uh like Amazon, Microsoft, Asia or Google or Corv and so you actually have a and even though there is maybe data centers around the world you have this challenge now you also taking all the data risks all the privacy risks of the users, right? You're handling all the user data as is this. Now the other alternative is

you just open source the model but the problem is it's not sustainable, right? You're not making any money. There's no way to reinvest and train your model and this is expensive process. So right now we have like few companies developing this a ton of people especially in universities s sitting on sidelines without access to compute. So we have unsustainable model and a very complex kind of logistics of compute and trust uh on the closed source side and so there's a lot of kind of challenges there because again as a company you don't trust external data centers because your models your IP and if this IP leaks then you effectively lose your business and so you don't trust with your model to some external

parties and so this ends up having this kind of lots of problems. One is you only trust a few data centers. Data centers don't build that fast. They need a lot of electricity. This is all in kind of some close proximity. There's like few cities uh kind of in regions in US where right now this is buildout happening. Um even though there's again lots and lots of space with electricity around the world. Uh which also leads to latency. Right? If I'm sitting in Japan and I'm using chat GPT, I need you know round trips to Texas. you have um data that you know gets uploaded and you know there's been stories of for example Samsung their data got leaked to uh their competitor through openAI because

they were just using uh you know copilot style product and their code ends up being in training data for uh openi model you have so this goes to you know general corporate security especially if you're handling users data this is even more private right a lot of legal firms financial firms are not allowing to to use AI on user data because they're afraid that this is gonna leak. And this a sovereignty question right at the end if I'm a country and I'm using a centralized provider of the models from another country that country effectively have more power over me especially when the AI starts to become really big contributor to GDP big contributor to automation big contributor to decision- making of the country. So all of the

challenges now on top of this there's kind of this more fundamental safety challenge right there's a lot of conversations about is AI safe uh how do we ensure it is and uh kind of there's always a reason oh we're not going to open source our model because it's unsafe and then people get you know open weights model and use it uh but generally the problem is the safety is applied very unevenly right we don't have kind of a universal way to apply safety rules across all models that and how people participate with them. So that's why we formulated a new way new uh vision called user on AI.

It's approach where AI is optimized for users well-being and economic success not corporate profits and it's built on top of you know all of the kind of approaches we've built in web 3. So what we want to make sure is everyone has access equal access to the models while the safety applied equally as well kind of safety constraints right nobody should be able to use it to design nuclear weapon nobody should able to design bioeapon um user data should be preserved in private unless user wants to contribute it to training in in return for something so confidentiality and permissionlessness is extremely important we want to know how models trained again right now even the so-called open source models are

actually not open source. They're open weights. We don't actually know what went into them. And if you don't know what went into them, you don't know what actually bias this model has, right? Is it leaning right or left in US? Is this more pro, you know, some religion? We don't know it. You also don't know if there is so-called sleeper agents. Sleeper agents is a special way to train a model where it will ex have a different behavior under certain conditions. For example, normally when you look at the model, it does behave normally, but when you're using it to build production software, it will actually import some malicious library and effectively infect your production system. So you want to make sure that you know what training data went in verifiably. And so you want to know

effectively the whole provenence of training of this model. Now, more importantly, this needs to be a state-of-the-art model, right? Nobody will use some meh model. Uh it people will prefer better product over you know values. That's always been the case. That's why crypto been so slow to adopt. Now uh you want to make sure that this state-of-the-art model has verifiability of data and model training. So we need a whole setup for that. And we need continuous research and improvements, right? Like it cannot be just like hey cool we build one state of the model and we're done. It needs to be a continuous process of getting state-of-the-art model and research. There's also a very uh you know important aspect that if you

completely open source that means that any other closed source lab can take it open source and build on top of it and so you cannot be at state-of-the-art if you're completely open source. So you need to create some asymmetry of information for this to be really true. So we introduced a solution called decentralized confidential machine learning at NVIDIA GTC. It's really about creating a platform for this kind of user own AI uh and the road map how we can get there. So it's it's really about creating verifiable and confidential computing that can be run anywhere in the world. Nodes can join it. Uh that allows to monetize models as well as train uh fine-tune models in a

verifiable way. and it offers a safety tooling that is kind of applied to everyone uh and nobody can get privileged access to the models. How does this work? Well, first it's built on top of trusted execution environments. So, as of about a year ago, uh Nvidia has enabled a confidential computing mode on their GPUs that works with Intel, AMD and ARM uh uh kind of T environments. And so uh which themselves have improved at least Intels and ARMs have improved uh since the past. And this environment itself allows to effectively run computing in a vault right in such a way that whoever

even owns the computer is not able to access it uh directly. It now has GPU enabled and interestingly it has this verifiableility property that this code is running on this computer. It ran this data and only this compute was run on this data right meaning you can guarantee that after the running on my data it will delete it and will never you know store it or use it again. So you have this interesting privacy confidentiality properties that you actually don't get even from some of the FHE solutions and definitely don't get uh with ZK and other kind of traditional methods. Now we all put it together into what we kind of this decentralized machine learning cloud where a user effectively

uh paying on chain uh for using the models and then can connect directly to the data to the compute providers and run inside the confidential VM different models and different agents and then there's a lot of pieces that need to be done to kind of tie it all together to ensure you know livveness of computers to ensure kind of this model marketplace uh But it provides us with this fundamental building blocks for bu kind of running and building user own AI. So we have a number of teams actually building this together. This is not kind of a single team effort in a you know decentralization spirit. And so uh we have you know on the near one side the team that builds near protocol they're

doing a lot of the uh key management and uh kind of uh encryption decryption. We have proof of response done uh to ensure the livveness of nodes. We have Fala doing trust execution environments and consension inference. We also have teams building on top of it everything from model training to benchmarking to hosting to rack to private memory uh and and various data synthetic data and human label data. And this is not a full list but this is kind of just to show you that this is a whole stack of teams working together. Now this actually is about deconstructing the AI lab right? If you go into one of the centralized labs and you open it up, right, there's actually a number of teams that working together and they're creating a flywheel of

research that's happening inside, right? And so it's important to kind of replicate this process in a decentralized way, opening up ability to actually source talent from everywhere in the world and give them opportunity to participate. Uh so it starts with data, right? You want data acquisition, ability to find better data, uh clean it and improve it, right? There's a number of techniques and hacks to improve this as well as using kind of a lower grade models to uh refine data. There's a synthetic data proc kind of reprocessing as well as user contributed data. If users are willing to contribute their data in various ways which here we can actually guarantee that nobody no single person can actually sees the data. It

will be cleaned in a very specific way and then it'll be used for training again without revealing the individual data. Um then we go to pre-training step right this is kind of the foundational part. This is about ali algorithmical improvements improvements to the training process itself. How do you do curriculum learning? How do you improve you know uh ability model to learn from less data from seeing less examples and then obviously research on how do we do it across a distributed and decentralized cluster. Right? There's a few teams that have been already working on this and making good progress. Um and decentralized training is kind of this uh I would say holy grail that people talk about because it would allow us to actually replicate the whole process

right if we can do decentralized training on DCML that means we have this verifiability of where data came from how it was processed and then this model artifact effectively came from this data now you can actually keep the model artifact encrypted right the weights of the model doesn't need to be open so open to everyone they can be encrypted and available in this decentralized network. This actually has all the properties of open weights. You can use it permissionlessly. It's available everywhere. You can use it even on your own hardware. Uh but it's monetizable now. You can actually charge. So even if you are like you know individual developer who just trained a model, you can now charge for using this if it's a good model. And so this solves really this fundamental problem that kind of

open source is not monetizable. We're actually kind of changing the game to have you know encrypted models with open source to be monetizable and at the same time have all the properties and so kind of if you simplify there's effect three steps. Step one create a market for inference where you can serve this encrypted models so anyone can participate upload their models and uh uh serve it. uh this solves for this kind of trust problem between model providers and solves for privacy from companies and users who care.

Then we can run benchmarks competitions for different aspects of model building. Refine data better, get uh you know get the reward, improve the model algorithm, improve the model training, get reward, run this continuous benchmark effectively as in you would run the teams internally with you know OKRs and uh bonuses on uh people actually coming up with new ideas. Step three, put it all together into you know training fine-tuning post-training process and actually deliver different specialized models for different use cases that people can use that are beating on this benchmarks. Uh and that's really all together this is how we build this userowned AI and this is what we believe is the way to actually

ensure we're not going to end up in 1984 you know big brother style world. So this is pretty much it. If you're interested to learn more, follow us on Twitter at near protocol at Elbugdragon. And uh thank you very much for coming. [Applause] Have time for one question, two questions. Hi, thank you. Very interesting. The question is the near AI which you're building can it be rolled out as a

wallet itself so that the agent is a wallet? So uh there's a whole kind of this is this is really the infrastructure and foundational side. There's also the application side. So on application side the wallets you know your user wallet uh will have your private memory private data for AI. You can have agents running there on your behalf but you can also run autonomous agents and autonomous agents have their own wallet. They can enable and transact and you know through near chain signatures and intents they can transact everywhere. They can even start buying physical businesses hire people can do all those things but they're autonomous. So there's effectively a dichotomy between agents that work on behalf of people behalf of

businesses and autonomous that run kind of themselves. Think of them as you know fully independent companies that may have shareholders through a token but effectively run themselves. Hi Pedro here. Thank thank you for the talk. Um I have a question related to the maturity of your project so far. uh spec specifically about the use of TE's and the GPU TE's. Are you guys already kind of using them heavily and do you have already kind of remote attestation figured out and how this integrates into your whole um ecosystem? Yeah. So, uh the T and confidential inference is live right now uh through follow and then uh what we're finishing right now is the

key management for the specifically recovering keys in TE. So you can decrypt encrypt models uh for this kind of serving but you could right now you can use confidential inference on open source models. So this is available right now you can verify all the signatures kind of invidia etc. And we have key management authority on near that can effectively provision all this. All right. Thank you very much.