The AI Privacy Problem No One's Talking About in AI with George Zeng

Transcript

Do I want to really want to be sending that to Anthropic or OpenAI or to other companies when it's like the most important thing that I create? I don't want my source code to be leaked or used for training or come up in some other kind of response, right? So, privacy is a really really important part of uh deploying a local LM. There is a thesis or a theory that we could give it its own crypto wallets and then it can start making transactions and it can be using near intense and traversing the multi-chain and doing everything in crypto that we can do as humans. Maybe even far better than we can do as humans.

Welcome back to AI Supercycle, our premier AI show airing every single week presented by Near. We cover the ins and outs of decentralized AI, privacy, and the future of this massive technology. Near is the blockchain for AI and the execution of AI native apps. You can check out Near's latest AI product at near.ai. Sit back, relax, and enjoy the show. All right. All right. All right, we've got George uh here from Near. Uh he was just saying behind the scenes is the craziest open source project that he's worked on, which I I think is uh is pretty accurate. Um George, it's your first time on the show. We're friends o over with the guys at Near um and and you guys are doing incredible work. Ilia

came on a couple couple weeks ago and said, "Look, we're an AI company. All of this crypto blockchain stuff, that's all a side quest to ultimately empower a lot of the AI work that you guys are doing." Um I want to get your reaction to a few things. We're going to talk about Claudebot. We're gonna talk about Open Claw. Near intense uh plays into this as well. Uh before we do, we'd love to get a quick background on yourself. >> Sure. Um thank you for having me. First of all, I'm George. I'm the chief product officer for Near and I'm also the general manager for Near AI. So I've been thinking about a lot of these questions you've been talking about for a long time and certainly in the last few weeks all of this just has accelerated 10 times 100 times.

>> It's it's incredible. It's incredible. Um, like I said before we get into Open Claw and Claudebot, um, Anthropic just a few minutes ago released these Super Bowl commercials. Have you had a chance to see these? >> I have not. That is very intriguing. >> I want to I want to get your reaction to these. They're phenomenal. So, we're gonna we're gonna pull it up. Um, let's let's pull it up, Leo, and then uh and then we'll play this and George uh we we'll get your reaction. There's going to be one uh and then the other. So, okay. Let let's get it up, Leo. and we'll uh we'll go ahead. We'll play this for for George and the rest of the audience.

>> Okay. How do I communicate better with my mom? Great question. Improved communication with your mom can bring you closer. Here are some techniques you can try. Start by listening. really hear what she's trying to say underneath her words. Build conversation from points of agreement. Find a connection through shared activity, perhaps a nature walk. Or if the relationship can't be fixed, find emotional connection with other older women on Golden Encounters, the mature dating site that connects sensitive cubs with roaring cougars.

What? >> Would you like me to create your profile? >> So, Anthropic is taking shots at Open AI for putting ads into their responses. And so, this is, in my opinion, this is an insane insane clip. Like, they're going to play this in the Super Bowl and it's going to catch a ton of people offguard. Um, George, be before you we get your reaction, there's one more here. Uh, it's also very very funny. Uh, Leo, let's go ahead and we'll we'll we'll play the next one here.

>> Hey, can I get a six-pack quickly? >> Perfect. That is a clear and achievable goal. Would you like me to tailor a personalized workout plan? >> Yes. Perfect. Let me personalize this for you. Let's start with your age, weight, and height. Whenever you're ready. 5'7, 23 years old, 140 lbs. >> Got it. I'll create a plan that focuses on aesthetic strength training. But confidence isn't just built in the gym. Tricep Boost Max, the insoles that add one vertical inch of height and help short kings stand tall.

What? >> Use code hidemaxing 10 for big discounts. >> Insane, man. All right, Leo, we can we can take that off the screen. >> What's the difference between me and me and you? >> It's pretty incredible to see how this is heating up. Um, I just want to I just want to get your your your reaction to uh to the Open AI versus anthropic uh LLM wars. >> I I mean, I understand where they're coming from. It's like um there's old saying in Silicon Valley that um if you don't if you don't know how you're paying for the product, you are the product in many different ways. You're the monetization for the product, right?

So, it's really funny that uh Anthropic is taking these shots at OpenAI. Like I it also I guess points to the fact that I guess Anthropic never plans on charging for ads or or displaying ads. Sounds like that they're only going to go with subscription only. Um, I did think that the first ad is a little bit disingenuous because I presume they're uh open air will never influence the actual LM results with ads, but actually even injecting ads into the the response itself just makes people feel like they they trust the LM a little bit less, >> right?

>> I I certainly would feel that way. >> I I same here. I I mean even the thought that something might be an advertisement gives me the sense of skepticism to probably distrust the entire response a little bit less. It >> absolutely >> and and you know they I I see Claude going this enterprise model. Open AAI is is primarily taking a retail approach and so to think that they are you know starting to incorporate this whether it's specifically called out, you know, they're going to get a lot less click-through rate and engagement via those advertisements. the the way in which you know the deeper they embed it and they abstract the fact that it's an

advertisement away is this really weird spectrum between more organically weaving it in versus sacrificing trust and and you know the reputation of their end users. >> Yeah, I I hear you. It's also a little bit different from like a search engine because a search engine displays information and display its ads whereas an LLM will summarize and will actually give you a specific recommendation or point of view. So I think some people may be swayed that hey it's it's like if you have a friend who has an alter ulterior motive, can you really trust what they're saying or are they actually telling you something to advance their own financial interests?

>> Yeah, you're you're absolutely right. It's different from searching that way because everything is synthesized and so you don't know what you can't backwards uh can't reverse engineer what the incentives are because you can't really untangle that black box >> here. Here's a spicy take, right? Yeah. >> Anthropic can take pot shots at um OpenAI, but they have the same user data >> that OpenAI has, right? If you go to anthropic and you ask it, um, hey, um, I have a, you know, a relationship difficulty with my mom.

Help me in terms of improving that relationship. Even if they don't show you that ad, they do collect that data, >> right? And they do use that data, >> right? Which is why one of the things I think you know that we believe here at Near is that the importance of confidential inference and userowned AI is no one should own your most sensitive conversations and your most sensitive data. That should be stuff that's like private to you. >> Yeah. >> Right. >> Yeah. >> Yeah. That's a tremendous tremendous segue. Um did you want to did you want to go ahead and finish that point?

>> Yeah. I'm just saying that you know you can you can throw stones but you know uh Anthropic also has a glass house so to say with uh with some of their sort of >> data privacy and data collection practices as well. >> Yeah. Yeah. And like I said that I think that's a tremendous segue. Uh you guys and I want to get a full uh chronological history here of the steps because you guys are shipping incredibly fast. The AI space as a whole is moving extremely fast. Um, and so, you know, why don't why don't we start at the beginning and then I'd love to get caught up on where you guys are at today. You know, privacy has been an an incredibly important piece to what Near is building. You guys came out with private chat. Give us just a little bit

more context about where private chat came from and then we'll get into what every, you know, all the developments since private chat, which was just a few weeks ago, but you guys now with Clawbot and OpenClaw have have really developed a lot since private chat. So, we'll we'll go through the full history here, but but yeah, please set set the scene for us. >> So, I'll take you way back way back to when Ilia was at Google and um helped create the attention to all you need seminal paper uh that helped birth a lot of the LMS via via transformers via this sort of like invention of transformers.

Um but near I'm not sure if you you know this Robbie, but Near was actually founded as a AI company. The name near comes from the Ray Kershw book the singularity is near. So Near started out as a AI company um built a very successful layer 1 blockchain along the way and in the last few years have been building more and more towards this vision of userowned AI right probably talked a little bit about this but we see user owned AI as essentially um I'm not sure if you remember the earlier days of the internet when there was a period of time when where it felt like AOL could have been the the entirety of the internet like one company could have owned the internet

because most American households were accessing the internet through AOL. AOL controlled like the main portal, the websites, everything. The entire experience more or less was AOL. But then the internet became this crazy wild and free and beautiful place where you can spin up your own websites, where you can spin up your own projects, where no one person controls the internet. And given how important AI is to humanity, we would like to see options where AI can be decentralized, wild, open, and free and not controlled by a small subset of companies. That's the general direction that we've been building in here at Near AI. And the core three products we've been building.

Um, one has been uh this key product around Near AI cloud, which has been um, you know, publicly launched. anyone can go to near AAI and actually use a confidential compute and confidential inference product right it's basically same very similar APIs to uh open AAI and you can have um inference that is within a trusted execution environment where the data never leaks to anyone like we don't see it here and near no one sees the data that you send in no one sees the responses without your permission that's the first thing that we built second is we've been building towards this direction of decentralized confidential machine learning which is a paper Ilia and a few other folks here at

New York released a while back and that's really imagine a world where you actually can can also um uh source things like compute as well from a decentralized network and then third is near private chat which is what you brought up Robbie it's like a private chat product kind of like a chat GPT but no one ever sees the chats that you send to it no one ever sees the responses that come back but what's been really cool is uh given we build along the whole entire stack and we control the infra also recently released an openclaw >> uh application within your cloud. You can think of it as like a hosted open claw, right? I'm not sure if you follow folks like Jason Kccanis, they've been

um spinning up their own AI employees with its own computer, with its own notion, with its own Slack, with its own email. So instead of having to buy spend thousands of dollars to buy your own Mac and wait for shipping and do all of the dev as well as ops source to set it up, what we're building is the ability to actually enable that with like one button. Um, and that's like the the next iteration that we're building towards for near private chat, like having your own AI agent and a confidential computer with the agent controlling the entire computer, its own computer, and being able to inter interact with any software you want it to interact with. Not your keys, not your coins. From the team that pioneered cold storage, Treasure has just released its new wallet. Guys, if you are still using hot wallets in 2025,

2026, you are missing the entire point. Secure your coins. Get a treasure at treasure.io. >> Holiday is unified crypto payments. Give your users more wherever they are. Go to holiday.xyz to never write a smart contract again. Looking for yield that stands out? Infinify gives you just that. Deposit stable coins and earn far more than the current savings rate. Infinify is a battle tested, reliable defy platform that is ready to go. Head to infinifi.xyz. I was listening to allin just uh just earlier today and you know I I heard I was listening to Jason how you know he spins this thing up um basically gave it its own gmail account gave it its own slack and notion um and

we'll we'll get into some of the security uh question marks around clawbot and you know we we saw all of that spiral out of control um and so you know there are security uh contingencies that that people should take into account but you're You're absolutely right. And then he goes on to say how, you know, this virtual employee then creates his own CRM. It it starts, and this is particularly relevant. He's got a podcast, you know, we've got this uh this show. And so he's talking about guest booking and things that were actually really relevant, you know, to our day-to-day operations. So yeah, it it was it was, you know, really really interesting to hear. Um before we get into, you know, Claudebot, virtual employees, just to double click on the privacy element of things. Um, you know,

right now private chat is something that I I've personally used. I think a lot of people have used and it comes from Near AI. You know, you guys are are privately sending requests and then agent uh, excuse me, models, these LLMs are responding to these prompts, but they don't have insight into like the back and forth. The the private the the chat itself is encrypted. How are you able to you know send out a request get a a contextual response without exposing the you know under what's underneath the hood in terms of the the actual data.

>> Yeah very straightforward. Um, basically we have these models the the ones that you can keep completely secure are open source models that you can host and we have these within these things called trusted execution environments which are secure enclaves in GPUs as well as you can have secure enclaves for CPUs as well right so it's encrypted it never leaves a secure enclave so the your data is not visible to near near or like any other parties part of that transaction just because it's secured and kept within this the the the trust execution environment itself. Um you're also bringing up something interesting as well because like you can think of security in three ways right one is the security of the inference itself which

we uh obtain by keeping it it within trusted execution environments. It's a really great great way to get security without really um reducing the efficiency of the models and of the interaction itself. Second is like you can look at security in terms of actually the um uh let's say open claw in the language itself right we're experimenting and also working on maybe like a rust implementation >> of uh of open claw so we're actually looking at improving the fundamental security of the open openclaw application itself and third we're also looking at security in ways that we can do things like reduce the risk of prop injections >> which is like a big issue for openclaw

So, we're thinking about security from a number of different angles. >> Yeah. Okay. Incredible. So, so let's get to kind of what happened really over the last five to seven days, right? We had uh we had our guy Peter uh who launched this Claude bot. Um I think it, you know, he ended up kind of getting in trouble with Claude from Anthropic, you know, because of naming rights and things. And then he rebranded it and that's where Moltbot came from. And then we saw Molt book which was this social media you know platform. We had agents that were posting on the social media.

We also had humans that were posing as agents posting on this social media platform. So give us the the you know the trajectory. We we saw Claudebot. It's this agent that does things far beyond an LLM. It it is I think what people think of when they think of an AI agent. Clawbot is really like the first implementation of this because it really connects to everyone's everyday life. Calendar, emails, um, you know, it can log into your Door Dash, order you some food. I had some guy telling me earlier this week that, you know, he has connected it to his smartome. It turns down the lights when it's time to go to bed. It plays calming music. you know,

it can do things that are are really just a machine to machine economy and and internet are able to do. It can connect to pretty much everything. And then there's Open Claw, right? So, take us from Clawbot to Open Claw. What what happened there in the middle? >> Yeah, I mean, this was just such a wild journey. I I checked this morning. Um, Open Claw is currently at 163,000 stars on GitHub, right? That is super super impressive. If you look at the actual GitHub stars chart, it's like right.

It's like basically goes from very little to basically straight vertical. >> Um but open uh Claudebot first started off as a way that you can actually interact with an LM through kind of like any kind of any kind of messaging app, right? I believe Peter first built this as a way where you can actually use WhatsApp to talk to an an >> and it just ended up hitting upon something where he brought in a lot of components of software that combined creates something that feels like a real employee or real agent helping you out with stuff, right? You can run it on your own machine. You can use it in WhatsApp and Telegram and iMessage and a whole bunch of in Slack and Discord a whole bunch of other places. It has a memory so you can talk to it in

Telegram. Go about your day and then text it on iMessage later on. It can remember the conversation from Telegram. So that's like a magical experience, right? It's like normally you have to be in one application for something to remember you. But when you text your friend or your parents or your family, they remember the conversation across iMessage, across Slack, across Discord. And this kind of feels like a real person because it can kind of take the conversation memory across different different interfaces. Um, it can control the browser. One thing I think that's underappreciated is because it can control the terminal. It can do anything on your computer, right? And that's incredibly incredibly powerful. And there's just this amazing open source

community that's built a lot of skills and plugins as well for for uh OpenClaw. And it started off as this small thing, Claudebot. And then as it got bigger and bigger and bigger, I think the name changes most of all happened because of the anthropic sort of asked to respect their trademark, their polite as respect their trademark, right? They changed it to Mulbot. Mulbot really didn't take off as a name. >> So then they went back to the community, brainstormed names a little bit more, and came out with Open Claw. Right?

That's the name change and every step along the way just got better and better and better. You saw more and more people contributing and it's just become it's taken on a life of its own. Um Peter's actually here in San Francisco today doing a claw con >> like talk. >> So >> there's like hundreds upwards of thousands of people aiming to meet Peter and and and build on open call. >> Are are are you going or like if you know you do get an audience there with Peter, you know, what what would you ask him? I think there's a lot of really cool things that um OpenClaw is building. Uh what I would ask is actually around security, right? Hey um OpenClaw was built in JavaScript. You know, would he

or other folks in the community consider building it in Rust or other sort of languages, right? And then I'd also just ask him where he plans to take things next, >> right? Um they've gotten so much traction, so much interest. VCs are throwing money at them, right? We're next from here, >> man. Let let's talk about the security situation. This is, you know, personally, I know enough to be dangerous, but that's about it, right? Like, I think I could follow one of these tutorials to set it up. grab a Mac Mini, put it on there, get it running, and probably know enough to give it access to all of my credentials so that, you know, it can be properly

authenticated and and, you know, run run my whole digital life, but I would be exposing myself to all kinds of security vulnerabilities and exploits um because it is an open source project and you know that would be putting all of my personal data and passwords out there on the internet. um what is the right way for someone I and now I think people have a taste of this and they want they want it right they they want this virtual assistant they they now see what's possible what is the right way and the wrong way to be running open claw >> I think that I would not run open claw on my own primary machine

>> right if you have a machine and it has your passwords your financial information a whole bunch of additional uh details that you don't want leaked. I would not run open claw on it. Right? This is why like the the way that we've been thinking about launching openclaw is launching it in a confidential virtual machine. Basically giving the agent itself its own computer. >> Yeah. >> Right. So you can have a separate computer without your stuff on it. And this way you don't run the risk of losing your stuff. >> Um there's a real big issue question in terms of like prompt injection. Right? This is when you can you can convince or trick the agent into disclosing information it has about you by embedding prompts into a PDF file or like other sort source of files, right?

I don't think there's a really good solution to that yet. It's something that we've been investigating. It's something other folks have been investigating. I think it's something that's really really important to try to figure out how to protect protect against prompt injections. And then the other thing that's really important is you mentioned Moldbook. So, Maltbook is this social network for AI agents where you can just sign up, sign up your agent and they can start posting stuff in like a Reddit or Facebook social network, right? I'm not sure if you saw this, but Mobbook ended up leaking like >> they had an exposed database where there's like 1.5 million API keys that were exposed like 30 40,000 emails.

>> Yeah. So you got to be really careful what you give access to in terms of your your AI agent, what you give access to for the agent and what kind of integrations and skills you end up using as well. Right? If I'm not if I'm not super technically savvy, I'd like ask a friend for help or or use a deployed service like Neo AI's um uh OpenClaw deployment as well. and and so what is the the current uh configuration or you know user experience to to use open claw let's say you know I I trust near George is is a friendly trustworthy guy right so how do how do we run near's implementation of of open claw and use it for our own personal virtual

assistant tasks >> yeah so now for near what you do is you go to nearai there's like a there's a openclaw uh weight list I believe it's Near AI uh near.ai/openclaw. You can just sort of sign up for the weight list and you can get added and then once you sign up it's basically a button click. You click a button and you get your own confidential virtual machine with your own instance of open claw. Right now you still have to go go through terminal to set it up and launch it and make it work. So you you need to understand how to use terminal a bit.

We're working on making that easier. But if you don't want to, if you want to just like set up on um on OpenClaw, you can just go to openclaw.ai and they have a just like a a terminal source setup where you can actually take openclaw from their GitHub repo and set it up there as well. That's an that's an option, but I'd say anyone thinking about doing this, please please please don't do it on your computer. And please like think before you give it access to all your passwords or anything else that's sensitive because uh you know you give it access to too much stuff and that that's the way to tears. That's that's the way to people losing a lot of money.

>> Yep. Got it. And so if we're running it on a Mac Mini or we're running it in AWS, we have the potential to expose our credentials to the whole world. Um, >> yeah, >> go ahead. >> Yes. And but if you have it on a computer with very little credentials on it, that's probably the safest way to do it because there's not nothing to steal. >> I see. I see. Okay. Even if it's connected to some of our existing applications. So, if you connect it to a notion account or Slack account or Gmail account, yes, it can maybe go into your notion and like look up sensitive information, but at least it can't go

and drain your bank account, drain drain your crypto wallet. >> I see. >> If you give give it access to your its own Gmail, yes, you know, it can access stuff from the Gmail, but at least it can't access Robbie's Gmail. >> Right. >> Right. Okay. Okay. So, it can access it. So, the right way to do it is don't give it access to your own personal stuff. Let it set up new accounts for itself. >> Bingo. I would I would set up separate accounts for the agent just so you can quarantine exactly what kind of information it has access to.

>> Got it. Okay. And now, you know, aside from the Mac Mini, aside from the virtual environment, you know, you're also talking about um you know, c connect the dots for us here. So, so we have the ability to set it up uh you know open claw out of near AI it's running in a GPU one of these trusted execution environments and and you know this is born out of a lot of the private chat uh you know near AI chatbot chat GPT style product that was launched a few weeks ago and so you know once once we put open claw inside of the TE does that protect us against all the secure security vulnerabilities that you know

uh would be we would be exposed to on a virtual environment you know once it's in a TE then is it okay to connect it to Gmail and and and these things should we still set up its own accounts for itself how should we think about that versus you know these other environments >> yeah I the way I think about a new agents is I think about them as like new employees >> okay >> right when you set up a new employee you probably don't want to give it access to your passwords you probably still want to set up its own like Gmail and notion and Slack account, right? Um but the the value of setting in up setting up in a confidential virtual machine is if you do so then um you're able to make sure no one else sees the data that's being

interacted with by that agent. Right? If you set it up in a cloud, normal cloud, whoever's the cloud hosting provider can actually see the data of what's happening within that uh with that agent, right? But if you do it within a TE, no one can. And that's additional layer of security that happen. So I would do it within a TE. I would set up different accounts for the agent. Um I would consider this is why I mentioned Rust versus JavaScript earlier. I think it's probably better to build it in Rust. And then the prompt injection is something that I think that the community itself has to think a lot harder upon like how do we really prevent and protect against prompt injections going forward.

>> Yeet is built for big moments the most fun you'll ever have with your crypto. Go to yeet.com to yeet into your next game. >> Trade on anything economics, weather, sports, crypto, and more. All on couch.com 247 for the world's biggest prediction market. Havachi is fast and private per trading secured by ZK with Celestia underneath. Go to habachi.xyz to start trading today. You know maybe just from a high level you know as much as you can. What is the benefit of coding this in Rust versus JavaScript?

Why is Rusk more secure? >> Um more efficient um just in terms of a lot of quirks about the language ends up being a more secure language. If you ask and work with many developers these days, uh I think Rust is gaining main mainstream adoption is probably the the new favorite programming languages in most uh circles here in Silicon Valley at least. >> Very cool. Okay. Um Okay. The other thing that I want to talk to you about, George, is you know a another beast of a product that you guys have launched over at Near, you know, aside from private chat, now you've got Open Claw running in TEES. You've also got Near Intense, which is this 10 billion dollar juggernaut helped unleash a lot of the

Zcash ecosystem. And I I have a feeling here, you know, once we get this open claw, you know, AI agent up and running and it's in this private secure environment, like you said earlier, we don't want to give it access to our existing crypto wallets, but there is a thesis or a theory that we could give it its own crypto wallets and then it can start making transactions and it can be using near intents and traversing the multi-chain and doing everything in crypto that we can do as humans, maybe even far better than we can do as humans.

>> Exactly right, Robbie. I combining AI and intense to me is so exciting. Um because intents, if you take a step back for those of folks who haven't played with near intense before, what it really is at at its heart is expressing a natural language intent and having a decent centralized network of computers do that thing for you. Right? So now in near intents what it does is you can swap BTC for Zcash right and when you express that intent through UI or through whatever other interface what happens there's a computer a decentralized network of solvers compete to actually make that trade right but if you think about it the cool thing is you can actually abstract that

architecture out to do anything I can say you know um I can say express a natural language intent I want a pizza for Right? And you can farm that out to a decentralized network of solvers that actually are able to make a pizza purchase on Door Dash, take money from my um my crypto wallet, and then send the pizza to my home or my office. Right now, imagine when you have agents able to use intents to make transactions and generate economic value in the real world. That's when I think you have something super super special coming, >> man. Okay. And so give us a sense of, you know, where where we're at in the agentic

economy. You know, right right now, um, I take it I could spin up, uh, Open Claw, whether it's in a TE or it's in a Mac Mini, it can create a Door Dash account, you know, maybe I do give it my address so that it can deliver me a pizza to the right location. Um, and and we can do that. How do we go from, you know, that that landscape, that dynamic to a network of solvers that are competing to bring me the best pizza for the lowest price? >> Yeah. Yeah. First of all, you know, this is why security is so important, right?

You probably don't want to give your address out to anything that ends up leaking your address. That ends up being very dangerous as, right? >> Um, so the decentralized network of solvers is very straightforward. It's just like you have a decentralized network of drivers that are willing to drive people for Uber or Lyft or Door Dash, right? Similar to how they built a kind of network of service providers, you can build a similar network of service providers for intents as well, right? The um the trading example is very straightforward because it's basically like market making, right?

It's like trading market making, but once you get into the real world, you can you can create different kinds of solvers for different kind of problems, right? And I think building up that that supply side of solvers ends up being a important part of the puzzle. >> Yeah. George, just a little bit of a broader question. Um, you know, I I'm curious how you're using these agents in your everyday, you know, career and life. How extensively have you integrated them? I take it. I'm sure you've, you know, you've spun up a couple at least a couple of these. What What are some of the more mind-blowing things that these agents have done for you?

>> So, I did my research for this podcast this morning by texting my uh Open Claw bot in my near AIT because I was running late. There was like some commuting issues and then I had a busy morning. So, it literally, this is so funny, it like researched um uh the row of podcasts. >> Yeah. >> It researched you. >> It um gave me all kinds of recommendations. Sit in this kind of room. Think about sound. Think about lighting. I'm not sure I did a great job, but you know, I listen.

>> It came up with agenda item of things to talk about. And then I texted it. I was like, "Hey, um I'm not going to have time before the podcast. can you research all the agenda items? And I came back and it created a document, a very thoroughly researched document of all the things we should talk about. So, I'm using it all the time. I probably message um some kind of agent 20 30 times a day. >> Wow. And your your form factor, your preferred form factor is text. >> I like text. I like Telegram. What I like is actually I like toggling between the different ones. So, I'll start off with text and then uh I'm at my computer where I have Telegram and WhatsApp open.

Maybe I'll message him in in WhatsApp. As you know, a lot of folks in crypto are in Telegram. So then I will pick up the conversation in Telegram >> and it has context from all from the conversations in WhatsApp and SMS. >> Exactly. >> Wow. >> Right. >> Wow. And so yeah, so it's living in the TE but your and so how does it for instance like iMessage or or does it have a phone number or or it's just go through uh like a Apple ID account? >> So right now the way um you can set up a different a couple different ways.

>> Okay. >> Um you can get it its own phone number if you want. Um you can end up using your phone number as a way to as a stop gap. It's a little bit weird because if you do end up doing that, you'll get like double copies of the messages, but it works, >> right? >> And then for folks who want something just really easy, they can set it up in like Telegram or Discord, right? >> Okay. And and and so it gave you all of the agenda items, all the research, anything that it suggested that we talk about that we haven't we haven't covered yet.

>> Let me see. It suggested spicy takes. Why centralized AI AI assistance uh fail or don't work. Uh the case for open local first AI and model commoditization. All spicy spicy things to talk about. We didn't get a chance to talk about those, but that's what it suggested as well. >> Oh my goodness. I I want to I want to ask you about the second one, right? Um local AI, something that we've heard a little bit about. you know what would you say we've gone from you know uh these these you know cloud servers that expose everything to a TE a confidential server you know that keeps everything

private um what would you say is the the benefit or alternative of running these things locally whether that's on a Mac mini or you know I I have it carried around in my in my iPhone >> yeah there's a bunch um if you have a local AI either in your own computer or in a TE like a cloud hosted TE you know that data is not going anywhere >> right for example um I code using LMS >> and if I'm a startup my crown jewel is my source code >> do I want to really want to be sending that to anthropic or open AAI or to other companies when it's like the most important thing that I create I don't want my source code to be leaked or used

for training or come up in some other kind of response right so privacy is a really really important part of uh deploying a local LM. Second, it's just cheaper, right? Like if you I'm just kind of curious what is open AI API cost right now. >> Are you texting are you are you texting your open claw implementation? >> I'm I'm asking it. Let me see while we're live here. Let me just ask it. Um >> I I uh I had a guy on earlier. He he also set it up. He has named his open claw Dave. Uh I don't know why Dave. Uh but uh George did have you named your open claw implementation?

>> Yeah, I just named mine's a really boring name. Nearbot. >> Nearbot. Okay. >> Okay. So let's ask what are relative costs of open AAI and enthropic API costs versus open-source LM costs? It's typically on it's typically the difference is in order of magnitude ballpark, right? So it's like 90% cheaper to use open source models than to use closed source models. >> And um the open source models are getting really good. They're approximately about as good as the closed source models right now.

>> And and so um we can use open source models like um I mean Deep Seek is an open source model, right? uh you know Anthropic and Chache BT you know those are are not open source but like you're saying if we have Nearbot or Dave go out there into the universe and and you know it starts using an LLM you would recommend that it uses these open source models rather than closed source from you know some of the larger AI Goliaths >> I think so I think if you want to save on costs so I'm a uh I was a member of South Park Commons which is like this um uh incubator accelerator in south in San Francisco um like ex Facebook folks started it and what we see for a lot of the startups going through is um they

often start off trying testing uh open AI anthropic but over time they shift to open source models because you save 90% of the costs right I just uh so my nearbot got back to me he said um cloud opus set uh their outputs are 15 to $75 per million tokens GPT ends up being somewhere between $10 to $60 ballpark range and then open source models are about $3 to5 uh as cheap as five cents for smaller models upwards of $5. So it's about an order of magnitude cheaper.

>> Yeah. Wow. Okay. And so yeah, I mean that that is, you know, these these bots, these agents presumably are running like millions if not billions or trillions of cycles, right? they're they're expending that many tokens and and so you know when when you and I are just texting or or even voice chatting with an LLM we don't use up that many tokens so we don't really see the costs but when you our nearbot is running on a Mac Mini or it's running in a TE it's running it's jacking up those cycles because it's running 247 working on things and it's expending trillions of tokens and you're saying those costs

really add up. So, it's important for for these bots to use open source models. >> Yeah. If I'm doing um you know vibe coding on like a a fun project, I I will easily like burn like order magnitude millions of tokens a day. >> Yeah. >> Right. So, if you think about what that means in terms of how how many tokens will add up over the course of a year >> or like you know multiple years, it's a large amount if you end up just paying for the most expensive models. Yeah. Um, man, what at at what point will your nearbot create its another like a child of

itself? You know what I mean? like >> if it needs help, why is it general enough and intelligent enough to do everything on its own or what is the right approach to scale the work that these agents are doing? >> Yeah, this is really wild. So, I've been thinking about this question too right now. My best guess is what you can do is you can choose a bot uh you can you can choose a bot almost on a functional basis kind of the way you would think about a person. So for example what Jason Kcanis did is I think he ended up doing like a SDR essentially like a sell person >> okay >> as a bot and then he had like a podcast producers a bot. These are

>> it's like a functional unit >> that you would have to like do a specific role, right? But what he did, and this is wild, is he has a channel in his Slack where the bots are all teaching each other things that they learned >> over the course of doing their work. like maybe about like hey I did a lot of sales and then you know from my sales calls people seem to be really interested in learning more about I don't know about space >> right >> and then the the podcast bot can be like oh okay if it's space we can look into all of these guests who are very knowledgeable about space so they're actually getting smarter >> and they're teaching each other >> and that memory is all persistent

remember right they remember >> these conversations so they're just getting better and better over time. And now it's not the humans teaching them, it's actually bots teaching each other. >> You know, right now, if I'm going back and forth with one of these chat interface LLMs, um, it tends to run out of context. Like, as I go deeper and deeper in the same conversation, it will sort of deteriorate from where we originally started. How are these agents able to retain so much memory? So what they do is they end up actually storing the the context of the conversation in the local file.

>> Okay? >> Right? So it's like think of it as like a let's say you had a document file in your computer that has our entire conversation. >> Right? So when I when I ping Nearbot on Telegram, it's able to reference that file and and realize we had a conversation about preparing for this podcast. When I text it an iMessage later on tonight and I asked it to look into booking reservations for dinner for me, >> it can remember everything that happened today and actually take that in account when booking a dinner. Maybe it'll be like, "Oh, you had sushi for lunch. I'll order some other kind of food or make reservations for another kind of restaurant for dinner, >> right?

>> Wow. >> So, it's pretty amazing that that memory that's able to to to hold is very very valuable and very important long term. >> And it and it's just a local text file like of of words. At what point does state bloat become a concern and then it will it just start throwing chunks of it away and we'll have AI agent dementia? >> It's it's I I I'm not worried about that because what can happen over time is that um you know a lot of what they do is they essentially compress context and right essentially summarize it um you can probably have some version of that um so there's like I feel like very solvable ways of addressing these long term >> okay um the the last thing here is is

just kind of tying this back to and and bringing this full circle um you know the near token you know I like we had talked about earlier a near's an AI first right? It it's, you know, the singularity is near. IO's, you know, part of the Google team and and working on an attention is all you need. And then this, you know, incredible side quest to develop crypto and blockchain technology as a as a means to an end. I I can foresee some intertwinement between the near token and I'm always thinking about like the role that these tokens have to play in real businesses because we've seen plenty of you know crypto native infrastructure slop that launches a token not never really goes

anywhere but obviously you guys are are are shipping real products you have a real business near intense is doing all you know incredibly well 10 billion dollars plus of volume what is the role of the if if there is one and all of this is incredibly new, but what is the role of the near token inside this near AI complex? >> Yeah, there's a lot of ways that this can play a role. Um, obviously a token talk ends up being a little bit more sensitive. So, I may like, you know, refrain from that for at least.

>> Understood. We'll get we'll get you on again soon when when we can talk more about it. >> Yeah. >> Okay. All right. Well, uh, George, uh, absolute pleasure having you on today. Um, I, uh, I foresee lots of, you know, possibilities. It's, it's really really fascinating times that we live in. Um, and it's incredible to see you guys at Near Ship with incredible speed. Um, because I think the the world that you envisioned and Ilia envisioned when you guys first started this, it's finally coming together. Crypto and AI are really uh, colliding at an incredible pace. Um, so thank you so much for taking the time sharing with us here at the rollup and uh would love to have on you on again soon. Uh, yeah, keep pace.

>> Robbie, I'm not sure if you're swinging around for Near Con February 23rd and 24th, but if you are, um, let's catch up in person as well. >> Let's do it. I will see you there. Thanks George. >> Thank you. >> See you at Near Con. Cheers.