Tarun Chitra: How To Optimize AI (And My Crypto Thoughts)

Transcript

So I think there is clearly something open source AI that will stick. Now does it need crypto or not? That's like where I'm like what are you bullish on broadly? You know category idea sector in the markets uh or outside and then what are you bearish or fearful about? Where do you think the Google search versus open AI prompt on a consumer basis stacks up? Welcome back to the rollup. We are your hosts. I'm Andy. I'm Robbie and this is your home to learn from the biggest guests, the brightest minds at the tip of innovation. Our goal is to help spread the collective knowledge of everyone in our community. We are breaking crypto out of its echo chamber and taking it to the real world. Sit

back, relax, and enjoy a quick word from some of our awesome partners who make this show possible. Crypto UX broken AI can't trust it. Let Magic Newton set you free. It puts your onchain transactions on autopilot. Does all the work for you 24/7 so you can go do what you want. The vision of Magic Labs, the creators of embedded wallets used and backed by the best. See for yourself. Simpler crypto UX, verifiable AI agents. Newton.xyz. [Music] What's up, man? Serena, what's up, man?

Good to see you. We are not talking about crypto today. Great. What are you talking about? I'm talking about your tweet here that keeps coming up that I'm really enjoying. Which one? It is your daily drivers and how they keep changing it. I gotta find the most recent one just to get a Yeah, I haven't updated in a couple of weeks. Yeah, I please update more often because I I've slowly switched off of Quad as a result of seeing your daily drivers. Let's get a a live update if Yeah, I'll give you a live Yeah. Yeah, live update. So, wait, hold on. About a month ago, daily driver 03. Is that still the daily driver?

Yeah, for the most part I although the problem is actually I keep hitting the limit my usage limit. So I would say it's a daily driver until the usage limit then I just go to Gemini. Okay analysis Gemini 2.5 Pro kind of coming back to in between 03 and Gemini um just seeing differences in like the in like Yeah. So like okay so for for 03 it's like very good at abstract math stuff then like you know there's like some paper that maybe I read and I'm like I kind of remember the conclusion and I'm like hey can you reproduce the argument and it will be able to do them but if you ask it to do very precise things like write me code to do x it will make a lot of errors and

I think the reason it makes a lot of errors is actually a kind of funny reason. So like it uses a lot of like shortcut handwavy things where it's like hey maybe in some piece of code they called some variable like Andycoin. Yeah. And it will be like oh Andycoin well the ticker was Acoin so I'm going to call everything Acoin and then it like forgets that it renamed something and then it'll go back to using Andycoins. And and I think Gemini and Claude are much better about this. They're like very consistent. They don't forget that they're so stuff like that. So it's like yeah if it's something where I I can figure I'm the one who's figuring out whether it's supposed to be the same or if it's something where it's like it's a piece of code and this software won't run because they like

forgot they renamed the variable. Those are the difference but but probably those two I think and that happens after like multiple prompts 10 prompts later it's like it's forgotten what it Yeah. And Gemini is really good at like you give it something and it runs for a long time isn't the best. Collage is actually very good at tiny tasks. Anything long, it starts messing up, but it it's not because it has no memory. It seems like it's it remembers things well, but it just like has a computation. It's like it's built around small programming task and you ask it some complicated math thing and it gets lost. I I like Claude though for the the way that the human feel is to it as as typing to it. It feels like you remember

what what her name is. a woman who's like their uh head of UX as this like philosophy professor. It gives me like this like very like so all right research you've got o3 and 2.5 pro. Yeah same still that combination that combo is like the best for so for any sort of like mathematical research deep level thinking. Yeah for like programming claude and gemini not really 03 cuz it makes like all these little dumb mistakes. Yeah. It's like it's kind of like the smartass in class where it's like the person who like never studies for the tests and still gets an A but like the teacher hates them. Whereas like Claude is like the goody two shoes who's not really that smart but like

memorizes and tries to do every extra credit possible. Yeah. Yeah. Yeah. And then Gemini is actually the smart one but then like sometimes loses confidence in itself. That's like if I were to anthropomorphize them. All right. Uh, your search engine 03. 03. Yeah. Oh, 03. And that's just for like searching for So rather than using Google, just pop into 03. Okay. Let me let me do let's look at like the other thing is I think the chat GBT app is actually good. Yeah. Whereas like the Gemini app and DeepSec app and stuff are kind of shitty. The mobile experience like they'll get stuck or like they clearly didn't spend time trying to make them like consumer products. Yeah. Yeah.

And so like I end up asking like you know for instance I asked something about like who what who are the sets of wholesale lenders in uh in traditional finance because I've been thinking about like these people who are syndicating RWAs are any of these people doing it yet? Yeah you know as like a is it really here yet? And so like you know I got I got a good like list of list of like the top 10 mortgage lended pennym only w Just so we're still early. Yeah. Yeah, we're still early. Exactly. But if you search that on Google, it would be it would be hell. Or the other thing I realized is the Google AI assistant, you know, when you Yeah. Yeah. That stuff is

like cached results. It's not actually running a live model and those results are often times stale. So like some news event happens, it will not get it. Whereas I think you know open and Google in particular cloud a little bit although it messes up the web search are very good at like going and like searching for going and searching online for like oh like did something new happen. Yeah. And I think that's where they they've improved so much it's like impossible to not use both of those. Yep. Um you've got uh coding for new code 2.5 crow. I think I think I think that that one is actually Cloud 4. Cloud 4 is a competitor there because you got uh Cloud 4 for

debugging. I think Cloud Cloud 4 has gotten better at at new code. I actually often have like I have some idea I write some pseudo code. I give it to 03. I'm like what's missing? Like how would you write this? Describe it to another language model. Yeah. Yeah. And then I take what it describes and then give it to Gemini and and Claude because like often times 03 is good at like planning but then it like again this is why I said it's like the smartass who like doesn't do any of the like homework but just comes in for the final. Yeah. Yeah. Like it clearly understands what it's doing. It just like makes no effort to do the basic boring thing. It makes careless mistakes. Yeah. Yeah. So So tell us more about that. So, do you like are you taking one like uh answer from a

one LLM and then telling it like make this ready for a different one and doing that a lot to like I don't know like double check your worth or like a it's sometimes double-checking um b it's actually oftentimes because a yes one of them is better at like being like one of them is being like claude is very good at like finding little tiny errors and like working until the piece of code works. 03 will be like, "Ah, it's so easy. Just do this." And then it doesn't run and you're like, "It doesn't run." And they like, "Yeah." Yeah. Ah, okay. I'll change one thing. And you know what I mean? It's like very lazy. Smart ass.

Yeah. It's like they would go on those. That's hilarious. Uh, God, I like your little children here, J. You treat these like people. It's like, okay, the other thing, another tweet I wrote about this, which is like actually I think all LM realize is everyone is just going to learn to become a manager cuz like this is like managing a team of people. Yeah. Like I know which task one is good at. I know which one I should be like distributing. I I have to like tell them, oh like you suck and this other model produce a better thing. Why can't you be better and then it'll sometimes give me a better answer. Yeah. Yeah. And it's kind of crazy because like you're just kind of a manager. You talk derogatory to the to the LLM. Sometimes they actually perform better. Yeah.

Although some of them like Claude is is is the sensitive one and you say you [ __ ] up and it'll suddenly get all defensive and I'm like all right. Do you but and so you're already starting to use another LLM as the manager of of one. Well, you're starting to kind of pass work between them, but you we don't have the interoperability amongst these models to the extent where they can manage one. Yes. Although you kind of do. So like if you use cursor, cursor actually basically every time you write a message keeps track of all the context embedded for all the models it supports.

So it's like you ask a question and it tries to be able to send it to 03 claude whatever so that you can move between the models more seamlessly. Now not perfect. Yeah. But it actually gets you like 80% of the way there. Yeah. Yeah. I think that's why you're seeing this like Silicon Valley VC Twitter trend of like everyone talking about who's going to own memory. And when they talk about memory they mean exactly this problem of like how do you build this interop? I'm kind of of the mindset that the application owns the interop. It's a little bit like DeFi where like the application owns the order flow. I kind of feel like the application owns the order flow in the context that the user is sending. Yes. Which especially if if you're using multiple models, it feels

like each individual model's individual value is like decreasing over time. But the value of like whoever owns the context that the user is propagating through as you're chatting with them, that is the thing that's most valuable. And like the cursors of the world own that not the open AIS, right? So like Yeah. So so so if if there could be a tool, let's say cursor that would have access to all all of your usage across different LLMs, that tool would be the most would be I think in some ways that's more valuable now weird exception because their consumer app which is the the mobile app and front end is so much more popular in terms of users than every other LLM. Yeah. that they own a bunch of contexts, but they own a bunch

of contexts that's actually cons like retail consumer usage whereas like Anthropic owns like US government usage and like Google owns actually a lot of programmer usage now. Yeah. Right. Google owns a lot of search which also could provide a lot of context but maybe not so much as I I don't where do you think the Google search versus open AI prompt on a consumer basis stacks up? Yeah, it's a really good question. Gemini, have you guys ever used the interface, the web interface for Gemini that like is Oh, yeah. Yeah. It's like not as not like it's not a thing about like the answer you're getting that's not as nice. It's like the UX sucks.

It's slow sometimes like or like Okay, this happens in almost every language interface where like one out of every 50 uh prompts I give, I have to go retry because it got stuck or like it started sampling tokens and then it stuck. Yeah. Yeah. And like obviously that is like the biggest UX hurdle I think for a lot of people is that idea that oh it stopped. Oh, it must not be working. I'm not going to keep using it. Yeah. Yeah. You jam the stop button and then sometimes it just kind of like sh Yeah. Yeah. Just doesn't stop or whatever. Google's is actually the worst UX for that. Like they actually sometimes it has the fastest token production on Gemini Flash, but then like the the number of times it just like not doesn't respond is like way higher. Yeah. So that's why I prefer the API usage or

cursor usage because all these other apps actually do a lot of like retry logic or like if you're writing your own code that's using open API, you can add your own retry logic. So I do think like this is where I'm saying I actually the apps that own the order flow will like be able to differentiate and like own a lot which is also I I think one of the reasons I'm pretty nebulous on most crypto AI things right now. And like was extreme extremely bearish at dinner last night. It was like dude 2 or three years like this [ __ ] is just AI is not good enough. Yeah. And also just like the security stuff is is actually much worse than I think people will give it credit for. Like Yeah. I I think there are really cool examples like Frasa um where

like people who are like trying to build L1 security via these crypto contests which I think is like that's an awesome way. Yeah. But they're nowhere near ready. like I you someone is going to lose money and hopefully not a lot but it's like it's going to be non-private. Have you invested in any like crypto AI things? Yeah. So and we invested in noose um and exo and basically I think edge usage of AI is going to be really important. So in exo case like I already see this myself like I bought a new Mac and I run 14 billion 14 billion parameter quantized models on my local machine now and like I was not doing I could not do that before like I didn't I think people underrate the Apple silicon

for personal use yes stuff I has gotten a lot better and so I think people who are using that for like edge compute I think there's going to be a market for that where like they don't they need to run local models and they're unwilling to outsource, but they own a bunch of computers locally and they can use this stuff. So, that's one place where I actually crypto is useful. Like the value capture for that is not clear, but at least it's like I I I can see it myself. Yeah. Um and the other first is like I I do really think News and Prime Intellect I think a lot of the RL stuff is more conducive to crypto now. Are they gonna make and again there's no real obvious

value. Sorry. Sorry. The reinforcement learning uh training. So like what Deepsee does what all of the the Yeah, that was the last one that we had for you with the fine tuning modern the modern. Yeah. Yeah. So so for open source models pretty much deepsee quen 3. I spent a bunch of time trying to fine-tune 54 which is the Microsoft model. Yeah. Uh did not work. Also it was just like just a little too big for for my local GPU. So, uh, main point is like I kind of think the there is a market for these fine-tuned models.

I just don't know if like the cost of fine-tuning is ever going to be, you know, the amount you make from making someone a custom model is ever going to be more than how much you paid to fine-tune it. And like that is the part where the economics are not clear. But Noose and and uh and Prime Intellect have been kind of focusing on two different directions with news kind of going the we're making an RL SDK so it's easy for you to fine-tune distributed and then Prime Intellect being like rent an H100 for a dollar which is I mean I I use that all the time. Yeah. Yeah. Introducing Summerfi, one of DeFi's oldest and most battle tested platforms.

Summerfy automates your exposure to DeFi's highest quality yield so you can sit back and enjoy this summer. Deposit stable coins at USDC or USDT and let Summerfy route your funds into optimized low-risk lending strategies. With over a 100 million in instant liquidity and support for over 34 protocols across Ethereum, Arbitum, and more, you always stay in control. Choose from a wide range of curated strategies reviewed by independent risk experts all in just a few clicks. So why work hard for your yield when you can be lazy? Start earning now at summer. The home of the lazy summer protocol. Innovating finance global institutional and onchain. Deep liquidity to drive capital efficiency.

Now maximize yields on institutionalgrade liquid staking with M. Unlock trillion dollars of capital via F Bitcoin. All of this secured by Mantle's treasury of billions in stables and ETH capital pools. This entire ecosystem is coordinated by a vast network of seasoned VCs that are aiming to create the next generation of banking and financial technology. This is Mantle. So that gives you like a like access to a a piece of hardware in in the cloud for a dollar and a dollar an hour. A dollar an hour and then you get like this maxed

out GPU that they can just ab you can just do whatever you want. And so that you can you can run fine-tuning, you can run these experiments and like that is I I was initially doing this because I was like I want to understand how the algorithms are working. I also want to understand the open source ecosystem because like I used to be a GPU programmer in 2015 2016. So I've wr I wrote a lot of CUDA code a long time ago and I'm I was like what has happened in the last 10 years? Yeah. Yeah. And it's gotten so much easier. It's like insane how much easier it is to go from zero to one. Yeah. So, I think there is clearly something open source AI that will stick. Now, does it need crypto or not?

That's like where I'm like maybe like I mean the thing we need is like the lazy man's onchain assistant or like like the lazy man's portfolio manager. It's like the type me a strategy that's going to make me a [ __ ] 30% with little risk over the next two years and like execute it for me on my behalf, rebalance for me, all that kind of stuff. I just think like we're not quite there yet, but and this is a tiny shell, but like we launched a rebalancing USDC project across chain. So like you put in USD and it'll it'll deposit on base arbitrum wherever that uh different yield is. And I think once the crypto side, especially in the vault side solidifies security-wise and like people are really confident that then I think the AI side

taking advantage of this type of stuff is easy. But I think right now the crypto side is like still not like it's not there for like the APIs and endpoints and the security model is not there for AR stuff yet. And so it is definitely a road right I I I think there will be a version of the world where crypto and AI interact more but I'm not I'm not like holding my breath thinking it'll happen tomorrow. I think you just have to be in both markets. Yeah. Yeah. Yeah. I'm curious about I mean the H100 stuff I I find pretty pretty fascinating. So like you would you would basically rent this thing for an hour. You would run some like super intense fine-tuning algorithm while you have this thing live

pull it down to your local machine and then you're you're good to go. Yeah. I'm I have to I someone will So one investment we made uh in 2022 which is ironically they started as a crypto company and then pivoted Yeah. with together uh AI which is like a they host models for you. So you have a custom fine-tuned model they'll make they're like Verscell for model host. Okay. So like you have some model where you fine-tuned it. They will host it and generate it and stand up endpoints and geo distribute it and do all like almost like a a node as a service provider. They're like a inference node as a service provider.

Okay. Um, and there's a ton of people who are doing this and that that market has grown. Like core weave, I guess, is the biggest, but there's like a ton of people who are in this market. And so it's a in that case they're posting it for you. So it's not local, but you can still fine-tune it yourself. Yes. So you fine-tune it and then they make it so that like say you have a website that has a chatbot and your chatbot is like fine-tuned on the rollup episodes, right? So that like people can ask questions about it. Yeah. Well, maybe you every week or every five weeks you have to re-update it with the new episodes data and you refine tune it.

Well, there's this whole workflow. It's almost like machine learning pipelines of like okay new data came in okay re-update the model okay redeploy the model so that the website is using the latest one and that whole continuous pipeline process you know in in in trading and in data science that has been always come under the umbrella like machine learning pipelines in AI the pipelines are kind of different um mainly because like some of the data as I retrain the model some of the data is actually all this RL stuff where I'm fine-tuning how I ask prompts. I'm saying I I ask sequences of prompts. And so there's a whole world of that stuff and there'll be developer tools and and things like that. Now, there was a while, especially in 2022, like pre

Luna, yeah, where there were a lot of people working in AI and they saw crypto go up like crazy and they're like, "Oh, you know what? Maybe we can do this stuff decentralized." And uh a lot of those companies were amazing investments, but because they didn't stay in crypto. Yeah. uh the the number of people who pivoted at that time like including coreweave actually was a very large and like those pe that cohort did really well. So that's sort of like how I've I' kind of been paying attention. Yeah. But like I I really do think there is there will be a world in which they're interacting.

We're just not there yet. So last thing for you, what are you bullish on broadly? you know category, idea, sector in the markets uh or outside and then what are you bearish or fearful about? I think an interesting thing is what I would say is and and I say this from the perspective of both an investor at robot and working at and running ga is we're kind of seeing I would call I would say this that we're at peak staking yield like I think staking yields have actually been kind of like going down quite a bit and if you've seen a bunch of the announcements like me and etc.

Yeah. about reducing inflation. I think we're kind of at the point that actually staking as a business model may have hit or we may have already passed peak revenue. So, I'm actually bearish staking revenue like businesses built around staking revenue because I and and this also sort of includes LSTs and protocols and validators and yeah, I think there will be consolidation in that market because as the rewards go down like there's there's not as much revenue for them. Um, but that's what I'm bearish. So, I'm bullish. All that capital has to go somewhere and the number one place is people want stable coin yield right now and every fintech wants to like get on chain right now.

Yep. I think that's where we're going to see increases and the main problem there is actually the demand side. Like where are you getting borrowers? And I think I'm very bullish on people who are bringing stablecoin demand of off-chain companies on you. Like taking companies that are borrowing from a bank and they're like, "Look, we'll lend to you in stable coin. We'll we'll do all the off-ramp whatever. We'll act as a company that holds the crypto risk for you and but we act as sort of centralized lender, but you can see what we're doing." I think you're going to see a lot of these like hybrid things come online because the rate spreads between onchain stable coin uh borrow rates and offchain is still big enough and all of this

capital that is leaving staking has to go somewhere somewhere. Yeah, that's so that's my bull and bear. They're very related. Yeah, they're very related. Yeah. Yeah. Tan, thanks for your time, man. I I appreciate it. I uh I wanted to go through that daily driver because I've just been tracking it. It was just I had you got the live real. No way. Seriously, we we'll have to update it real time. Yeah. Always a pleasure, man. Thank you. Thank you. See you. See you.