Cysic Founder: Why The Compute Market Is Getting Ready to Explode (Biggest In The World)

Transcript

All righty, guys. Welcome back to AI Supercle episode number 43, our premier weekly AI show powered by our friends at Near. Visit near.ai near.com to get started on your onchain journey. Start using their private inference tools and more. Fully quantum proof release happened recently. Today we have Leo Fan, the founder of Seisic, a full stack compute network turns GPUs, A6 and spare compute into uh a variety of different assets uh focusing on verifiable AI. Leo is a Cornell CIS PhD um and a deep researcher in both the ZK landscape as well as AI. Leo, good to have you back as always. The state of AI is uh quite rapidly shifting. Uh there is new

sectors being created, memory, storage, compute, inference. There's investing happening in the stock market that's going crazy. There's the national there's the national nationalization of AI between the the US and China. There's the compute bubble. There's the open source versus closed source. There's just a lot to cover. Leo, good to have you, man. Thanks for joining us. >> Hello. Hey. Hey. Hey. How are you? Like here a lot of like words like we we're here to throw them out. >> Yeah. Beautiful, Leo. Well, uh, great great to have you back. I guess just to start off, man, this is what people want to hear about. There has been a release of Kimmy K3 model as of late which is challenging the assumption that the

value and the pricing structure of of inference and compute uh both but primarily the actual intelligence that you're paying for with a given model is significantly overpriced by the labs out west in San Fran and the frontier labs are kind of in this race to uh figure out the best way to kind of rival what's happening out in China meaning that there is a lot of sim similar intelligence being produced at a significantly lower cost with these open- source models and it's kind of causing a lot of rhetoric around is this AI bubble over because we've you know gone through this token maxing phase and people are really just looking for answers. Leo, from your perspective, what is happening right now with the

pricing of these models and all this intelligence as it relates to the open source kind of push out of China and the frontier labs pushing back against these models and at the you know at the expense of their profit margins they would like to keep their dominance. >> Yeah. So to be honest as a as a like grad student like coming out of China like I honestly I don't think like right now the open source model like from China is like catching up with the clos model like from from the United States. Yeah. As you know like although like the Kimmy or like the deep sick they are like catching up but there are still like some gap there like probably like several months gap there like to to be catch up with with the latest model from

say like open AI or anthropic you can always like engineer this like model like the the say like Kimmy or like deepse to be like perform like very well in some like benchmarks but like if you do it like in like very broad way like to do it like in in in the different task like I don't think it's still like right now on the same part like for the like US like closed source source model. So which means like I don't think like right now the bubble is over and it's still like several months like we are like catching up like from the open source model. Yeah. So that's uh like we try a lot like we we we we get like a lot of like this like native model from

China like I and based on this model based on like our experience all the for all the like different task and I like right now still like several months gap uh between probably like you can see like Kim right now is like on the uh on the same level of like oper 4 point 4 4.8 eight but it's not as good as like fable five like in like I mean like in general task like of of course you can engineer this like model to be like perform very well like in some task but like if you want to do it like in your general task then I I don't think it's like on the same part yeah >> and and let's talk about how how these models are affecting the compute market

because right now you know there was a ma Kimmy came out there was a massive influx of users that went over there and started using it. Um, and then they had to shut down because they realized that their servers were getting maxed out. You know, there there was, you know, they couldn't com keep up with the compute demand and so they didn't just didn't have it in order to supply. You know, you guys are on the compute side. Um, and the theory is that like, hey, why am I going to pay for the premium compute of, you know, one of these frontier models when, you know, in a couple weeks, you know, they're going to catch up and I can just use maybe, you know, maybe it's 5% degragated performance, but it's like 50% or 80% degragated cost, right? And so, how does this look on the compute side? I would

imagine that every time there's a frontier model comes out, everything gets maxed out, right? And then there's a compute bottleneck. Do you think compute is still the bottleneck or has it been commoditized? What is the how does the compute market react when there's a new model that comes out? >> So I was definitely saying like there is like a huge like bottleneck in the computer site. Yeah. Since like if you use like the latest uh GPU card to train the model, you still have to use some like some other car to inference like based on the train model and the inference car will be like stay close to the uh you users to have a nice like user experiments. Yeah. So like for that

one like if you try the Kimmy like the native model from Kimmy you can see like the response response time is like very very long and and also like the token token price I I would say like is on the same price on the same level of like oper 4.8 probably like is probably like a little bit like close to like Fable 5. Yeah. So like which means like if we have like enough computing resources since like Kim is like a open source model we can download the open world and then we run it by ourselves and then we do all the things by ourselves. We can generate a token for for the users. But right now we cannot do that since like the the model is like very large which

will cost a lot of like reans to to do the inference like based on the model. And as you can see like right now the memory and also the the GPU cars in shortage which means like we only cannot get enough computing resource to do them inference like even if we have the like I mean like the open source like model there like we cannot have like enough like resources to do the model like like we are right now doing like decentral decentralized AI which means we are like using a lot of GPU cards to generate tokens for our product and for the product which will be consumed by our customers but we actually we cannot get like enough GPU card to do the inference

like for our customers which will cause this like shortage. >> Yeah. Okay. So there's a a shortage of GPU cards that is causing an inference shortage. I is is that correct? Yes. Okay. And so those GPU cards being in a shortage causing less or a shortage of inference causing the price of inference or intelligence kind of, you know, effectively to go up. Um, and so you're saying that even though there's this significant progress happening on the open- source models from China, they're still, you know, equivalent to, you know, the Kimmy K3 might be equivalent to the Opus 4.8, but not

Fable 5. But you're saying that that's not even the problem for the cost of the compute. The problem is not about the intelligence at that inference level. The problem is about the shortage of the GPU cards that is leading to the expense the price of those going higher and thus the price to even obtain that intelligence is thus higher because you have to get the GPU card and then you know get access to that inference. It am I is that kind of right like like yeah >> yes that's right. So actually like in addition to the GPU card, you need also need to have like enough memory to store the like the models like locally to do the inference like I I'm like if I remember correctly like Kim K3 have like

2.5 t like parameters which will cost a lot of like memory to store all the parameters like inside to do the inference. So that's another problem like in addition to the G GPU shortage. So like you need to have the GPU plus like the memory like to get the like I mean like the nice like inference results like to for people to validate that actually like Jimmy is like Kimmy is like doing like actually like better than oper like 4.8. >> Yeah. Okay. I see. And so th this is something that we saw play out in the market right because initially this started out with like okay AI is a huge opportunity the mega cap stocks like the

Google's and the metas and the Microsofts these are the hyperscalers and they were going to you know raise money to go build out their AI infrastructure then once they got that money from executing these bond sales they started spending all of that in the market in order to buy up all the infrastructure that they needed the raw materials that they needed to create and produce this intelligence right and and basically you know uh create these tokens and so then they started you know spending on RAM they started spending on memory they started spending on the semiconductors and and all of this right and we saw that take place in the market I'm curious like what comes next right we saw the growth of the hyperscalers we

saw the growth of the frontier model companies this happened in the private market and the public market with hyperscalers. Then we saw this rotation where all this capital expenditure went into the memory you know industry that went into the semiconductors. Now once you know they start to put this capital to work the memory companies and the semiconductor companies now they need to go buy raw materials in order to produce more semiconductors and and and more memory. What is the raw material that is used in order to create the semiconductor? Is it silicon? Is it like what what what is the next step in the supply chain where the memory companies

and the semiconductors are going to start to allocate capital. So like before like moving to like manufacturing like more chips, I would say like actually we can use some like the current problem like not the latest chip to to also to deliver the same results if you like dive into the engineering effort like more like for for for instance like if you want to like say like run like IPS v4 like pro which is like open source model like probably like the native the the strongest open source model before Kimmy come into the place like yeah you can use like 4090 GPU card to do the like

same delivery and like BC say like 300 cards yeah like you can use say like probably like 16 149 GPU card to deliver the same results like which will be like the same like from this like eight like B200 like GPU cards. So there like a lot of like engineering effort we can dive into to deliver the same result with say probably like the current or the not the latest uh the hardware like we we got we we can still deliver that but so that's for the inference part but for the training part we still need the latest GPU car to get the best result like coming from the the training but

after the training like you can use some other say like another latest native GG GPU card to do the inference. So that's so there is a gap and we can use a gap to deliver the best results. So, so that's like something like before like moving to the uh to the like new chips like for the new chips I would say like right now like we are moving from like 5 nanometer to 3 nanometer or even like two nanometers there and yeah it's not about the materials it's like about like how we how you can have like very advanced uh cart on the chip to make sure like you can you have a very

precision like things like to cut the chip to cut the things like in the in in in the thing like the the figures like in the diagrams you want and then you can deliver the results and of course the the yield or the like the I mean like the uh positive of the chips will be still like as high as say like the probably like 90% from the some like previous like uh notes So like how we can get that part how we can use a very advanced node to do the chip cut compared with something like five nanometer. So that's the bottleneck. We still want to get a very

high yield. We get want to get a very high positive results and we don't know that and the materials is not a very much like bottleneck. we can always get the like the real chip like from the sand. So that's that's something like we worry about like but how you can use the current precision to get the very nice like chip out of the say the the wafer. So that's something like we are like more concern about.

>> Yeah. You know I'm curious about something else with regards to the AI rollout. Who do you think seems more prepared to for this AI rollout? US or China. China's got a ton of of nuclear power plants that are approved to start building. The US has been far slower to approve nuclear energy. There's obviously other kind of resources like copper um and some other kind of rare earths that are becoming uh kind of popular and wanting to be acquired by governments and uh private companies. How do you think about the roll out here globally whether it be US, China, Europe, etc.? How do you think about the advancement of this technology? who's best prepared and suited to kind of roll all this out at scale.

>> Okay, that's a very >> risky question to ask. >> Okay. Yeah. So, >> well then we can we can just move on if you want. >> Yeah. So, like I have like some very wild prediction. Yeah. Since like I as you can see right now like the US like the the states they have like open AI or like anthropic they are like have this like closed source model. It's closer will like attract like more users to to it and more users will like attract more capital to it which will like causing this like flying wheel but there's then there's like the the previous like coming from like deep think where like deep think they show like you can actually use the open source model to

get almost the same result like from from the closed source model and then and then like the I mean like the most recent one coming from from Kimmy which also to show that if you can use like say the open source model to get almost like the nice result from the co- source model which I actually like I think like will cause a non for the US companies they will think probably like they can like dive like more into like how they can how they can like doing the exploration on the research sides like for the model instead of doing some some like marketing or like access control to see like how the users can control their

model like how they like want to bind bind this like users like to explore like more on the research side instead of like more on the like marketing side. So like in that way then they can like compete like since like the Chinese company they just open source like all the all the way there like you can just download you if you have like enough like GPU resources you can just run the model but for the US like they like all like they are like opaque to to the customers. So I guess it will like the competition between this like two countries will like be like better for the customers as customers like we are like using the latest model. We are like like doing the best

leverage on the components to get the best result of of ours. I I I was saying like I don't have a like [clears throat] prediction for now. >> Yeah. But like this like competition between the two countries like will be like very good like for our customers to explore it has like very nice space for us to deliver the best results like for for our C customers. >> Yeah I think I mean I think we agree you know I'm curious what your thoughts are on some of these um compute financialization happenings. So, you know, Koshi uh launched this daily compute product where you can speculate on the hourly price of a Nvidia

uh computer for uh each day. There's going to be there's compute per that are starting to come up. Like what do you think about and maybe you could explain to us the different actors who kind of make up the compute supply chain and why having the ability to like hedge your costs or the price of compute or go long or short as a company involved in the AI supply chain makes sense. Like maybe you could just kind of explain and give us your take on the financialization of compute as a new asset class. Some people are saying that it it may be the biggest derivatives market in the world in the next couple years.

>> Yeah. So like as I say we have this narrative called computer. So computer which means you can financialize [snorts] or commercialize like your hardware but it's like very different from like they say like the deepin like the deepin like it's just you own the hardware but you don't know how to program the hardware like for different like uh uh computing demand so that's a very big difference. The computer file announced people announcing like the user like I mean the computer providers which own this like hardware to use some like native software to program their hardware to suit for like different computer demands say like for

say like for for instance like for like pre previously like like the majority of like computer demand is coming from the zk generation but I think you can see like uh since the the layer tools they are like decliming like we are shifting a lot of like our computing providers to to the AI providers. So the AI providers were still like having the same hardware but they are running like different software to provide the same like this like AI tokens to say like SCOPS or a lot of like AI products on on SCIC they like still like making a lot of like

profits probably like they are like only the same hardware but they are like running different like software to still getting like a very nice yield. So computer like in our opinion like will be how you can pro program how you can incentivize like people who own their like hardware to to divide to devote their like hardware into like new say like an emerging like um market. Yeah. So like as a as a part as a project in this like market like CIC is like doing our best to to like to adapt to to the market to trying our best to get the most uh out of like the yield of all the

the owners >> and and what is the opportunity here like uh say we have some you know GPUs lying around or just you know a spare CPU MacBook right um you know this is compute that we can contribute to the network. Uh if someone missed out on mining Bitcoin, can they, you know, start mining some tokens and get in on the game? What kind of yields can uh can we expect from these things? >> Okay. Yeah. So it depends on the category category of the hardware like say like if if you have a Mac and Mac as you can see it's like very uh like it's not very weak but it's like somehow weak in say like the var and also in the GPU prep

processing. So like in that way like you can use your like say like the uh your MacBook to do some proof zip proof generation and also to verify the zip proofs like instead of like doing some like uh the AI token generation since like right now if you want to run say like for this like AI models on your MacBook it will be like a very small model probably like it's like 0.5 or like point like probably like at most like one billion parameters um model on on on your Mac and the like one one billion parameters like models will not like doing like very good in in the profit like for you like on on your

MacBook. uh but but if you have like very powerful say like the GPUs like say probably like edge like 100 or even like one a 100 or like B200 this kind of like GPU cars of course you don't want to use that to do the they cap generation so that will be like a waste of your like the hardware you will use probably like use that to do some like AI training or like AI inference which will generate a lot of like tokens like for like I mean like the AI tokens instead like the crypto tokens. So you can use the AI tokens like to get a lot of like profit.

So it all depends on like what kind of the hardware you you want. Like in science we have this like infer bench will coming up soon. So you can use a info bench to to test your hardware to find out like which hardware like which model you are like you are getting the best based on your common hardware. So I would say that that's like a very great way to to find out how you can get the best of your power. >> Yeah. Yeah. Leo, last thing from us. I just thinking about where where the financialization of compute is going. Do you think that this could be one of the largest commodity derivative markets in the world? Oil, gold, forex trading for example. These things are massive

markets. Do you see that the do you see the derivatives market for compute growing significantly to rival some of the largest commodity markets in the world? >> Yes, I I do see that like I see a exponential growth of like the market for I I mean the market for demand of computes. Yeah, as I just previous mentioned like the models are go are growing like larger like to train this like larger model like you need like like the like more powerful like cheaper cut to train the large model and after training you still need a very powerful card to do the inference for this like very very powerful model. So like I I so at least

like in the say probably like coming like one year or two like I don't see any like a declining like in the uh in the demand for for computing. Uh yeah so that's like all due to the uh increase due to the demand of the the AI computation. >> Yeah Leo, thanks for joining the show today. This has been AI supercycle episode number 43 with the co-founder of Cisic on the state of the AI market today. US China compute financialization the inference battle and more. Leo thanks for being part of the show um and uh keep building the in this financialization of compute market as uh as we both expect it to grow significantly.

>> Thanks. We'll see you next time. >> Thanks. >> All righty. study. See you