Auto Research on Yukon

Transcript

Welcome to Zero Knowledge. I'm your host, Anna Rose. In this [music] podcast, we will be exploring the latest in zero knowledge research and the decentralized web, as well as new paradigms that promise [music] to change the way we interact and transact online. This week I speak with Shabik Deb, head of research at IEN Labs. We explore their new project Yukon, a platform for open collaborative auto research competitions. We cover the origins of the project with the initial development of ECDSA.fail launched in May of this year as a challenge to match Google's undisclosed

quantum circuit for breaking elliptic curve cryptography. The result of ECDSA.fail fail exceeded everyone's expectations and so Yukon as a platform was born. Since then the platform has grown to host competitions like snark.fast better.codes and the he problem. Shabic shares how these competitions are framed around wellposed questions where the breakthroughs have come from and the vision for Yukon as an AI native scientific institution. This is an episode that follows a bit on our recent interviews with Ron Rothblum and Benedict Buns as well as the lean Ethereum series we did earlier this year. I'm very excited about this one since this is part of a larger story that we're seeing emerge from the ZK

community. That is the acceleration of cryptography research and engineering which is unlike anything we've ever seen before. We have many more episodes in the pipeline focused on this topic, so keep an eye out for those. Now Tanya will share a little bit about this week's sponsor. Today's episode is sponsored by Sveright. Sveright builds the critical infrastructure that preserves your right to choose. Sveright has built two products, Argos and the Sovereight mining pool. Argos is a free tool that lets anyone with ZEC stranded in an old unmaintained wallet client recover their funds. Enter your seed phrase, generate a new address, and your money is yours again. No fees, no complexity. The sovereign mining pool is Zcash native

and built on stratum v2 where miners choose their own transactions. Every block carries a verifiable signed attestation and payouts arrive in fully shielded ZEC. Two essential problems that have been overlooked finally solved. Discover more at sovereight.com. So thanks again Sbrite. And now here's our episode. [music] Today I'm here with Shabik Deb, head of research at IEN Labs. Welcome to the show Shabik.

>> Thanks, Anna, for having me in the show. >> Today I think we're going to be spending a lot of time talking about the platform project called Yukon. But first, why don't we find out a little bit about you? What were you working on before? What have you been working on at IGEN Labs? Like were you part of the IGEN layer launch and all of that? Yeah, tell me a bit about you. Yeah. So my background is in is originally in information theory, communication system, signal processing and I did lot of work on 5G telecommunication indust industry and I did my PhD under Shuram at University of Washington uh Seattle and my PhD was on blockchains like I worked on peer-to-peer peer-to-peer networking fair ordering of transaction consensus protocols and layer.

>> Okay. uh and and DA and I have been with Labs since day one. some of the the IN token intersubjective white paper and all this thing I wrote it if you have to play with in like in collaboration with Shuram Gotham and few other people and so if you if you know whom to blame for coming up [laughter] in subjectivity yeah and we did lot of work on this the layer DA and AVSS in the past and now we are working on some new cool projects that we that we'll be talking in the show today.

>> Very cool. I Yeah, quick disclosure. Uh we were an early I think quite an early investor part of ZKV. We were early investors in IEN Labs and it was cool to see all the progress on that front. Now today I feel like we're going to be switching gear though because Yukon is quite a different project. I want to understand a little bit where it comes from. There's a few websites that I've been seen floating around like ecdsa.fail fail or snark drop fast that I found out kind of after the fact was part of like coming from you guys. I I knew about it more from the domain specific side and not that there was a platform underneath that was helping to like put together these projects or these competitions. But let's go back to the start of this project. What is the

kickoff point of Yukon? So it all started from March when Google published their paper on coming up with a quantum circuit that is far more resource efficient than the previous state-of-the-art quantum circuit in breaking SACP 256 elliptic curve that is primarily used for public key cryptography of Bitcoin and Ethereum. >> This is the like they made steps towards Shor's algorithm working. >> Yeah. And and the implications are that if you find a much more more efficient this quantum circuit, the Qday becomes much closer and closer.

>> And Qday is this day that quantum computers could like break cryptography, right? [laughter] >> Bitcoin specifically, >> Bitcoin, the coins of Satoshi Nakamoto can be moved, which has not moved in years. Yeah. So that is the implications. It it will break all of internet cryptography. >> Yes. But what Google published was not that that had been solved just that there was like an incremental improvement that they had discovered but they didn't actually reveal it. >> Yes. And there there were lot of stories why they didn't reveal it but they used a zero knowledge proof to prove that they own they have a such a circuit with certain this kind of resource efficiency that they own nothing more than that.

>> So cool. We talked about this a lot on the show. We were so excited that they had this like they'd used ZKPS in this amazingly innovative way but it did like on the quantum front I think people were left quite unsatisfied because they weren't actually given the results or that they weren't shown how it was achieved and the downside of that is that if you want to defend against Qday you should know what circuits that are coming how to robustify the system against that what are the defensive mechanisms right and if you don't have insights towards that it will it is bad like you are not prepared for Qday Got it.

>> So those are the that is the big implication of like not revealing quantum circuits. >> Okay. So that happened but what prompted you to run any sort of competition around it because you're not a quantum computer company >> right [laughter] >> as far as I know. >> Yeah. So at the same time frame what happened is that the auto research a concept called auto research where you where you use your agents to just optimize on a given problem statement with a verifier and the agents keep running by itself and just keep optimizing it. It became kind of popular. It it catched public imagination and we thought that ah so is it was just a spark that one of our teammates Gotham got is that maybe we can use auto research to end the use the

zero knowledge prover that Google's team is using which they made it public the zero knowledge prover >> or the verifier >> the prover also. Oh, the proof of al so and you use that to take a crack into to take a crack into the problem. >> That's the start of Yukon, but Yukon didn't really exist then, right? Like how did you first launch this? >> So, we launched a a competition called ECDS.fail fail >> to to come up with a circuit that with the goal the goal was to come up with a circuit that is as resource efficient as the one that Google came up with but it will be all in the open and public and that's kickstarted the whole >> very nice I want to go back to that May when you first launched this project do

you launch the ecdsa website right then >> yes so that was sort of the first project and I had heard of that but I again I didn't I actually don't know if I knew it was I en layer related like it was just sort of something floating around. So ecdsa.fail fail a website where people could sub like this is what I didn't quite understand about this. Is it a platform where people could submit their answers? Are they tapping like what are they locking into so that they can actually use this verifier? >> Yep. So what happens is that we are in our back end we are running this verifier which basically is verifier that the Google's Google's SP1 Google used SP1. So since SP1 for that that's basically the verifier and we are running that in in the back

end. Now if you go to the website you will find how to participate. We wrote this CLI. You use that CLI to get your agent to start reading through the details of the competition. Second how to pull up the past submission. Suppose you you submitted a circuit I can with my CLI if I put it my agent I can access your your submissions also what circuit and I can start building on top of your submissions. It's very it's very different from how competition it's a chain. It's this is how research works, right? You build on each other's. So this is the the matter matter thing is it's an research at agentic speed, machine speed. You build on top of each

other. This is how research works. It's not a like a IMO olympiate competitions where everyone submits independently. And >> wow. See, I totally thought it was I thought it was PVP. Like I thought it was like everyone is like in their own like sort of on their local machine doing everything and then submitting and then competing but you're saying it isn't. So it's like over time >> like someone does a leap >> and then everyone will grab that and continue to work on it. >> Yeah. And this leads to a this leads to a like a huge amplification of growth because I don't have to discover it independently the whole thing. And this is this is what we want. This was a thesis when we launched. So we wanted to see if it's true and now now we know it

holds true. This multiplayer auto research framework uh it it works. >> That's so cool. Okay, going back to my question though, what was the improvement over Google's result? >> Right now at this point in time, it is 2.6 times better than Google's result in terms of the product of cubits and TOF. >> So cool. And I want to ask how did this come about? So I sort of picture if you're building on previous research there's probably these like step changes like when the French paper came out all of a sudden everyone's like the speed leaped up >> and it was like before you were 1.5% or sorry you were 1.5 times slower.

>> Yeah. >> Or less good. >> Yeah. >> And then all of a sudden it's like okay we're at par with the Google result and then we're going further. What were the big breakthroughs? So like one was this like figuring out the method that Google used. Were there other sort of big step changes? >> Yep. So there were I I in my mind there were three big breakthroughs I think that came up. The first one was making it external. It helped people who were this was even before the French paper came the mining of French paper that happened. Uh all these people who are experts in building their agent harness the persona they were able to attack this problem because they have much better. In fact, one of our team members who was not participating it when we

were doing internal testing the the one one day before we did internal testing, he spent like $5,000 in codec subscriptions and made huge leaps that our lead engineer was not able to figure out and like it's [laughter] just out of nowhere. This is basically and then we made it external. There were a lot more people who were much more sophisticated in their in their harnesses. They made lot of leads. >> So step one make it external. Let it step out of your company. >> Yes. Step two, this mining of this paper, especially the French paper, that was very important like they mined the French paper and then they did lot of hill grinding and all this hill climbing on that and it just went beyond the Google's paper within 2 weeks. Now after that for 2 months there was like if you

go to the chart you'll see that for 2 months there is almost a plateau. >> It was flat. Okay. >> Then something happened in middle August. >> Yeah. So in middle middle of the August someone one of the auto researchers his name is daddy he's from starkware he went back to the starting of the whole competition and he kind of summarized all the tricks but he he summarized this like bunch of hills and all this thing what hills have been >> it's like a survey of knowledge >> survey survey of knowledge yeah he did that but then he started from this beginning and he started building like getting his agent to explore like he's not building on other people's uh ideas.

>> He went back and said, "Here's everything we tried, but now fresh start. Don't get confused by what already exists, but now you know all these techniques." >> Wow. >> And and he and and somehow he came up with a technique called ping GCD pingpong. >> Say, can you say that again? >> Yeah. GCD pingpong. >> GCD pingpong. Okay. >> And some kind of ping ponging is happening. And >> what is GCD? Is this like a known thing? It's some kind of trick within the quantum circuit. >> It's some paper. It's a reference to some it's like the KZG of quantum or something >> something [laughter] and and [clears throat] before his technique the plateau was stuck at so current circuit is for leaderboard is 49% better than Google's classified

circuit. That's why it was stuck for almost 2 months. He made that submission and sudden it improve upon to 57% better than Google's submission and then people built on on top of his submissions. it now going beyond 60%. This this improvement that was done by Teddy, it's an illustration of like you keep the competition open, you let >> the all the past submissions be open be retrievable by anyone and they can like you don't know when something that has failed before can be used. These are failed ideas. You can access the failed ideas also not just the successful submission, failed submissions.

Sometimes what happens in research is that something failed because something else was missing but then later on some successful trick came you combine them you make huge improvement >> for sure. So it happens all all the time in research and something we is what we are trying to replicate in in this like our whole Yukon and this is what happened. This was an illustration of that that Teddy was able to access the all the past submission the past starting point and like make this huge leap. >> Nice. What was the name of the person again? It's Z is Zaddy. Zed A >> Teddy. He's from Starware.

>> Oh Teddy. >> Yeah, Frankly Teddy. That's his Twitter handle. >> Okay, nice. So this is the start. Everything you're describing is this ECDSA.fail. It was an experiment. I one thing I wondered how are the winners quote unquote being rewarded if it's an ongoing research project that like never really ends. >> Yeah. So the that comes into the whole platform how we are doing this. So right now if you go to Yukon there are a lot of competitions. What we do is we give a weekly like we have this $10,000 air weekly where we give people to the active competitions who are actively participating proportional to the outcomes improvements they are making.

>> I see. >> And each competition has a weight on top of that each competition can have their own sponsors. >> This is something for example with we did one competition with lighter >> so where they sponsored that also. So they sponsor some amount as a prize money for improve because they have their own problem statement they're coming up with and they want improvement. So they put sponsorship money which if you are participating that you and you are in the leaderboard based on your contribution you get something which is on top of the weekly reward platformwide weekly rewards that we have.

>> But on the initial ecdsa.fail site what was your cadence? Was it weekly? >> Yeah at the beginning there was no rewards. people were just participating out of their enthuser enthusiasm. They are burning so many tokens for for this [laughter] I guess was it was it partly just the challenge because Google had been like you can't know >> yeah there are two things one is that one like beating a golad all this like collection thousands hundreds of David golad >> the goliath >> yeah the second one is it is a very fascinating problem because it impacts the Q day how soon the Q day is coming like so >> people got really fascinated that Okay, let's do this. Like we want to see how

close we can get to that. >> That's so cool. So the ECDSA like had no prize money. It was just sort of for the lulls for the >> to win the competition. >> What else? >> Yeah. >> And and one thing I will say that there was it it led to a very good community. >> Yeah. I bet >> Justin has played like significant role in the initial disc cohering bringing all the people and then suddenly there were so many fascin interesting but they are all talking with each other it's like a real research call if you go to that telegram channel Slack I we can we can share that as part of this episode >> and people are just discussing with each other hey we'll break this cubits today this much hopefully to put today

>> so cool just going back to this May sort of period. At what point in the ECDSAFIL experiment did you decide you know what there's more than just this particular competition we should make this into a platform when did that happen? So first we we thought okay we should do more competitions yeah not platform we want to see that can we replicate this thesis that multiplayer auto research where it's people are collaborating at each other can is it truly a pattern that is people are enthusiastic about or is just a one time thing where because the problem statement is so fascinating people came so we wanted to check that so right after this within two weeks period mid June this thing happened we came across sassin's paper uh about

assessing that aspress. So they had that paper flock. Exactly. >> Yeah. We actually had Benedict and recently Ron on to talk a little bit about flock. Cool. >> And the one thing I think it was late June when they published that somewhere around June period. >> Yeah. >> And and and one of the things that they were doing was optimizing this flock on on MacBooks on Apple silicon. >> Mhm. [clears throat] and and we had experience already an experience with Apple silicon because of our other project called dark bloom. So we thought okay so this is the metric that they are targeting is how many hash compressions they can do on Blake 3 per second that's that's the that's the major metric they are targeting >> and again there's a verifier I guess

>> exactly >> okay nice [laughter] >> so we reach out to sassing's team and sassing team has been really helpful especially was the was was chatting with us to build like he wrote I believe he was a lead writer for the flock flocks implementation and he put us in touch with the rest of the team and others >> and and we did this yes Benedict and then we did the snagfast on Apple silicon which surprisingly led to like more than 3x improvement >> but it did it also lead to some other competitions cuz you found out there are other parts that needed to be optimized [laughter] [gasps]

>> it led to like a succession like we did a version of the snagfast which is about x8 86 on X that the t the regular architecture besides MacBook all all of them uses x86. So we did a version on that which is I think it it recently wrapped up I think. So that was the other competition we did and one thing that in that same time period that happened that helped us in conducting snobfast on Apple silicon and some other competitions on Apple silicon is that we invested time in building good sandboxes for Macs. So people talk all about the sandboxes but they are all for none of

them are for Apple. interesting >> at that time period. Right now there is some projects coming up but it's not it was not available. So we internally wrote sandbox it was we wrote that sandbox for a different competition which was about inference optimization for MLX which we we partnered with pool site which >> which had the most the best western model at that point in time the Laguna model series >> and we put lot of investment in engineering effort in writing a good sandbox which is stable enough because you need to run the verifier in the same hardware so that you proper validation and and we once we wrote that sandbox

for MLX. We reused it across other competitions where a different competition is happening but on the Apple silicon. >> Okay. >> So it became our core infrastructure. >> Oh man, there's so much I want to uncover here. I don't even know where to go. Like okay let's go back to snark.fast. I want to understand when you talk about it being on a specific hardware it's using a Frontier model. It has a harness of some kind on a laptop on a machine. Probably not a laptop if it's x86, but so it's sitting there, but I don't understand why. Like, is it not the frontier model that's doing most of the heavy lift lifting or do you need the hardware itself to be run by the agent in a certain way? Like I kind of Yeah.

To paint the picture of how you're actually like cuz for example, the ECDSA.fail fail example is very clearly software, right? Like it's a software problem. It's an algorithmic problem. Like you're trying and that to me is like the bread and butter of Frontier models. Like they're very good at that. But when you're talking about improving hardware, I guess are you talking about like the way that the software is interacting with the hardware and the agents are changing that and then you're running it on these machines and trying to figure out what kind of thing you can get out of it.

>> Uh so I'll give an example on the snog.fast itself. So the tricks that you do for accelerating say in the snap. That flock prover is very much dependent on the underlying hardware architecture that you are running. >> Yeah. >> Apple silicon has a very different architecture. It has it's neural accelerators they call it A&E and and is very different from say you run this flock on GPU. It's very different architecture. Now when you run this suppose you are running this flock prover in in your apple silicon you have it should use all these tricks and techniques that are enabled by that particular architecture this how to use Apple's neural accelerator it's it's it's a very particular tool even within

within say GPU each series like you have ampere you have hopper you have grace they all have have different architectures and each like software and you write it and you want to take advantage of the underlying architecture for optimizing it. It has to be very particular to that architecture. >> I see. But you can't change the architecture can you like is the architecture the hardware architecture? It's >> fixed. You got it. Yeah. Now you have to use your software. So what does what does software using and like a say apple silicon what is it optim? You have to ensure that the me whenever you are doing processing you are caching at like you are caching based on how frequently you are accessing the data. If it's a very fast like you need that data

regularly, you should be accessing it somewhere very close. There is all these bandwidth limitations, memory limitations are there. And so when you are running this competition, the snagfast, the agents are actually optimizing using your app your software. You need an Apple hardware for participant. It's in the snag.f that Apple silicon one and the agent your codeex or whatever agent is trying to run locally. this solution whatever solution it is coming up within your laptop >> trying to come up with that and then if it think if it sees that it is beating the current leaderboard then it submits it to the platform >> okay but is like going back to the

because it's a snark it's a it's flock right so >> yes >> I'm trying to picture what the submission looks like what is the actual action like are you is it approver that you're generating a proof and you're seeing how quickly you can do it like what you know what are you kind of trying to figure out here? >> So so in the snark.fast first competition the pro the flock thing we what we what we did is is that some of the files can be changed by submitter you can't change the whole thing it it's it specifically puts that as the a constraint think of it like a constraint and when I'm as a as a auto researcher running it locally my agent knows what it can change okay

>> and it's trying to optimize that like that that those files and those data loading and all these It's trying to change that. >> And was that defined before by you? Was it you said here's the proving system. Here's what we want to tackle cuz it's not go ham on everything. It's here's like the specific >> bottlenecks that we want fixed. >> This is the hardest part like building a verified is the hardest. So that's why you need domain experts. So Ron, Tamir, Benedict, they all like these are the domain experts coming up telling they because they know what is slow, what is can be optimized. and we work with them like just ensure that these files can get changed >> then this is very important because I

can just maliciously change arbitrary any other file I don't have to ser like follow those rules but now when I submit to the platform the the platform the back end our back end infra should check that it's not changing some other weird files that is violating the constraint of that competition >> so that's a part of verification also >> so so cuz I remember Like when we had Benedict on and more recently we had Ronn on, we did talk about snark fast but actually not in as much detail as this in that like the flock paper has a verifier but that is verifying the flock prover that they created in the case of the verifier that you're talking about for snort.fast fast. Is it different?

Like is it the same >> verifier? We the same verifier. But then how do you improve on the ver like then is it just oh they're able to get the verifier to to accept it faster. >> So in this competition we are not changing the protocol. >> Okay. >> The cryptographic protocol remains the same. That means the verifier can remain the same. It's just the implementation. Can you make it more optimized for number of blake three hash compressions per second? That's >> but then so it is speed. It's just you're you're trying to like get the verifier to sign off on a proof faster. Yes. Like to say like the prover is proving >> and then the time from when the prover starts to the point where the verifier accepts it gets shorter as you get better at the engineering >> on this particular hardware

>> in this particular hardware. >> Okay cool. >> So you see there is so much specification. >> Oh my gosh. So it has to be so building a verified auto research you have to be very careful what you are specifying all these things. I wonder like are there some problems? I mean I want to go through more of the projects but are there some problems that you sort of thought of doing but then decided not to do because you can't put it in the framing that you need. >> Okay, this goes back into a much more bigger problem. There are a lot of problems which are illposed.

>> Illposed like poorly put together. Yeah. Poorly asked. >> I'll give an example. Yeah, I'll give an example. Build me a quantum computer. That's it's a it's a it's a very problem. Yeah. Two problem. We don't know what is. But if you had broken down this this problem build me a quantum computer then you'll find also that the problem state from from ECDSA.fail is also one of the sub problems. >> Yeah. >> You are now defining well like well problem. You are coming up with a series of sub problems which are well posed and then you do auto research for each of them. So there are like lot of such problems we we have hope for but you need to break it down into this oil post problems >> crazy

>> this is similar issue with proximity conjecture so when we saw this proximity conjecture like after our ECDS fail success one of the things we wanted to do was h so there is a EF has put $1 million for a for proximity conjecture >> well the proximity prize yeah prize so so this could be if can we do auto research around that >> wow Oh, >> and we we did we we did like we didn't have anything internally. We didn't know like how to break this problem because it was the way the statement is written in the proximity gap conjecture the price is it's it's it's a it's a binary either you prove it or not prove it.

It's it's very hard to write a verifier for that. And what happened is that later on people Alexander Hicks and Gako who was in your show also recently probably and so they wrote they're they're writing this lean verifier for for a particular instantiation of this proximity gap conjecture here instantiation means the field size and all this thing the which code to use and that's what became better.codes >> better.codes codes which is the proximity prize like on the Yukon platform. It's a competition. That's interesting because actually so proximity prize was announced quite a while ago. I think actually on our show Justin announced it back I I feel like

when was this? I guess it was last year he announced it and but then it was interesting because it was almost like the prize was announced but the actual framing of the prize what it what is the competition what has taken some time I think to properly define and what you're talking about here is like this is sort of the conclusion of the first part which was figuring out what the problem is and how to gauge it and now here's a here's a chance to run a competition. Yeah, I not say it's a conclusion but it took lot of effort from Alexander and Gakomao and others this like coming up with a lean verifier making ensure that it is correct like to a correct instance that is helpful because you can't take just random fields randomly

>> and I think that's what we saw in the research like we saw a lot of attempts to sort of solve the problem but without the constraints we saw them kind of go in very different directions and you would get like lots of variables and different yeah kind of what you just described >> so That's right that having domain experts is very important. >> You just mentioned sort of the formal verification side of things we're using lean in for this particular problem. But is that something that you expect to see more of to be used with these competitions? Like do you kind of need something like that in order to even run them?

>> Yes. for for lean verify. This lean provers and verify they are very helpful for doing checking >> that whatever whatever the auto researchers are doing they are correct or not >> but it also constrains the type of pro like how many problems you can solve. Even though AI is becoming very good in writing this lane prover, what ideally will be much more helpful having natural language verifiers >> like AI just trying to check by itself that oh this whole this submission is it correct or not instead of running through lean because lean is actually slow in the better codes there were like some of the auto researchers they were saying that ah this is so slow it takes

time to do that it's kind of I'm sure that as like AI is becoming better and better and coding So I'm sure this will this problem will be much more like it will go away hopefully but still the natural language verify is the fastest thing >> okay >> that you can have. So ideally that's what it but then you get stuck with the problem that the AI might have the biases. What are you going to it's not objective. Lean is much more objective. >> I see. Okay. >> It's it's a deterministic software at the end of the day. >> And le when doing a natural language with AI is like who god knows what is like what is it bias? It's you have to then you have to be doing second guessing.

>> Got it. >> It's it's just hard like this is a harder problem and we are working on that. >> Uh but on an immediate time scale. Yeah, the having the lean is proving to be very helpful. >> It's very helpful right now. Okay, let's keep going through. So, we've mentioned now ECDSA.fail snark.fast better.codes. You mentioned MLX. I don't know what the second part of >> MLX.Fast >> MLX.Fast There's a bunch of projects on Yukon. If you actually just go to the website, some of them you mentioned were kind of like boxed like there's a time frame that they're actually running it and then it ends. Others are constantly active. Let's go through kind of some of these and like and maybe actually just cover the ECDSA.fail,

snark.fast. Are those kind of perpetual? You don't have an end in sight. >> ECDSA.fail is perpetual. >> Uh snark.fast it has ended. >> Oh, it has ended. Okay. >> Yeah. And and and in fact, Benedict has written a rise a nice summary of what improvements they have gotten and how they're adding it to the actual flock's main branch now. >> Yeah. >> So that's kind of cool. So it has entered MLX.fast over >> MLX.fast has like a it's it's a it's a meta program where we optimize different different open weight models and each open weight model optimization has a limited time period 2 weeks 3 weeks. So we started with Laguna's model then we did Quen 3.7 model then we did one for Jamaat which we got a shout out from

Google's Jama team. >> Cool. Wow. And I see you're doing CUDA.fast fast which is for the Quinn 3.8 flash next which by the way I've been using and it's like insane >> it's like uh it's opus on your computer it's crazy so one one of the things that that is happening with the MLX I so I want to picture where MLXFast comes in a broader thing so we have this dark bloom project which is about you can plug your unused Mac max to to be a provider for inference and you you use bunch of the openweight models that you can load into your own MacBook Mac ultras and all Interesting. So the way we are seeing MLX.f is a is an R&D pipeline an open R&D pipeline for dark blooms in

production inference network. >> Okay. Wait, Dark Bloom being your project? >> Yeah, it's also our one of our project and and right now it is it is the top provider for Gemma force 26B model on open router. >> Oh wow. >> Uh and we are loading new openweight models into that. >> Cool. The dark room is a whole open network for you can plug your MacBooks unused MacBooks to serve inference. >> Nice. >> And this MLX first is like an open network of R&D for dark bloom. And we are optimizing this on Apple silicon.

>> We actually have to talk about like the connection to IGEN layer too. But I feel like there's the step seems to be like I layer to dark bloom to Yukon and specifically these competitions that are like MLX.fast and I guess CUDA.fast Fast is another one in that >> MLX.Fast is one of the competitions that we are running under Yukon that directly ties up with DLO. >> Okay. Does CUDA do fast also? >> Right now, not yet because on Dougloom we are only serving on Apple silicon. >> Oh, I see. Okay. Oh, and CUDA.fast would be on uh NP. Okay. Okay.

>> Nvidia GPUs. >> Got it. Some others that I'm actually just seeing on your site. You you did mention the lighter.fast. That one's over. >> Yeah. So this was from a company like one company one project was like I want to accelerate >> I so there is a good story around that. So so lighterfast they already have a inproduction lighter software for listeners who don't know the lighter it's one of the perpetual like biggest perpetual exchange >> we had them on the show so they're like a zk rollup based per deck. >> Yeah. So they are using all the their Mac max hardware for running their the software the zk prover on their end. So now now the question was can you accelerate that? Can you can you optimize this thing because the more

optimized their approval will be the faster the throughput per second on their on their exchange will be which is directly helps into the PPS trading. POPS is all about timing right so there's a direct correlation with the PPS trading and optimizing the ZK prover in their case. So we partnered with them and initially we the like their expectation our expectation was that yeah maybe you can get 1.5x to 2x optimization. >> Yeah. What did they get? >> It led to it led to more than 10.5x optimization.

>> Wow. They must have been so >> in terms of throughput. Yeah. Throughut per second like and and it was crazy. So init initially the prize money was around 15,000 lit tokens for this and the the amount of optimization that happened ended up increasing it to 25,000. Wow. Yeah. >> Tokens. Yeah. It's like right say 90,000. >> I wonder for the participants right now. Just a quick kind of side on the economics like they're spending tons of like cash on running these models and inference. Are they getting like if they win the prize, do they actually get money or do they lose money on this?

Like overall it's probably a money loss because it's like shared among many participants. But like >> yeah, I can talk about that. So, one thing that is that is people are realizing is that if you take a $200 subscription, you are actually getting $20,000 worth of tokens. >> Okay. >> So, if you win prize money which are in like $1,000 and like Yeah. It's it actually makes you whole. >> Yeah. As long as they're within that value like as long as they're >> within that value. >> So, the way we are seeing this is that suppose you from the other side. So you this is we are talking about the auto researchers but you have the other side is the problem creators.

>> Mhm. >> The personal people who domain experts the people who have the problems. It's like a open infrastructure where you come and deploy your pro problems. You get access to a wide pool of people who are specialized who are very much better in using their agents and models and harness and tool calls and everything. So you're exposing your problem to a very good like a wider group of people and and you are getting access to a wider share of tokens being used for solving your problem. >> Cool. And different models and different setups and harnesses and >> so ideally what you do like if you do it a single player is you are using a same model you you probably don't have

diversity of models. You can probably get a diversity by hooking up cloud codecs groi something like that but then you don't have a diversity of prompts you don't have a diversity of memory you don't have a diversity of tool calls you don't know what people can use I'll give an example on this like this is very relevant for proximity gap conjunction better doc codes so better codes there is two bounds there's a lower security bound which is the soundness bound and there's a upper security bound which is about attack >> so just a reminder to the listener this is the proximity prize competition. Yes.

And the lower security bound traditionally for for many many years I think decades people have known that it is stuck at some certain bound called Johnson bound. >> Mhm. >> And what we saw in the better doc course that specific instantiation the whole competition was stuck at on August 21. I specifically remember those day on August 21 at 63.99 bits. 64 bits was a Johnson bound in that particular instantiation of the competition. Okay. >> And then then then what happened is that for 6 days there was no progress. No one was see it. Actually I'm looking at your chart. It's just a flat line. Yeah.

Until August 28th. >> What happened on 28th? That's the very important thing. So one of our colleague he contacted one of our a big auto researcher from ECDS. Bartos. He's a mathematician at Poland >> and and he's he like he's very who has been who has made like huge contribution in ECDS like to to actually like to make a lot of progress. So we thought maybe he has such a good setup that he can take it correct. >> He's a algebraic geometry guy. He's not at all a cryptographer. He has never heard about this problem.

>> Wow. >> He comes and within few hours it went from 63.99 to 64.01 01 bits and it is the it it led to a like a like it is it is it is a first time be where it goes went beyond Johnson Brown. It it was surprising and the tricks that it used is some combination of algebraic geometry something something >> we have to get some one of the cryptographers like again probably to to know this the details and >> and the surprising thing is that this is what Barto speculated that that because he has been using his codeex subscription lot for his own personal work around algebraic geometry maybe when it tackled this problem it applied

techniques from algebraic geometry for this thing. >> So this is this is open innovation at its best, right? It is a person who is not even who is far from cryptography. >> Suddenly comes and deploys his existing to set up and takes a crack >> and then suddenly the moment 64.01 it went the whole some some happened the whole world broke down by three slipped something happened. Uh people were chatting on Twitter about this Ariel Gabizan and Kobe and everyone was chatting. I think it might be worth bringing one of the cryptographic expert to go over in details what exactly has happened.

>> That's cool. >> Yeah, it's it's something happened and then the story didn't end here. >> Mhm. [laughter] >> Then few days later some of the other cryptographers they have a they have a whole other bigger problem on coding theory which has been open for since 30 years. I think to come up with some kind of like to prove some kind of this security bound and there are three papers that came up within days that's that say that they got inspired from the techniques that were used in breaking the Johnson bound.

>> Yeah. >> That led to a break some kind of breakthrough in this 30-year-old problem. [sighs] >> So cool. >> No one would have conceived that this series of events would have happened. >> Yeah. This is this is like open innovation, collaborative research. >> And it's it feels like it's at its infancy, too. It's like the very beginning where people are kind of like, oh, >> whoa, okay. These problems that we thought would not be solved for a long time >> can be pushed along at least, like maybe not full solved, but like improved, improved, improved. That's so cool. >> I want to keep going through some of these projects because a recent one, I actually saw this on Twitter. It was about Heish. I don't know if I'm pronouncing that right.

>> Yes. So this is a I mean as far as I can understand it it's like you build a shape and then it needs to with 11 hexagons I guess. What is it 11? Yeah. >> It's it's it's pieces some pieces >> some and then you have to be able to like using these shapes like kind of build circles around each other. I mean maybe you can explain it better but it's it's they've maxed it out like they've never reached a certain level of rings and they're trying to reach this level of rings. I mean what kind of field of math is this? This is like totally different from what we usually talk about [laughter] like shapes.

>> Yeah. This is some kind Yeah. shapes styling. This is whole geometry. >> Geometry. Yeah. >> So the person who is like the domain expert every problem has some kind of the domain expert for this is the mathematician Bartos >> like this is really such a good story like Bartos he we came to know him from from ECDS. >> So he's been working this is the same guy. This is the guy who has an amazing setup. But is this his actual field? This is like what he actually works on. >> This is Yes, this is his fate. And he has actually got featured in the open chat GPT release about how using math.

He was one of the three people who was featured. >> Okay. >> So he helped in the this breakthrough better. And now we are working with him on a series of math problems. So the first one is this he problem which you mentioned. Yes, exactly. This is the tile that you have to come up with a piece. You put the piece and then you have to use the same piece to create layers around it. >> Yeah. Which is like a full ring which covers each surface. I don't know if that fully explains it, but it's almost like we'll add a link to the >> It fits the whole thing. >> But yeah, it's pretty cool. It's quite easy as a just a viewer to kind of understand what they're trying to do.

>> Yeah. And the piece. So there is a constraint. The piece has to be made of triangles, squares or rectang rectangles. This thing not like irregular sheets. It has to be a combination of that and the current record I think believe it's the known literature record that was it at four you can create four tight layers >> uh with this piece if five will be a breakthrough like it's a mathematical dis huge mathematical discovery >> and the way the competition is growing it's very close it's some you can make an approximate 4.98 layers >> yeah but when I look at these just to check like it looks like they're hex there's like 11 hexagons. Is that what it actually has to be?

>> It's it's a combination of triangles, rectangles, squares. You make it into >> like that. Okay. Okay. Just in this graphic, I got it. >> Yeah. You can come up with any tile shape. >> Okay. >> The constraint is you can you build a five layer out of that shape. >> Yeah. Exactly. five rings basically you're trying to get and it looks like the last submission the kind of furthest along submission has just a few points where basically you want the ring to completely cover the outside surface but they haven't quite made it to five. So right now I'm seeing like 4.9881 and you see these little points the outside ring still has like a few points where they can't quite fully cover it in gaps. Yeah. Okay.

>> And this is actually a hard problem. What we are also doing is that with this hish problem exactly we launched a feature on the yukon which is about discussion format. >> Ah >> so many times what happens is that as you are exploring research you're doing research you don't fully explore the whole thing you go until something sometime and you you have like lot of other things that you want to but you don't have time. So what happens is other you hope that some other people will take up the remaining pieces and explode that. M >> so what we are doing is we piloted this with the hish problem this GitHub discussions where agents it's it's all there in the CLI like you don't even have to think prompt your agent it's in the CLI and the agents when they are

doing research doing hill search trying to strategize in your local site they are also pushing discussions into a this GitHub discussion board okay >> and uh other agents are picking it up and sometimes in this case Barto Barto is responding to this uh thing It's like you can manually steer like when part B was the domain expert here is responding then then the agents are actually reading those response and they're getting steered by the domain expert. This is something we want to into like make it flushed out completely is that the domain expert should be able to see that what where this open network of auto research has gone >> in their problem statement and then they steer right they they have the best intuition.

>> So they steer it they left clues they left like what should be explored and all this thing. So we are piloting that also. >> So cool. Okay. Now I want to hear more about the sandbox the platform and we still have to talk about the connection to iigen layer because I layer is a very different project from what I remember. I maybe it's not that different but it was like traditionally all about restaking. So you could run things like a DA system on top of it. But how does Yukon connect to iigen layer and is dark bloom in between here? Yeah, I can start from the origin story like from our labs. So our goal always has been to unleash the open innovation.

>> Okay. >> Uh we always believe there is more outside talent than internal within an org within this thing. So how you build products that lead to this kind of open innovation like enabling this people outside to be able to build something that others others would like to use. So I can learn what's the thesis of that right you there's a platform there's a capital like Ethereum like ETH ETH there's a lot of ETH restake into that can you have a software that you build that you can you reuse the ETH security for economically securing your software >> you don't have to build your own blockchain or all these things so that was open innovation it's the same line of thinking that is happening on Yukon and Dloom in Yukon it's a platform where

you can come and launch your problem statement if you're a domain an expert. >> Mhm. >> And on the other hand, there there is a lot of outside talent which are might be much more proficient in using agents. How to you like there could be other domain experts who know how to navigate the hills and they are trying to attack the problem >> except does Yukon use any of the like as sort of resting? No, it's completely different platform. >> It's a completely different platform. >> What about dark bloom? So dark bloom you were saying is a place where people can actually sell their compute their unused compute. So dark bloom is is a place where so what happened is that when open claw was its hype right people bought all these MacBooks Mac hardware Mac

hardware are not even available for a long time after that so the hype died down after a month and then there is lot of unused hardware what are you going to do with that >> what we thought is that this dark bloom is the place where you can plug your unused Mac hardwares >> and on the other side there is this inference people who need inference. >> Yeah. >> Who where this this this inference will get routed to the people who are serving models and they will process it and they will send a request and in in return for certain fees.

>> Now the one important thing is that you need to have privacy. Suppose suppose you are I'm one of the I'm one of the people who are running a MacBook in this dark blue network. I plugged it into this >> and I send you whatever I'm asking but it's so it's so in the Yeah. It's like completely clear text. >> Yeah. You don't want me. So Apple has bunch of privacy preserving mechanisms within that. So we took advantage of that to enable >> keys mostly. I guess >> it's their enclave secure enclaves. It's a version of T but within Apple silicon they have their own version >> and we took advantage of that that any one providing the ser inference request serving the I not able to see your request.

>> So basically like if I sent my request it goes into your secure enclave that you cannot actually access either. There is a network the coordinator there's a central coordinator which which does a loadbearing load balancing request to see who whom it should get diverted to >> based on past performance or the availability of the machine right now and all these things >> and dark bloom primarily looks at Apple >> it's it's for Apple silicon right now >> okay and how are people compensated are they compensated with IEN >> so we started we piloted this whole thing with uh US dollars actually US dollars not not >> USDC and yeah not even USD We expand we expanded we are expanding to that >> and recently we had a partnership with open router

>> where they relay the request to to lot of inference providers if you go to their board you'll see this list of these things so dark bloom is now listed as one of the inference providers dark in open on open router and right now for gamma 26b model it is the top most network inference provider >> cool so if you use an agent with open router and you pick like you can choose with an open router that like dark bloom would be an option but what models would actually be served I guess it's all the open models so like K3 and >> not right now we have no not Kim the small enough models that can get fit into an Apple we plan to have this multi-Apple devices that can serve

bigger models but not right now >> okay cool >> I want to return to the story of Yukon because it it it just does feel like very much on the frontier of of a lot of things that we're track that I'm tracking that I think a lot of our community is tracking right now. You have I don't know how many like 12 active challenges or challenges that have been that have been hosted. How do you go forward? Do you imagine a world where you have hundreds of competitions going? Do you think that Yukon will evolve somehow? Because I like I love this. I think a lot of people who really get inside it are kind of amazed, but it also feels like that's it feels like something that works really well maybe

because it's small like also if you guys are providing the prizes like there's only so much funding I imagine that you're going to provide to this platform. [laughter] So I am curious about like where does Yukon go? What do you sort of envision as a future here? So this is very relevant question I think with the current scenario how people are thinking about AI and for scientific discovery. We really see Yukon evolving as the AI native scientific institution. >> Oh cool. What does that mean is that research traditionally scientific research academic research industrial research has been always about building on top. So there are certain traits that are there. What does like this

institution should emulate this traits right that so I want to list out the traits you like research means building on top of each other's idea >> second you should be giving credits on whose idea you have built third even the failed ideas should be given credits for that because some many times actually out of thousand failures you get one success that's kind of the thing that that's always there in research so these three traits are so important and what is happening with with the with the arrival of AI is that the whole life cycle of like doing a research has accelerated from like months or years to like days. this ECDSA. fail better these are actually an example of how

breakthroughs can happen in like within like hours or days like it's it's just crazy like I was thinking that these are like a bunch of new couple of move 37s that have happened on an a like on research >> and we are like as as part of this yukon's evolution we are trying to have this credit system partial credit system where people can credit each other's ideas negative negative space credit like where people fails Failed ideas are also given credit and then coming up with a system where if you have a illpost problem can you break it can the system actually help you in breaking down into a sub series of sub problem that are well posed

>> interesting these are the things that we are trying to build on yokon because this is how we see that the future of AI research will AI enabled scientific discovery and scientific research will happen. >> Cool. The way the way the the current system like if you see the Twitter the how that mathematicians are saying that that AI should not be used that for doing math because frontier labs are kind of stealing potentially the thoughts of mathematicians if mathematicians are using the models and then those models all of a sudden like have these breakthroughs possibly due to like the use of the product it's problematic a little bit I think that seems to be the question today >> yeah this navia stroke books saga drama

has actually put that into a limelight. So this credit systems and all this are so important. >> Yeah, >> I'll say that bitcoin's fundamental technology of proof of work was built on credits, right? You build on top of each other's block. You you can pro verify other people's like proof of work like whether they have actually come up with the correct nons for that hash. It's something similar to that like can you have a a programmatic system where people can ver verify that that person has do done a real work can you credit that person's work even the negative work this is a hard problem and especially with AI being like enabling like making everything fast I think this this is the problem we have to tackle >> yeah yeah yeah but what you just said so

future of yukon is building tools to be able to break illosed problems into subpros that are more t like something you could tackle with agent And that it's like clearly there's a verification step. Then there's build is it building the community too? Like you need to have all of these people who are aware of this and are locked into it. Y >> and then I guess you have the credit. So you want to be able to like make sure that you can associate each contribution with some some agent. It's it's still agents. It's not really people but yeah. [laughter] >> Yeah.

>> It's their they paid for it. So >> y that's what we want to do. It's it's some manner. It is like from first principles. How do you build an AI uh native institution for scientific discovery? >> Crazy. But then how do you fund how like how are you funding the participants? >> So this is goes back to that examples I was giving for lighter.fast and better. Is that the people who want some problems solved and they want to take benefit of this collaborative research auto research network >> that they should be putting some rewards that incentivizes people to participate and tackle their problem.

>> Okay. So you you expect it to be more like some team like the lighter example where like a yes a project comes up and says I need help with this XYZ it could be like a big could be like Google or some like big you know tech company comes forward it's like we're trying to tackle this engineering problem can you and then you you help them frame it they offer the bounties or the the funding mechanism which could be a weekly check-in on like how much contribution has happened that week and then you start to develop it but what about those really hard mathematical problems that like maybe companies don't care about.

>> Yeah. Yeah. Yeah. I I want to get so there are two separate personas of of this part. There is the problem creator who is the domain expert and there is a person who is interested in seeing that problem solved but has the funding for doing that. This this always happens in the traditional research institutions. Many of the prize money say millennium prize problems or solving rare disease problems. The the funer is a very different p per person person who cares about the problem but the person who is the domain expert who's putting out the specifics of the problem is it is a domain expert he's a different person.

So we we envision that there will be people who might say it's I'll take the hish example like Barto's he's a domain expert coming up with this hish problem but there'll be someone else who is who's interested in seeing the beauty of this problem wants to see that it gets funded >> because that's how traditionally research this starts from very nice renaissance era Europe right where Medici families they were not the domain experts in art but they were finding my funding Michelangelo Leonardo Vinci and all these guys right >> so it's it's a RS has been there in history like they're two different personas.

>> We need to find a way to incentivize that sort of fun persona though. If it's not a company, >> right? >> It's so interesting as you go through this. There's echoes to me of things like Gitcoin where there are like these experiments that were run in the past. We were pretty involved in that. We had a grant. We also helped organize the ZK track at Gitcoin. That was like projects building a certain technology and then people out of the community wanting to fund that those things. There were I mean the reason I think of Gitcoin is they also did project specific fund rate like funding rounds. So like Zcash ran one specifically just to Zcash like anything built with Zcash or like kind of around that ecosystem. And it also reminds me a little bit of ZK hack which

I keep trying to insert into the history of things. [laughter] But basically we were running this like test for it was like these problem spaces that anyone could participate in and it was a way to sort of highlight talent. Like a lot of people got hired because they participated in this. They learned they excelled and then they were hired. In this case like the participants do you see sort of a future step? I mean you kind of talked about Bartosh who who you've like who you contacted you were like hey man you had a really good setup can you come help us over here like do you think that kind of thing could start happening you will see that many people when they are they start as a auto researcher >> if you go to ECDS many people who went started auto now they have become kind

of semi-expert right now on quantum computing it's crazy like so then that means people who want their other problems get solved they can they can be in touch with them getting the feedback on that >> and of course if there are more quantum computing competitions they will be the people who will be most excited to participate because they have built a skill set I I'll consider them as semi semi-domain expert in this and this is one of the things that is that we are seeing is that this is something like even though I have come through the whole academic structure but one thing I have always it has not like in my mind is that there's a lot of gatekeeping is there in academics >> you have to pass through this you have get selected for a PhD program. Someone takes it but there has been always lot

of interested people who are who see some of the problem statement as a hobby. I'll give there's an example like there there's one of the top participants in ECDSA. Fail is a landscape designer. >> Wow. There's another guy who is a rancher but now they in the side hobby they were particing EC they know now know a lot about quantum come circuits >> which is crazy like our traditional academic structure research structure was never was never set up to enable such kind of people >> it's always about papers and all this thing but like there are a lot of people who are very scientifically curious about learning new problem statements trying to attack them >> I mean as much as we're kind of down on the academia though but I I think the

reason it's set up the way it is is because in the past it's been so difficult to coordinate massive amounts of brains. >> Exactly. That is >> because you couldn't deal with tons of brains. You had to pick some brains and they had these huristics and they're not perfect but they did pretty good. they got here, right? Like they tried to sort of find the brains that they thought would be the most effective at at solving the problems and those are the >> machinery was missing. The machinery just the machinery to tap into all of the brains and also the the heruristics were always imperfect. So you ended up with like I mean human heristics are always like political and like I like this person therefore I'm going to kind of give them a little bit of a leg up.

as much as they claim to be objective, it's inevitable that that's that's inhuman. >> But yeah, with with huge coordination like technologies that have come on, thanks in part to the blockchain stuff, I feel like there's so much more potential to tap into that plus we have the acceleration of the agents because we shouldn't pretend that these are just people, right? [laughter] >> Yeah. [gasps] >> They're people running their agents >> and also these are not just agents. There are actually humans who are prompting like the the discussions that are going like exchange of ideas among humans that is happening in ECDS. better dot codes that I saw in the like in in the telegram channels and all the slack channel there's lot of human element that is there people are sharing and

it's so fun like you're sharing ideas what is the next strategy you want to attack you build on that the other person builds on top of that it's it's just fun >> amazing so shabi I want to say thank you so much for coming on and sharing with us the story of Yukon I've been so curious and kind of diving into it with you and going through some of these cases it's really helped me to understand what you're doing. How do people get involved and maybe what can they expect in the coming months? Because right now, like we said, there's about a dozen projects up, but tell us how they get involved and what to expect.

>> Yeah, the best place to know all the active competitions and what has happened in the past is go to yukon.org website and if you scroll down, you'll see the the list of competitions. Go any take go through the competitions that you like and you will have guidelines on how you participate, what prompts you can give your agents to start optimizing participate in those competitions and just directly DM us like how you can get into the Slack channels and all like you can you will have the the Slack channel link in the website itself just join there and you will like have access to like our fellow auto researchers who are also hammering down these problems. So that's how you participate on coming days and weeks and months. Some of the

verticals we are targeting we are going deeper. Okay. >> So cryptography is as a vertical. It is very nicely built for writing verifiers. >> Yeah. Especially ZK ones cuz there's verifiers right here. >> ZK ones. Exactly. And this somehow echoes with Nicola's blog post on generic cryptography. >> Yes. Generative cryptography. Yeah. We're going to hopefully get a chance to explore more of that on the show. So cool, >> right? And so we we'll have lot more competitions coming up with on on cryptography in part partnership with EF and other people. The second vertical that we are targeting is around MLX first which is inference optimization.

So we have lot more more models we are doing. This is primarily like why we are going deeper on MLX first. This is two parts. one is for a dog bloom and second we are like we really believe in open-source AI to be as a deterrent against hyperentralization. So we want to enable open-source models to be as good as possible that serves people's needs and this is our way this MLX fast is our way of like contributing towards making local AI with open weight models as competent as possible to against the the close closed models also.

>> Cool. Besides these two verticals, we are trying to expand to other verticles like biology, chemistry, material science specifically and and cyber security also. So hopefully we'll have something on those new verticals soon enough. >> Very soon. Very cool. Thank you so much for coming on the show. >> Thanks Anna for having me. >> All right. I want to say thank you to the podcast team, Rachel, Henrik, and Tanya, and to our listeners. Thanks for listening. >> [music]