PromptPiper: Prompt Compression — ETHTokyo 2025 AI-Track Runner-Up Demo

Transcript

Hello. Yeah. Uh good morning Oscar, good morning Kitcon and everyone who came here. Thank you very much. And we will briefly talk about uh our experience during Tokyo and uh also about our project. Um so I'll briefly just mention the conference. Sh did you like the conference? >> Uh yes I did. Okay. Yeah, the conference actually was pretty nice and uh there was also a couple of talks about the importance of hackathons in the space and uh for the hackathon itself we got like uh 42 projects competing with uh 124 hackers and we also got italic joining us to check the demos. Um yeah, there was uh three tracks, cyber punk

and uh DeFi and AI. And I'll just mention a couple of projects that didn't make here, but they were supposed to present to you. One of them was rap monero is a device solution if you want to use rap man. I thought it was a really cool and clever hack that got the second place. And then there was a cyclone cache that's um was improvement for a tornado cache. and to minimize the yeah the cost of using and finally there was also AI track that we also participated and this team shout out to them didn't make it but they actually won the first place we came second and this is like a mental health AI system that they built

and did you have any favorite projects >> yes that' be promper >> okay so yeah we also were competing in a truck and um Our project is about the prompt compression. We had a nice time demoing to Italic as well. And um yeah, also thanks to all the judges and we hope to see you all next year in Tokyo too. And that's just a brief wrap about the hackathon itself. And uh yeah, we can just switch to our demo then.

Yeah. See you. Uh yes uh prior to us jumping to the demo I'll just quickly walk you through uh who we are. So we are prompt piper and what we have built is we have essentially built a CLI tool that would help you uh help you compress your prompts so that uh your context window you can use that fully. uh the AI problems, right? So, uh as you probably know, AI has uh sort of two sides of the same coin wherein on one side you have a bunch of people trying to uh trying to increase the context window through the models as much as they can.

Uh on the on the second side, you have uh folks who are trying to optimize the prompts itself. So we fall on the second side wherein we are trying to optimize the prompts so that you can take full uh so you can fully utilize the context window window that you get as you see uh you pay for the for for the number of tokens uh that you consume right so we try to reduce your cost through that uh so far we have two modes uh through our CLI uh and our UI uh the first mode would be the rule based mode that that we have also pushed on on the IPFS storage. The second one would be the AI

mode wherein we we have also uh created a a small model that could help you do the same thing without the rules. And both of these are hosted on IPFS and and free to use. Uh you can see some stats here. Uh based on a bunch of tests uh that we did with the help of uh weights and balances, we saw that uh the the the number of tokens uh and the the cost cost per token was reduced from 40 40% to 57%. As you can see here, uh this this CLI works really very well on uh on complex code problems trying to help uh reduce uh

the number of tokens you use through your prompts. Yeah. Uh finally we yeah we we we have pushed all of all of our models and our rules on uh on IPFS and we also have made our CLI uh open source and we have also pushed this on uh uh uh so you you can download this even right now uh by going to promppiper.com and you should be able to start using using this right away. So yeah a a bit more about how this this whole thing works. So we have uh three steps here. The f the first step we have is we have we try to uh get the large prompts uh and trying to trying to to

extract some context out of those prompts through the first step. the through uh on the step number two we try we we try to uh uh we try to annotate the data that we have by trying trying to categorize the prompts uh on the basis of if that prompts prompt is is for something linked to code or which code which language so so on and so forth and finally we have the the the trained model uh which we have built through that yeah jumping jumping to the to the demo right now >> yeah yeah So we'll show you two ways to use pump piper. One is through interactive tool uh the C cli that we have.

Let me check. So yeah we have um CLI tool and just for examples we have a couple of s simple examples where you can compress a pretty simple prompts and they also pass directly to cloth in this mode. Um, so we can can observe what was going to happen next, right? And uh we can also do more complex like passing a solidity contract that's going to be then stripped of let's say white spaces that also take tokens and then again we're passing to claw to analyze it and um save tokens and receive some answer if you want but you can also use it just in a dry mode. So you can see okay we got the answer and then we go back to

prompt piper and let's say check the stats okay so we save some tokens great uh you can also use this as a yeah we can also demo the UI UI demo um so it's available at pipiper.com and it's pretty much similar just you get more visual out of this you can choose choose which compression model you want to use. It's like a small language model that you can download in your browser and will work um on your machine.

And yeah, you can just use couple of examples. Um and then choose your compression rate. Let's try to compress. Yeah. So, pretty straightforward. And if you increase compression rate, you'll see your pink being reduced grid. And this could just work behind the scenes. And yeah, we can go like pretty hardcore with the compression here. And finally, you get to then send both prompts to um choice of your LM provider and see the results, right?

And um yeah, so we were testing that on a couple of models and for this safe of the the cost here like can see here the first prompt was uncompressed the other was compressed. So we see 50% reduction and the same was true for uh using let's say open router. Yep. That's prompt piper in the core. >> Yep. So, please do visit promppiper.com and yeah would be happy to h have any feedback after this. Thank you so much.

Thank you very much.