Flash Boys or Superforecasters? Prediction Market User Base | Ethereum Korea One: Genesis

Transcript

Hi everyone, thanks for coming to the talk. My name is Danning Sui, I'm a research director at Pantera Capital. Been with them for a year and a half. My whole career has been in crypto, doing data analysis research, used to work at Census, and then Flashbots, where I was looking at DeFi, DEX trading data, and MEV. So here today, I'm specifically trying to share some analysis and research which we did on the prediction markets, market growth, and also the user base, cohorts, and some MEV strategy we found out that is happening.

Should be pretty interesting. So this is the agenda. We want to use a data set that comes from the first half of this year, covering Kalshi, Polymarket, and Polymarket US, which are the biggest players in the prediction market landscape, landscape and discuss about like what are the key metrics that are reflecting all the key growth and economics and fee capturing on these platforms and to highlight this outlier about the Kalshi's crazy growth on crypto short-term markets and then we'll go to the user part and then the case study about the MEV. So to have a lay of land first of all to look at the whole markets for the prediction markets, who are the biggest players, how to see market grow, here is a

chart coming from Dash dashboard, data dashboard from Dune, he's the most famous data wizard gathering all these data sets. So a few things to lay out first, right, prediction markets is adjacent to the concepts of traditionally sports betting platform where DraftKings and FanDuel, all these players already exist, but there are a lot of restrictions to run a sports, sports betting business. It's regulated and ruled differently by different state governments in the United States.

So now there is this new license, DCM license, DCO license, where it defines the outcome of a bet as a binary contract. And so now if you acquire this license, you can run a prediction market business, which no longer is just a sports betting, and you, you have the federal level, how to say, rights to run across states. That's why a lot of people told me after this, Polymarket and Kalshi are the biggest, how to say, valued companies that are spending so much money on all ads and marketing as of today.

We can see that in this chart, the green is Kalshi and the blue color is Polymarket International. So Polymarket traditionally launched first outside of the US until they actually finally acquired that license. And you can see this light blue here is actually one thing I want to highlight today, which is the Polymarket US venue, which launched in the beginning of this year and has been growing tremendously across their venue. There's another third player that I highlighted, Novig, which is primarily exclusively focusing on sports betting. It's an entire portfolio.

They now have 2.3% market share. year. So that's roughly the landscape. So the first thing we highlight here is that the Polymarket US has been a new venue, has been growing tremendously with a lot of momentum. If you compare the Polymarket International, which is a very different setup, it is a prediction market that has an off-chain central limit order book where users can submit orders and get matched together off-chain, but the orders are getting settled on Polygon on-chain. And compared to many other players like Kalshi and Polymarket US, they're different, right?

Like all these CFTC regulated platforms, they are purely off-chain. So they have like, Kalshi has an off-chain, order book also is off-chain. That basically brings a lot of, how to say, differences of how you can access the data and analyze the differences. What data are accessible, for example, Polymarket International, because they're settling on Polygon, we can access every trade and we can access every user level, like wallet level data, right? But for all these US regulated platforms, there's no way for us to know user level attribution, which also makes wash trading and all these detection harder.

But to look at this data set, you can see the Polymarket International has been having a steady level of volume of $3 billion per month, measured by the taker volume, but Polymarket US has been growing upwards and pretty much like up only, by end of July they have around 67% of share compared to international, and per the data last week when I prepared this slide, the US venue is already flipping the total volume in September compared to international. They settled on 2 million dollar taker volume and international was 1.99.

And also another caveat here to clarify that, taker volume is different from when people say notional volume. Notional volume is a very complicated, non-standard term, every platform is calculated differently. So we try to unify here by checking specifically on a taker order in one leg, what's the total volume. volume. So here are the three things that I think are very important to look at when you look at the growth of the platform. One is the taker volume, measured in the same way. The other thing is open interest, right? Open interest basically means how much money are parked in this platform on these markets that are pretty much willing to place bets anytime.

It kind of has an analogy to TVL, but not really. But I think oftentimes, if you look at the turnover, which is actually volume divided by open interest, it can give you a really good signal if all this money are being put into the trading and the fee engine with a lot of frequency, and does that make sense, or is there possible that like a very little amount of open interests are generating way a lot of the volume, potentially can be artificial volume. So I think turnover is very good metrics here to look at, and so by comparing the three platform here, Kalshi is absolutely leading in many metrics.

They have, this is a snapshot for July the month, and they have around almost $1 billion of interest parked in the platform. Polymarket International has 400, Polymarket US has around only 100 million because they're the new platform as well. Kalshi has a $13 billion volume for one month, and International and US venue for Polymarket together has $500 billion dollar, so that's kind of roughly the scale, but one thing I want to point out is that when you look at the turnover number, Polymarket US actually has the highest turnover, daily they turnover around 0.6% every day, and why is that?

So I think one thing to highlight further to understand when you look at this platform is that these segments or distribution of different type of market category is very important, right? So a platform can have a lot of sports betting activity, and another one can have all just crypto price, Bitcoin up and down prediction. Another one can have a lot of geopolitics and like Trump related, like election thing, and all these different market category have very different participant and different behaviors and how active, how seasonal these like volume can behave.

So when looking at Polymarket US we realize that pretty much all of their volume and open interest are sitting into sports and gaming. So I believe that's a growth strategy to focus on growing sports as a category as they can identify that this is the highest turning and fee creating and generating category. So pretty much Polymarket US is a sports betting platform and that's why it has the highest, the turnover rate. The other category that has really high turnover rate is the crypto betting markets. So to briefly explain what is the crypto market, you can basically go to Polymarket or Kalshi and say, I'm going to bet on in 5 or 15 minutes, Bitcoin price is going to go up or down compared to the start time of this

market. And there's actually a very interesting case that we're going to share where people are trying to manipulate this kind of market. But to look at this chart, we observed that for Polymarket International, where we can account their total fee actually collected in the platform, 98% come from these two categories. So crypto and sports. World geopolitics actually has a really high amount of interest sitting there, but they intentionally kept that market for zero fee, so it's also a very interesting choice by design, where they really are trying to charge, the capture the value coming from these two market category while having the world geopolitics become more of like an entertainment value for users to keep them sticky.

So when we look at the same chart for Kalshi, we notice the very big outlier data here. So if you look at their sports, crypto, and world geopolitics, it also showed that a lot of the open interests are sitting in world geopolitics. But meanwhile, crypto is actually very little in terms of like open interest, but it does actually show six times turnover daily compared to the volume is generating. So this is pretty crazy. When you compare the growth of the crypto market volume this year across Kalshi and the Polymarket, you can see that this green chart is just going crazy.

It divides into different durations of markets. So people can bet on the Bitcoin price trend in 15 minutes or 60 minutes or even longer. Or on Polymarket you can bet on for 5 minutes, 5 minutes, but all together you can see this trend where Polymarket had a peak volume around March, April, but it just has been pretty flat afterwards. Meanwhile, Kalshi has been volume generating like crazy. People are starting to be very sceptical about this. So I'm just not sure how many of you will follow all these Twitter discussions. In the very recent week where this analyst of Bending, he kind of pointed out, hey guys, if you look carefully into the filings that Kalshi is submitting to CFTC, you pretty much can see like they are actually providing, because they have

to disclose all this because they are regulated, you can basically find out that actually they are incentivizing makers for the exact amount of the fee that they are being charged, so pretty much the maker takes zero cost to, to trading back and forth. So that was a thesis that people were saying, oh, actually a lot of this volume are pretty artificial, potentially wash traded. It's not exactly like wash trading, but it's like zero cost, inflated volume. So that was a big revelation, million people looked at it, and it's a big debate, but to caveat as well, this filing was talking about specifically Kalshi's perp volume, but people are suspecting that they're pretty much providing the same incentive program for makers, right?

For the crypto trading point. And this pretty much incentive behavior is very, how to say, very non-sticky, right? So when we looked into why Polymarket's volume peaked in March but contracted pretty much after, we realized also this is in line with what happened in their incentive program. So there was a maker rebates that is saying, hey, if you're trading this amount of volume, we'll rebate you back this amount of fee. So you can see that the fee was pretty much encouraged, but when that incentive program changed, the volume was pretty much gone.

So all these volumes are not sticky. This is incentivized, very elastic. This is good learning for us. When you design a market and understanding where are these economics are being driven by, this is an underlying story that basically is being revealed, and yeah, so we kind of observed all these behaviors and understand that if it's incentivized to make a volume, it's not gonna be, it's gonna be pretty personal, and this is like in line with the spirit of MEV and arbitrage as well. Kalshi has responded and saying like not really, but I think it's like a, it's really hard to deny there's artificial situation going on.

The second part, I want to go over the user-based analysis. One thing to clarify first of all is that, as I mentioned earlier, because all these platforms are pretty much off-chain, we really depend on whatever data API they expose to people for us to analyze. Polymarket US, Kalshi, for good reasons, they don't want the user level information being exposed and being analyzed, so people can maybe like front-run people or arbitrage or like potentially have a malicious way to extract value from other users. To protect the user privacy, they are not really releasing any user level data.

This part about the user-based analysis is purely using Polymarket International. This may not represent the whole markets, but I think it will still be very interesting. So the first thing we did here is to divide them into different cohorts using a bunch of behavioral heuristics. So if this user is trading over more than 100 trades a day or more than like a thousand different markets, it's more likely automated bot behavior rather than a human being clicking every day right?

And if this person is trading for a 50K bigger volume within a very specific concentrated category, we believe that this person think they're a professional trader or like informed player, so they're like focusing on potentially having a strategy for a specific vertical, we call them like informed or professional trader, and the rest of everything is the retail producer, and so by this cohorts definition we pretty much found the 80-20 rule here, so we can identify that across almost 2 million wallets that was active for the first half of this year, 89% of them are identified as retail behavior.

But these people are only generating 20% of volume. And so pretty much like 77% of all the taker volume are actually coming from bots and pro. Which is actually very small, like 10% of the total wallet account. The other thing is that we can pretty much identify that bot participation differ by market category. So the highest participation or penetration by bots participation is the crypto category.

There's a lot of wallets that are trading bots-alike behavior, all these like Bitcoin price, Ethereum price up and down prediction markets. There's also bots participating in the sports category. There's also the third largest category with bots participation is weather market. So weather markets is actually pretty straightforward for you to automate because it's always using the same API to determine for a certain city what's the highest temperature that ever happened. So it's very formatted and it's more prone to bot automation.

So very interesting here. Another zoomed out overall user accounting chart, for anyone who does data science, this is a very standard chart to look at user growth. We identified that all these different bars here represent, the blue is all the new users every month that is arriving to the platform, and the yellow part is how many users were active before, but then inactive, but then resurrected and coming back to participate again. The retained is the user who traded last month but also traded this month.

So they retained and stayed active. So these all together are monthly active users. And it looks like it's pretty flat. And the red part here is actually the user that I found who churned, who was active last month but then left this month. We pretty much can see that every month there are new wallets rolling in coming from all these promotions, ads, KOL, referral, but there's also pretty much the same amount of users are leaving the platform.

So it seems like it's a high-churn business, and it's kind of like taking a lot of marketing costs to really keep the user to stay in the platform. And when we look into how much people are losing or making money on the platform, like looking at the P&L aspect of the user base, we saw that actually even the bots and the informed traders, they were kind of like, like the median value are losing money. So altogether as a category, the bots are making, this is the total accounting for the first half of this year, are making a hundred million dollar profit as a whole.

The informed trader are actually losing money, but to caveat, this is a gross P&L. Now we're not accounting all these rebates and potentially incentive that they're receiving, right? So counting on that part, the net profit might be different, but um, retail are absolutely the one that's losing most of the money. Um, one other thing to mention is that even the bots as a category are making a lot of money all together, the median bot wallet loses 70. This is coming from a lot of like not so smart bots. Nowadays there are so many, I'm sure you guys have tried with setting up your lobster, having your autonomous AI trying to make trading activity for you.

And the first MCP being recommended is the Polymarket API. Try to make my AI agents make money for me using Polymarket datasets. So I believe there's a lot of those bots are coming from all these early experiments of AI agents trading activities. activities, potentially why they are also losing money. There's another interesting data is that if we track across every month, how many wallets are consistently making money. There is 146 wallets that seems like has been making money along the way every month.

We potentially believe that either they are super skilled or maybe they are insider trading, more like, there can be more study studied on this category, but this is a very interesting thing too. So here is the last part, but not the least, that I think is a very interesting story to share, the MEV strategy happening in the short-term market. So to explain in details what is the crypto short-term market, when you go to the Polymarket, you can see some sort of interface like, like this saying, okay, the market starts here and the price was 65K.

And your job is to predict that after this market ends, let's say in five minutes or 15 minutes, is the price gonna be above the starting point or below the point, right? So this is very interesting. And because Bitcoin price sometimes can be very volatile, sometimes can be pretty quiet. And first of all, we checked how many wallets are trading on this category and are they bots alike or not. It seems like 80 of the bots are trading against the bots, so it's pretty much like a bot war going on, first of all. Second of all, we noticed that if you compare the Binance price, like specifically Bitcoin price on Binance, and map the time stamp of the price trend with the Polymarket start

time and ending time, you realize that in the case of when Bitcoin price is pretty flat overall, you can see the trading volume on Binance spiked up 12 times compared to when the Bitcoin price are very volatile. This is really weird, right? Like so there's a pattern here that being matched with the Polymarket timestamp, and you can pretty much see that this pattern only existed when Polymarket's 5 minutes market were introduced. So what's going on here? You can also see that for the 5 minutes and 15 minutes market, the Bitcoin price on Binance right before and after the timestamp of Polymarket's market close is very flaky.

It will go up instantly, somehow matching at the ending time of the market, but then drop down instantly. Like the price will go back to the same level right before the second, but was somehow artificially pushed up to this some point. This is a lot of evidence to point to in hypothesis. So the hypothesis is that there might be a lot of people who are participating in these like crypto price trend markets who have a position and skin in the game already. They are expecting the price not to really move during the duration, and when the price is near the end of the market duration, they will just spend money to push the price up, to buy Bitcoin right before the market closed.

So to potentially influence the final closing price and the results of the market. And right after the market conclusion is landed, they will instantly sell the Bitcoin. So in this case, they don't hold any exposure to actually the Bitcoin price risk. So to put that into a diagram, you can pretty much see that like, because people can see that, okay, Polymarket is using this Chainlink data price to determine the outcome of the market, what if I, you know, manipulate the results of this? So this is pretty much what we are suspecting happening, and there was another study coming from Stanford that kind of confirmed our assumption as well. The question now become what kind of market is more likely to happen with this kind of strategy, or

potentially manipulation. We noticed that naturally because when the price, when the market duration is shorter, the price will move less historically, then it's more likely people will have lower cost to manipulation, right? So for example, this chart is telling us that if you just observe across a period how Bitcoin price moves, you can see that within five minutes duration it's pretty much, on average it's only moving five bips, but when the time window become longer, like four hours in all 40 minutes, that was like way higher cost to manipulate. So naturally shorter market will be more prone to manipulation. That's one conclusion. Second of all, you can also like just looking at the same thing but

different way is that like for five, five minutes market, 81% of them close to within 0.05% price difference compared to the beginning. This is the same story but it's totally in different metrics. This study I believe kind of informed the Polymarket platform to make a difference, make a design change in their platform, and now they're moving towards a TWAP benchmark. So they're not only just sampling one price point, they're actually aggregating like weighted price across time within a short duration to actually determine the outcome of the price, determine the outcome of the market. So I think this is a great, great like improvement and informed the like research, and I'm pretty

proud that the researchers were talking about this, and I think this is a great outcome for the markets to be more resilient from potential manipulation and value extraction. So yeah, putting all these insights in one slide, I think we learned that sports and crypto markets are the fee engine, are the growth focus for a lot of these platforms to make revenue because of their high turnover rate. The user distribution showed that there's a lot of bots going on, there's the volume are mostly driven by a small amount of people, and lastly, I think market design can really shape behaviors and how the outcome of the platform, how fair a market, market participation can be.

So I think I would love to see more research and people to push all these Oracle solutions and prediction market, insider trading, and wash trading analysis to be better. So yeah, that's all I want to share today. Thanks for the time.