#DappCon22 : Day 1 - Ceramic: Scaling Web3 With Composable Data, Joel Thorstensson

Transcript

hello can you all hear me how are you doing great all right it's nice to be in Berlin again I just came down from uh Denver actually where I was at the Dow conference and there um it's really interesting to see like what people are imagining Building without and really really see the purpose of like hey we need to actually make all the technology we're building uh really scalable I see that there's a problem with the Emoji so we'll see how the slides look um all right so I'm Joel I'm co-founder of three box labs and we're building ceramic and so I'll sort of dive into some of the problems we've observed and

seen over time um and how we can actually go about solving them so if you think about in blockchain ecosystem where where is your data stored and if you think about it for a while you realize that uh you have probably interacted with one or multiple blockchains and you have one or multiple wallets your data kind of ends up in the transaction history of all these different blockchains so we have like all this information available but it's segregated across these different blockchain ecosystems um when you have maybe different Wallets on like ethereum different DVM chains but also like maybe polka dot or Solana

or whatever um and you don't really have like a cohesive view of your identity this data usually contains like a lot of sort of Rich social uh definitely financial and other sort of reputational context that can be useful in a variety of different different applications and it would be nice sort of if we can compose this data together like use it across applications that are not only like tied to one blockchain ecosystem because web3 devs tend to love compostability like if we have a smart contract to be able to compose on top of that for financial things that's really nice can we do the same for data well I think we can if we can create a shared metadata layer that sort of lives across and between all different blockchains

sort of like a mycelial network if you're familiar with like uh mushroom stuff um so how do we get there uh or why do we actually need this sort of metadata layer uh doing what can we actually build with blockchains Beyond finance and the obvious thing that most people come up with is social media we really need a substrate for communicating globally that doesn't polarize us that's a big problem right now that like social media actually tears us apart and that's not really nice I think if we can solve that and also like start building platforms that

enables us to have like Civic collaboration and scales have a collaboration basically how we can contribute as Citizens to this the the health of our whole sort of societies uh so there's a cool project called Civic abundance by Richland Marquee basically they're building a dashboard for um a city and they sort of surface relevant metrics for the city and there's also other sorts of like more uh out there or maybe more future looking sort of ways to collaborate I have two examples here on these these graphs you can see in the middle on the top you have the tech trees from foresight Institute and what they want

to try to do is basically map out how we can build really futuristic technology so imagine you want to build a Dyson Sphere like what type of Technology do you actually need as precursors to be able to build the the core technology so you probably need way better space propulsion you probably need way better solar cells and other sorts of materials so they want to build trees for like different types of Technologies and so we can more easily collaboratively map out how we get to this sort of sci-fi future things below that we see the discourse graph and this course graph is really a way to um I should say it's a scientific discourse graph it's really a way to map out science and understand things so you basically have a graph of questions claims and evidence

and all these things right now sort of live in a web 2 context but these projects want to move it to like a decentralized substrate where it lives it is verifiable is censorship persistence you can see which user actually contributed to these graphs of knowledge and I think what's really cool if you tie this together with something like retroactive funding or git coin grants you can actually retractively fund people who contribute to large-scale sort of scientific projects online um and you can see sort of like how these things compose because you have uh maybe this this Tech Tree uh this this tree of like how you get to some technology needs the discourse graph to be able to like map out and understand

pose questions about the relationship between Technologies um so why don't we like we need authenticated actions we need essentially persistence why can't we just put this on the blockchain well blockchains tend to not be super scalable they're really good for financial applications and that's sort of what they're built to secure in in distributed systems they use something called strong consistency which means that essentially all nodes agree on the state at any point in time and of course this is great for for financial things like managing funds funding public goods using nfts but the fundamental limit of scalability in blockchain is the the block producer throughput and that's basically the node

that makes the block uh that that node has to process all the transactions that comes into the blockchain and there's different character approaches for trying to scale blockchains one is just like let's make bigger blocks so that's sort of what Solana what are we what Celestia does we like different ways of providing guarantees about the availability of that data there's also a proof of storage which basically is fine with like the smaller block but basically you can make deals with miners or other nodes in the network that hey store this big Bob of data but if you have like hundreds of millions of users that all sort of make small changes to their data at once you can't really put that in a blob transaction because each

individual user serves signs and makes their own transactions so still sort of hit hitting the limit of the block throughput so what can we do to get around this well let's look at web 2 applications how do you how do they scale like generally so to to your left here you see basically Facebook architecture you're going to call it Tau and it's just a I think it's mongodb no sorry it's it's mySQL database that's just sharded then they have like layers of cash and when you write this database it sort of like propagates through the layers of cash down to the core database but there's no like strong consistency so you couldn't build like a financial type application on top

of this but it's really good for like it's big social graph another thing on the right here you see sort of like a representation of an event Stream So in Kafka uh Kafka is like a distributed system for event streaming and you can sort of think as of a database a table as an event stream because all the actions can sort of be converted into the event streams and then convert it back into a table and what's really neat about that is you might want to build a table based on event stream but you might also want to like take some other action or build some sort of uh different view on the data Maybe aggregate a bunch of different event streams so the event

stream has a primitive at the base layer kind of makes it much easier to compose at that base layer um and so we taken some inspiration from these sort of things and we've created something called compose DB which built on ceramic ceramic sort of handles the base the kind of underlying event streams and making sure they're authenticated assigned by the user composed to be sort of Builds an index on top of that and it's exposed as a graphql API and so the way to think about compose DB is sort of like as a graph database that is sort of decentralized lives across

the internet and you can sort of choose which pieces of this database do synchronize and that you help um keep available so you can think of it sort of as a blockchain this is synchronized but you don't you only pick and choose which pieces of data you synchronize so the core sort of building block of composite B is a data model so these are created with the graphql schema definition language there is an example down to the right here you see that there's a proposal imagine your proposal for a Dao and that has a relation to a comment so you can have relationships between these different models um and they're also discoverable and composable so if I see this proposal and

comment um data models I can actually create a new data model that's like a like on a comment and so you can think of these uh if you see the graphic top right there is a model the yellow pieces and you can see each individual users documents that they've created that conform to the model so for example we can see that user3 created a proposal that both user 3 and user2 commented on and user want to use it to liked one of the the one of the comments uh so in this way you sort of have an open write system you're not actually need to have an access control system on the data model itself but anyone can write to this

um to the system so you can sort of like add the database at the indexing layer you can choose which which users you actually want to include in your data set and each user sort of has one event stream per model that they're interacting with so taking these data models into something usable in compost DB we actually have something called Composites that basically takes one or multiple models and makes it into sort of a view on top of this data so so node a composed to be node basically index a subset of models so we might only care about model 102 or model 104 and you can choose to only index these you can sort of take existing Composites

that you find on the internet and pull them together pull them apart and this allows you to like um sort of make queries across the models that you care about without having to synchronize the state of the entire network and finally one thing that we're adding to compostdb is the ability to sync historical data so one example of this is git coin runs their passport with stamps that they have for for the ground surround on top of ceramic so once they sort of like migrate to the new version of compost DB anyone will be able to spin up a node and replicate the data set that they've generated and anyone can spin up a node and actually just like use the git coin passport data

in their own application if they want to build their own sort of quadratic voting or use the simple score for something different and this is really kind of what enables the composibility I can choose to spin up a node index a subset of models and now get shared access to this data so how this works under the hood is that it's using event streaming so an event stream in ceramic is an independently verifiable event log which means that anyone can spin up a node synchronize this event stream and validate it without having to validate the state of the entire network this is very different from Smart contract blockchain we need to synchronize the entire blockchain to sort of see the state of one smart contract so here there's no

interactions between eventlocks that are sort of like independent the way you authenticate and to be able to write into one of these event streams is using your native wallet so like your metamask or some mod in another blockchain ecosystem and essentially uh I'm not going to go too much into details here but you can basically generate a session key and sign a signing with ethereum message and now the session key is allowed to write to your data streams and your behalf without you having to sign for every single comment or like you make in the system which is sort of like very much needed to give users a good experience these event streams are totally peer-to-peer you synchronize them you

run up and now you say like hey I want to see these streams you ask other notes in the network you synchronize them and now you sort of can get a validated or completely valid for yourself uh and one thing that I think is really interesting this so right now we use this event stream to build this composed DB graph database one thing you'll be able to do in the future uh is that we can potentially use this event stream to just build composable data pipelines so imagine you have um you could have like something that looks at blockchain data and runs on computation and outputs that competition puts that into an event stream and some other system that maybe take some real world data put that into event stream

and you can sort of have other um data processors that just plug in to use an event stream as an input and use an event stream as an output and you sort of have this verifiable history of like what the computations that was done over time I think this can be like a fundamental building block for generally a data Marketplace um all right so now that we have this sort of building box we have this composed bead which can act as like a scalable um substrate for for web3 applications and like sort of like a shared metadata layer across blockchains uh we still have this problem of like okay I have these different wallets um that

in on ethereum I have some data and I have some data on ceramic tied to my ethereum address and similarly maybe for like polka dot or Solana or what have you um so how can we actually like bring this together well we've been thinking the way we've been thinking about identity in ceramic is that identity is not really something that you get from an institution a government or whatever sort of like the kyc approaches someone says that hey this is a real person this is a legitimate person instead if you think about your identity as what it is in your real life it's generally your relationships with other people your relationship with the world uh and

how you interact with with both of those so why should it be different online so in ceramic we see identity as an emergent property based on your interactions and your data so to enable this we sort of allow users to link their wallets together if they choose for that to be public or for that to be private this sort of like allows you to build up contextual reputation you can look at the graph of interactions from one or a set of wallets and sort of compute some reputation score based on that in and get going what they're interested in doing looking at this graph of like well first connected accounts but eventually

over time more and more data in the ceramic network is to calculate some sort of like social proximity score and this enables them to do sort of much safer much much more uh stable quadratic voting and finally one important piece here is like we want to be able to do selective disclosure I want to be able to like tie all my accounts to one identifier keep that private and selectively discuss information to the parties that I want to have a look at um so I think I think this sort of like notion of composable identity is really what can bring the web 3 ecosystems that are scattered across multiple blockchains together into one cohesive thing that we can use as our sort of

uh main interaction point that we bring along into different applications all right so um if you're interested in sort of trying out compost DB we have a developer site called compose db.js.org you can check it out it's very new we basically publicly announced it today so if you want to check that out and if you're interested in building these sort of Highly scalable web 3 distributed systems we're hiring so you can find our community or go to our website in general thank you [Applause]

thank you Joe now who has questions come on don't be shy hello uh what would you do you have an idea what would be the best solution to have some kind of encrypted data and have a token gated access is that also something you have thought of or like for example an nft would basically if you're the holder then it would be

the condition for you to Grant access behind some yeah on a decentralized data storage is it some something that could be achieved or yeah I mean I think it can definitely be achieved so this is something that we've explored a bunch and it's there's no straightforward easy answer if you want to do sort of Access Control based on a wallet not a token then in certain wallets like metamask there's a hack you can do which is basically like sign a deterministic message and then use the entropy of the signature to generate an encryption key now this encryption key will live in your browser and so it won't be like super secure but at least you can have like Access Control based on wallet for certain other quality ecosystems like in

polkadot they don't have deterministic signatures so you can't really use this trick um but then if you want to get access based on an nft or some other token you need to use some sort of network and some sort of threshold encryption scheme so there's a bunch of different protocols building this sort of thing uh I think so let's see like we have threshold Network I think maybe entropy is building something like this lit protocol is also building something like this and so these sort of systems can do that but they're all sort of like um like yeah they're don't feel like completely ready yet but threshold

encryption is the way to do it and that's just a matter of like how do you trust the nodes that uh it is holding the threshold Keys okay thank you