Interview with Juan Benet of IPFS at DEVCON1 in London

Transcript

Right now I'm with Juan who I suspect could just be the next Tim Berners Lee. Uh uh he's he's building a system called IPFS which is the interplanetary file system. It's meant to complement in HTTP in the beginning but then it might end up doing just a lot more than HTTP and uh be a replacement to it would also be a replacement to it. So Juan let's have your introduction first. uh my own introduction or uh so what's your background like how short uh short background I was a distributed systems researcher uh for a while like I studied at Stanford I um have been working on open source and knowledge tools for

quite some time and ran into a problem around the lack of a a good storage interface for the for the internet as a whole um like a good protocol that uh leverages the speed of the network where wherein it matters and that allows different kinds of distribution protocols to be plugged in. Um and I found that there's actually many many problems with how the web is structured today that we can fix. And I built a system uh that then turned into IPFS that solves a lot of these problems and we are in the process of deploying it widely in the network and a lot of people are using it and it's growing and it's great. I don't know. Is

that Yeah. So, uh, so, so for our listeners, we already have an episode from Juan on episode of Bitcoin. We recommend you to go and watch it because it's it's it's really good and it really clears up a lot of things about the future of the of the web in general. So, right now we'll we'll focus specifically on the combination of IPFS and Ethereum because a lot of developers seem to be building on it. So, u, Juan, what kinds of projects have you seen at at the DevCon? that combine IPFS and Ethereum. Yeah, actually a lot. So, um I got so I spoke on on Monday uh kind of about IPFS generally and I included some uh examples there. So, they vary between

uh applications that are built with Ethereum that have some other component outside uh and they are usually using IPFS to address all of their content and to distribute all of their content. Um there are applications that live on IPFS. So these are DAPs that all of the assets are stored and distributed with IPFS itself and they just connect to Ethereum uh indirectly. So or so they they're actually distributed through IPFS and live on the IPFS network. Um there is there are the most interesting use cases for me at least are when uh so one of them is when people are actually

using smart contracts that address objects in IPFS and when people when the Ethereum chain itself uh both the state the Merkel Patricia state tree and the Ethere the entire Ethereum chain can be layered over IPFS and can be made into IPFS objects and so that's something that uh we've been talking about and discussing I've been discussing with the core team uh for quite some time already and that is an interesting thing for the future. Um it's not clear how it would be done or whether it would be done but uh that is a very appealing way to uh layer the projects together and leverage the strengths of of IPFS for Ethereum. Um, one thing that Ethereum nodes would get is the ability to use to address

Ethereum content both um, contracts or state in the state tree or chain the chain itself like elements of the chain itself and create links that link into that chain that other IPFS nodes can understand uh, that perhaps don't implement Ethereum specific uh, stuff. So that means that as the IPFS network increases in size and as many more IPFS nodes exist uh applications can be built that move around Ethereum the Ethereum chain itself through nodes that don't themselves know anything about Ethereum and this is a really powerful way to get Ethereum to be used across a whole bunch of places where people may not want to directly uh create nodes and use them

but you can now send them an application in JavaScript that runs on IP fast node and then can move around Ethereum state which is cool. Cool. So, so, so let's let's break break down break down these two these two examples. So the first one is um an example where you have a contract like for example uh there's an artist he he releases um music and he has data about how much he wants to charge for that piece of music on the Ethereum network and on the Ethereum network somebody can pay this contract uh in order to compensate the artist. Mhm. Now the problem with the Ethereum network is that uh storage is extremely expensive

cuz uh every every piece of data needs to be stored on each node of the Ethereum network. So what you're saying is instead of the music file living on Ethereum, the music could itself live on IPFS, but uh the payment could be processed on Ethereum, right? Yeah. And the the point is like the link that you put on the chain is an IPFS link. So this is true for both Ethereum and Bitcoin and other blockchains. people are starting to put links to content that is uh you know large files that you don't want to include on the chain itself. Uh and these are not necessarily files. You could be this could be a large you know record system, a whole file system like whatever. It's just any kind of collection or bundle of data. Um and people are putting links to that stuff with IPFS because that guarantees

you a a link that is not addressed by location. and it's addressed either by content or by key authority which gives you either immutable content that will be the same forever or it gives you mutable content but only based on a key uh which is very different from the HTTP world right so if you put a a URL or URI into a blockchain it might not if you put a URL it may not survive you know 100 days or you know however whatever the mean lifetime of a of a website is these days um URIs are better and IPFS links are URIs Um, but it gives you a nice way to resolve all these links. Uh, because what people were doing before was just putting hashes into the chain and these hashes are just hashes. Like

where do you look them up? You don't have a a standard system to look up all these hashes and that's what uh IPFS is one use case for that. So in this particular use case, what what kind of interests me is the um so let's say let's say you have some data in IPFS and the smart contract actually wants to compute on that data. So um I don't I don't have a very specific example coming to my mind right now but it could be well there's there's a good example. So um Christian uh from consensus built persona and persona puts the profiles and like all of the data um on IPFS. So they have a set of IPFS nodes and they back up the Persona

content and but it's all addressed with IPFS. So you could add your own node and help back it up and eventually there'll be some incentive layer to get nodes to distribute that data as well. Um but it's all addressed this way. But now let's go into this scenario where actually a smart contract. So let's say let's say there's a profile data on IPFS through the persona system and the smart contract is structured in a way that u when a certain transaction hits this smart contract it wants to make a change to this data item on IPFS. So it wants to actually transform that data item from A1 to A2 and link it with uh with that.

First of all, can you do something like this? Uh I believe you can. Yeah, I think that uh so I I don't know if the smart contract as it is now can mutate any state beyond the state tree. Um I'm not 100% sure about about that. I was in I've been in discussions early on where that was a possibility but uh it may be that the AVM right now is just pure and there it can do no IO beyond its own state tree. Um but uh you could totally see that being a trivial addition like you could add certain up codes that mutate state on IPFS. Um but you can also reference the state tree as being itself stored on IPFS. Okay. And suppose a contract for

its functioning depends on some data which is on IPFS. So there might be a contract that does one action if this the gender of a p of a profile on persona is male and some other thing if the gender is female. Now if you have a contract like that uh it's not trying to change the data on IPFS but it's just acting on some data on that is on IPFS then u First of all, can you build a contract like that? Um, I believe that's the case. I I have to look carefully at how uh the persona contracts are set up, but I believe like the the addresses that are stored into those contracts are

pointers to uh are basically IPFS addresses. Uh so yeah, it can manipulate it's just a string at the end of the day, right? So you can manipulate strings. I don't think that the EVM itself can load data from IPFS yet. um it might be able to I'm not sure. So um yeah but this kind of kind of yeah so this actually leads to the other other application you're talking about which is to basically remove this barrier between the Ethereum state state tree and and the IPFS Merkel DAG altogether and have the Ethereum state tree as a part of the IPFS Merkel DAG itself. Exactly. that would be the ultimate thing because then uh uh a contract could basically refer to anything else that exists on the uh

on the IP address network. So that's an extremely powerful use case where you suddenly get the ability to reference any data on the network itself um and act upon it from the EVM. So the EVM can then create references to objects that are outside of its normal state tree um and that are linked. I mean you can think of these as like git repositories or uh even even uh other blockchains like you could address uh objects on on bitcoin or some other blockchain or the web itself right so there's a lot of websites that are now being distributed with IPFS there's people creating websites and web apps where the data models of the of the application are just nodes in the IPFS graph so you can

now have a way to address all of that content and manipulate it with uh with the EVM even if it wasn't initially constructed as part of the if it wasn't it wasn't initially an Ethereum application it can now become one which might be a powerful way of of increasing adoption so how would that work so in in in a sense today when I want to run an Ethereum full node I have to download G from GitHub and it it basically starts a full node it connects to other nodes in the Ethereum network fetches data so instead so so G would basically have an IPFS module inside it so when as soon as I start uh G um IPFS is initialized in the background and it's uh trying to

gather all the data and just as it pulls Ethereum transaction data from the network it also starts pulling in it can also has the ability to pull in other kinds of data yeah it could be whatever is is uh linked to right so you could potentially create something like um Bitub so BitHub was this program a while back that was a bot that would tip people with Bitcoin so it had a it had a fund and it would tip people with bitcoin uh for every commit that they did to a repository. So if the commit got merged then people could claim some reward and that reward was just sent to them as a bitcoin transaction and this was a program that someone had to run and you had to run this bot right but you could just do this as a smart contract on Ethereum itself like you

need to not run no infrastructure uh but then the interesting part would be being able to do this as just a contract uh manipulating the graph and checking the the commits themselves without those commits being part of the Ethereum state tree. They're you know part of the git and they're part of GitHub or whatever. uh but now uh the Ethereum the EVM could load those up and manipulate them. So yeah. So so so so in a sense right now if uh so what you're saying is right now it's like the smart contracts and let's say the git repository they are on two different systems. Yes. And when you merge these two into one giant one giant Merkel web plus Merkel Patricia tree then uh things like you commit something

and it gets merged and you get paid automatically start to open up. So what happened? So in the in the 90s right uh before the HTTP web before Timley created the HTTP web uh there were a lot of different hypertext systems uh and these were they you know there was a gopher protocol and even before that there were a whole bunch of other applications that were hypertext systems and some of them were connected through the network but they were totally independent and had different formats and so on and one of the major contributions of HTTP and HTML and the and the web model was to bridge all of those together and create one web and create one network that could address every single other system. And you know, there was a lot of work put into

creating the URLs and the URIs to make all of this actually make sense, make it all universal, make it all be able to talk to each other. And a lot of people started building just simple uh translation layers between HTTP and those other systems. And it worked out extremely well. And for and then for the longest time, we just had one web that everyone contributed to. Then kind of some cracks started appearing because HTTP is not the best uh transport protocol for everything. Um you know Git had to be its own protocol. Bit Torrent was his own protocol. There were a lot of other peer-to-peer systems that were their own protocols. Um Bitcoin had to again implement its own whole peer-to-peer things. Ethereum the same stuff.

So all these different webs are starting to emerge again. But these webs are different. They they they're not just hypertext systems like the old old days. These are hypert hyper media systems where the links are actually Merkel links. These are links that are addressed by hash. So they're content address links. Uh but all of these data structures and networks have the same property in mind. Uh so just like in the old days of the web, you could again create the same kind of unifying layer that can address and point to from one any one of these networks to any other network. And so we in a that's one of the big contributions of IPFS coming up with the right way to address just Merkel based authenticated data

structures uh you know git bitcoin uh ethereum bit torrent uh just there's a ton of this these and address them in such a way that you can create links from one to the other right and so why is it that a git commit uh doesn't have like a proper identity on persona right like you could have that uh or why is it that like a uh yeah, you should be able to like create a contract that causes people to become seeds on the Bittor network or that causes people to like distribute files on on IPFS, right? And so all of that could be possible if we start adopting like one common way of linking between the systems. Yeah. One we should have one web, right? Like we shouldn't have multiple webs. We should have one web where you can address

anything from any any other place. So that seems like a very very great idea. But so let's also say that IPFS is also taking a position on how data should be moved across uh across physical networks across the web, right? So you have this bit swap protocol to move ah uh bit torrent like based on bit torrent you you have a way of moving data across the web. Yeah. Well, we don't we don't claim that that's the only way. So the thing about IPFS is that it's a very modular protocol that there are some problems we have to solve, right? we have to be able to move the content and so on. And so we have some protocols for that. But IPFS itself is designed so that you can swap out all those pieces. So if you don't

like moving things with the swap, if you just want to use HTTP, um, or you just want to use FTP for example, you could totally do that. And if you didn't want to use a DHD and you want to use some other routing system, you're totally welcome to. And those are still IPFS nodes. And so we're actually taking all of the peer-to-peer stuff and removing it, moving it out of IPFS into its own project. Um, and we'll have a lot more to say about that, but it's going to be called lip p. And the idea is to create like one central place where we can write all of the different peer-to-peer protocols. Um, because there's a lot of use cases where uh, you know, one use case is not going to match every other use case and one solution to that use case is definitely not going to be the

same across the board, right? Some applications want to work globally and are fine using a DHD and so on, but other applications want to work inside of a data center and they need really fast resolution times and they need to be able to move content, you know, less than a second. Like really, we're talking, you know, tens of milliseconds and we're dealing with tens of milliseconds. You can't afford to go across the network. So, a DHD doesn't make any sense in that or like a DHD that's global doesn't make any sense. Um, and so for those kinds of use cases, you need to a protocol that is able to have modules that can be swapped out by the people building the applications on top of them that are using the the the node. And IPFS is such a system. It allows you it is well layered. Um, this

this is an idea that's very old like uh the IETF constructed a whole protocol suite that is well layered where each set of protocols can layer over other protocols, right? um you can layer your own application on top of TCP which layers on top of IP and IP layers on top of Ethernet for example or you could layer on top of Wi-Fi or you could layer on top of uh you know there's a whole bunch of different kinds of network networking protocols and so it's that same idea of saying let's pick one middle layer that's very very simple and that's just a way of defining the data to have Merkel links such that you can create a path traversal over them over that graph uh and then uh create room for all of these these peer-to-peer systems to be plugged in uh and for you

to be able to choose what whichever one makes sense for your application whichever one satisfies your requirements and constraints uh without forcing you to use one that doesn't at all. Um yeah so what that means in a sense is like let's say let's say 5 years into the future a better bit torrent is like something better than bit torrent is invented to move around data IPFS IPFS let's say let's say let's say the next thing comes out I don't know uh intergalactic something I don't know comes out to move data along that doesn't matter to IPFS or to Ethereum if they put all their state tree into IPFS because that new system can the old old way of moving data in IPFS can be

removed and you can swap in this new system and the the whole uh permanent web keeps going on without a Exactly. Right. without skipping a heartbeat. We already have this where there are groups of people making private networks, private IPFS networks that have different uh routing protocols. So they don't use the global DHT, they use either their own DHT that's that's um you know smaller or something or they use other routing protocols. Um and so these yeah discovery routing uh and distribution like the exchange part all of these are layers that people can write different systems for and plug them in easily.

So, so the what kind of future invention can uh so like IPFS today from the from the perspective of the peer-to-peer community it seems like a very great idea because it's like an like an this integrated system that can handle all of these peer-to-peer applications into one like what kind of future invention can can break this and say okay we are not satisfied with IPFS anymore and and something is it is So fundamentally that that would have to be a better way of storing data, right?

That well it would have to be a better way of defining a Merkel link which I mean there's only there's only so many ways like when when you look at the heart of IPFS it's just a very small format. It's a tiny little thing. Um and so yeah it would just have to be a different implementation that either if they just don't want to use a different format then that's possible but and it would have to be like a really good reason for that. I guess in a sense uh yeah like we we are different from the regular web in that IPFS data is not all HTML files that's one example so we don't use HTML style linking um but we do still use linking that can

work with the W3C and so on so we're still compatible um so in a sense these are these aren't systems that replace each other wholesale these are systems that start collaborating and and complementing each other and over time are used by different kinds of systems, right? So if you look at HTTP pages, many of them include links to FTP and many of them include links to Bit Torrent and many of them include links to a whole bunch of other systems and the whole point is to have a web that can address every kind of system out there. And this is not IPFS's idea. This is an old idea. This is Tim's idea about creating one unified web. we're just extending that unified web to a whole class of protocols that weren't uh that required a different kind of

distribution mechanism that were more peer-to-peer that were decentralized that were uh that used Merkel linking as the core basic idea. So yeah. So uh so one of the one of the components of IPFS that sort of came a little later is the idea is it was in there in the design from the beginning but that was implemented perhaps a bit later was mutability. The idea that um I can give a link to somebody like IPNS/ something and that always goes to a single page but that page could keep changing over time. Yeah. So basically the the core IPFS data structure is a Merkel linked

data structure which is immutable uh kind of like git underneath the hood has a bunch of immutable uh objects but you still want to be able and so you can hand out references to people that are uh that are always going to be pointing to that file and so nobody can change it. Those are immutable. Now, you also want to be able to hand out references and links to that point to a mutable file that changes over time that you know today is is a web page that like a blog, right? For example, like I could have a blog and I could have one message today and I give you the URL to that blog or the URI to that blog and tomorrow you can check it and there's another post there, right, without that name changing. And so the the point here is uh how can you do that in a

decentralized way? And so the the construction of IPNS is that it's a name system that instead of defining a new naming naming authority like so many other systems do we instead want to work with everyone and say you can follow whatever naming system you want and IPFS nodes will try their best to resolve it and so we'll have a bunch of them baked into the protocol. Uh, one of those systems is an SFS inspired key naming system. And this is kind of conflated with IPNS or it is a core part of IPNS, but uh the way that it works is that you can hand out a reference that is just to the hash of your public key and that addresses a pointer that uh to a file, right, to IPFS content. And so people

these are a authenticated uh and self-certifying links because when I give you that link because it's the hash of a public key uh when you try to resolve it the pointers that you find in the network like the records think about it in terms of DNS right if you were looking up for you know food.com on DNS um you run a DNS resolver that goes and talks to other DNS servers and gets you know some record that tells you where you know gives you an IP address for food.com um the DNS system uh has some authentication system but it's different from this one. So the idea here is when I give you the name it's actually the hash of a public key which means that the only people capable of changing that name must control that private key. So

it's a way of giving you a naming authority that is cryptography itself. So it's not mediated by anything except cryptography. You don't have to register any name. You don't have to like pay anyone or anything. you just create a public private key pair. Now the thing about that is that it's not human readable. So it's not a nice and memorable name. So uh you know that that doesn't it's not very nice. You have to copy paste these hashes. But the benefit is that you don't again you don't have to register anything. You can just create them entirely distributed. So you could have a regist you don't need a registry that's centralized. Right? So all name registries need require the a form of consensus to clear them. Right?

So whether you're registering a name through uh you know the the DNS registars or you're registering a name on Ethereum or you're registering a name on Bitcoin and so on, there is a central body that's doing the allocation of names. Even though Bitcoin and Ethereum are decentralized networks, you're still participating in a central system that allocates the name space. and and that is is good to be able to do human readable names, but if you don't require human readable names, you can create a pointer and just do it entirely decentralized and like really honestly completely decentralized. Uh you're just mediated by physics, right? And like that's a very kind of appealing way of of doing names, right? you don't have.

And so this is something you absolutely need to be able to do names um you know like we mentioned before across planets because if you had a registry um that took like syncing across planets before you could register a name then you would have like a huge lag time before you could actually get a name. I guess like DNS is actually so bad that uh that the DNS register system is so bad that is worse than if you had to like go send messages to Mars and back. But uh you know today you can register a name on Ethereum in like you know a block time right so it's super fast or I guess it's two block times I think maybe two block times but um whatever it's it's you can register names extremely quickly and you can also register names on Bitcoin as well and again it's super fast uh way

better but it's centralized and so if you wanted to have a blockchain on Mars or you know in some area that's disconnected from the backbone so you can't reach the Bitcoin network like then you couldn't register a And that's an issue, right? And so IPNS is there as a layer of indirection. And so the way that it's really used is that you take your human readable name and you point that one to IPNS and then IPNS points to IPFS content. Cool. So has that been implemented? Yes, it's been live for a while and people are using it. Uh we still are we're still not recommending that people rely on it yet because we are uh working on a whole bunch of optimizations around it and increasing that consistency and and a whole bunch of issues. But um it is there and you can play with it and you can check it

out. Uh and yeah. So what what new cool stuff will you deliver next for the IPFS system? Ah so I think most exciting is our completely uh JavaScript implementation. So today we have a Go implementation but we'll have a a fully full implementation in JavaScript that will be able to be dropped directly on a page uh on the tab of a browser. And so that way you can have zero friction to using IPFS. You can write a website on IPFS and ship it and then give people an address and that address will first tell them to download this JavaScript implementation and then load everything through IPFS but the user will not have to install anything like it'll the browser will just do everything. So basically I open a tab and there's an

IPFS node running on the background. I don't even realize it and I'm surfing a website on IPFS and I don't feel the change in system at all. Yes. Exactly. Zero friction. Zero friction. It's it's not as fast as if you had the implementation running on the browser itself, which is what we're also building. Um, but you actually need these two things. Uh, so you need a zero friction install and you need a a browser uh implementation. Okay. So, where can our listeners get in touch with you or or your team? Yeah. So, you can you can the website for IPFS is ipfs.io and you can come talk to us on GitHub. You can file issues there. It's really a very open and and uh active community. You can also and we have tons

of repositories uh lots of really interesting collaborations with lots of other groups. Uh and we have a very active IRC channel. So if you if you're old school like us, you can uh before before Slack took over all the centralized communities. Uh you can log on IRC and come to pound IPFS on free node and you can talk to us there. Uh, by the way, yeah, if you if you have a decentralized or distributed project and you're using Slack, you should think twice about that. Uh, that's true. Uh, but anyway, I actually really like Slack. The product is is very good. Um, I just don't like that it's a centralized uh system that doesn't have end to-end encrypted data and it's actually a huge honeypot for hackers that are going to come in and steal all your data from Slack servers.

So, uh, good luck if you're if you're if you're a company and you're using Slack and putting valuable data there, good luck. All right. So, consider this this as your warning. Yeah. Yeah. Exactly. Um, and if you're part of the Slack team, please like just give us authenticated end to encryption and let us host our own nodes and just have us pay you for the software. You can totally make money this way. Anyway, uh, so now the whole other thing. Um I guess one last thing to say is I have seen so I I came to give this talk about IPFS uh at Devcon because a lot of people are using IPFS in the Ethereum uh networking community. Um but I was extremely impressed by by just how popular it is.

I didn't know that so many projects were depending on it. And so that's that's really exciting and and cool. And so I guess as a message out to all of the Ethereum community like if you find any um if you have any features that you want any any um anything just please come come and tell us and we'll we'll improve the stuff for you. And um and yeah I saw like really cool all sorts of systems are are addressing stuff on IPFS like from the persona stuff that I mentioned uh people are storing like music and video files. People are building um DAPs that do like uh I saw one that was kind of like a Dropbox file syncing thing. um that used uh that use IPFS to address all that content. Um yeah, lots of really cool cool stuff. Uh

so definitely check it out. Check this out. Great great to see your progress one and and we'll have you back on the show again. Yeah, sounds good. All right, take care.