Scaling Ethereum RPC APIs with Amazon Web Services

Transcript

thank you hi everyone i am everton fraga i work for aws as senior blockchain specialist essay that's solution architect and that means that we help companies businesses individuals startups and businesses of any kind they want to migrate to the cloud and succeed in their uh workloads so just a little bit about me i used to work for the ethereum foundation previously for over four years i worked on the missed project so there was the dapp

browser also a little bit of the ethereum wallet and and some other applications like the grid was a tool for providing a graphical user interface for ethereum nodes uh and then lately i worked on the ethereum js set of libraries and well it is a big big pleasure to be here i wish i could go to to france but due to well all the travel bans concerns that i couldn't um so i'm based here in in new york city but i'm originally from brazil so let's get started here for our uh session today

we're talking about uh we'll talk about creating uh fast scalable rpc apis using the the power of you know amazon web services and and and and let's review the agenda here so first we talk a little about about latency and then concurrency and elasticity processing time caching strategies and then we can talk a little bit over how to optimize even further all right so let's get started just with some level set

on the nodes you have well two networking stacks that we can so to say you you can you have on the bottom part the peer-to-peer networking uh that works on um on parts 3033 that's basically for udp and tcp so it allows node uh discovery and synchronization there is also one other part but that's for boot nodes that's 30 31 and there's the on the application layer of what the users and applications would actually use are those the on the top half so you can see

85 45 that is the json rpc um endpoint you also have the websocket interface you have graphql as well and so we're talk we'll talk about this uh this top uh half so application so just some a note about connectivity here and how how does that represent your reality so let's talk about these rpc and latency from this rpc angle so it is basically mostly influenced

by the physical network route that those all those packages need to go through traversing the public internet until they need until they they meet your your actual note so here this uh map to the right we can see uh represented in a in white it would be your your ethereum nail like hosted in the us for example and then a user in south america trying to access it uh it would go over the public internet at first right uh and it is also of course influenced by the physical location of these servers so let's move on

here in this example uh we're picturing yes oh okay let me try again wow thank you i had some permission errors here from firefox let me try again okay cool yep okay we just got to the

graphical part so no harm thank you for letting me know okay so now going to this map representation here we have a node in in the u.s and we have like users all over the globe in this particular case what will happen regardless of the infrastructure provider you use by default all those requests being made they will traverse the traversing network the over the public internet making

several hops concurrent to many other information that that's traversing there uh this uh can can can provide some some more well added latency given um that you know it would be really competing with many many other uh people or applications uh so it was provides several azure locations uh not only about it's not only about regions but we're present on several parts of the globe so instead of having those requests being traveling the entire or half of

the world just to get to the node we could have those users accessing specific aws endpoints and then those requests would enter this in these orange lines those are real really faster connections backed by aws backbone so they got trans ocean cabling just specifically for aws customers so that really helps in these cases but then the okay we kind of help to solve the um how comp how you're competing with those

requests but then there is also i talked about the number two which was the the location of those servers so you can even optimize that further deploying your application to your ethereum nodes to several regions so here you would have a really simplified version of it because there are many many more regions but we're gonna we're gonna see in a few minutes uh but each user would access um their own uh the closest region uh and notes that they are deployed there

so just giving here an idea of how can we achieve that so introducing those you know to to services cloud front is a it's here and both services are here for quite a while um and so cloudfront it is a cdn but not only that so they're present in those edge locations and they take multiple origin types so it is mostly used for content deliberate networks so use it it is a lot with s3 that's the storage solution and but it also can can take several

other origin types so in this case servers or load balancers or even playing playing http urls a good news is that it supports both the well the rpc methods like uh in the http or websocket so you could achieve first these using cloudfront and even these as well and route 53 it is edible as a dns service so you can you can have it all working from a single url from a single domain so it would detect

it it allows you to to configure uh specifically what would be the routes for every region so um so a user in asia to make sure that a user in asia can access uh the content really really fast from from apac regions so japan singapore or you know any closer region and of course both services offer some security capability so um so ddos protection and several other rule-based access control and several other security features and this is uh just a little bit about

the global infrastructure so the charts before were really simplified but here we can have a better idea of now what are the edge locations the points of the points of presence for aws and where you could access that you could place your your servers there so the orange circles are the regions where most of these services are offered and the blue and purple dots are edge locations so there are a cell a selected services that are offered there so you can place those uh cloudfront distributions and then whenever a user

accessed that edge location you would have a high speed internet braille to access your your servers so that's you know pretty interesting for for the latency part just as a curious fact this is how it looks like in europe so it's got pretty pretty dense all right we can we can talk about concurrency now so it is basically the concept of how many requests you can you can handle uh at the same

time so let's say let's now look at like just like one region if you deploy your application to a specific region if you have well just one node it would have a limited capacity when the capacity overflows well the your node will be more exhausted it might it might be with a higher cpu usage excuse me or with the load average will be skyrocketed making making every request any requests take longer so we do not want to exhaust our nodes so what we typically do is to deploy another one

and then you can scale or horizontally to meet the needs in the user demand this is typically achieved with this mechanism here so we have first a load balancer that would um that would distribute the load to one or more servers but then when you want to just add a new a new server to this to this region we can just use this what we call auto scaling group which is a set of automated rules we can you can place in order to determine when

should we spin up new servers or shut them down when when you have lower incoming requests so it is really solving kind of two problems one is the cost savings because uh you're not provisioning you're not over provisioning resources for your use the second is adding the the elasticity because it meets your your api demands so we can have a brief spotlight here with those two services so you can well with the load balancer you

can distribute traffic among multiple targets it supports multiple availability zones for failover so availability zones are an interesting concept so inside a region let's say us east one there are multiple availability zones that are pretty much data centers those are interconnected by really high speed fiber connections uh and then they are used for failover so should anything happen with one region um the other one would if you configured it properly so you could you could route your traffic

just to to the other one that's uh that's up so these different data centers they use uh well even redundant power they they use different different services different utilities so they shouldn't be affected there shouldn't be more than one availability zone affected in the case of now i mean anything bad that happens and these auto scaling groups those are based on rules so you can deploy or shut down those compute instances uh with the idea to maintain your performance and again only pay for what you need so you wouldn't

end up with a huge bill if if you just left those servers running let's say in the weekend or during the night there's one thing here though um when you deploy new servers given that we are an ethereum world you have to sync those nodes or at least get some backups and restore them um and by experience we've seen that this thing can take a while so for sinking um due to the incredible work by the gas team and and several other people involved they can achieve with snap seeing something under six five to six hours you can sync a

full node but then that it might be a lot for depending on your business needs and if you want to just restore a backup that you're making yourself uh it can also takes quite some time and and then more operational burden so you would have to manage those backups automate them and then sometimes you would need to stop one server take the backup save it start elsewhere and then you know turn that server on again turn that gath on again so there are several you know considerations uh on why you would operate your own ethereum server this way

so here we have the the this service called amazon managed blockchain it was launched in a few years ago but first it was only for hyperledger fabric that's a private blockchain and this year we had the general availability for ethereum instances so it is a really really easy way to deploy full nodes uh using amazon web services technology so from from the console you can you can just you know click on launch instance select your network select your instance type choose the availability zone and and that's it when you create when you

click on create it would take like um under 30 minutes for test net and the one hour for for mainnet then you have a fully synced ethereum node at your disposal so it is fully managed and that means uh you as as customers don't need to worry about um managing the storage of that instance or managing the uptime of it so there is a the service takes care of these things for you the same thing with um patching software patching um updates to geth node so we have an upcoming hard fork

and aws team is responsible for that and will make sure that everyone every node will be updated on time so it is available in several regions so it has it has support for for one of those architect architectures i showed you on the map and there are also several instance types available so you can have or general purpose using the t3 family compute optimize use the c5 family also the memory optimized using the m5 family and they are well they're easy to integrate with other aws services so they're like a first class citizen

within the eda blast family of offerings so they're they they provide proper end points for you so when you spin a note i you would you would have access to an http in in a web sockets endpoints so you can use them behind a load balancer or expose it to your application so really you choose how to operate moving on let's talk about processing time of those requests well it determines how many requests per

seconds per second a node can serve if well that's that's an obvious statement but i guess that's a good start so you can have uh several use cases right so you can have cpu bound rpc methods or use cases some other use cases might be more memory bound or network bound so it is really really important to identify your application needs do several benchmarks so you can really understand what is the specific instance that you need you can make a big difference

especially in in the billing right so if you're using uh an instance that's not ideal for a use case uh you might need more of that same instance than that might cost you so uh as i mentioned before on amazon managed blockchain you have the compute memory uh optimized uh but this is actually a general concept so you will find on the ec2 page that's the elastic compute cloud service you have a whole list of instance type so you can you can see if you have a really large need for networking you can check

there there are many uh instance types and sizes available for for networking intensive workloads same thing with memory or i o so that's actually also important uh so when you're syncing your ethereum node uh well it would require lots of io so you can have um so by having a i o optimized instance you can really take advantage of it and it makes a real difference for you to use that so let's get to this uh tricky thing

called caching by now we have already just a crop yeah okay this okay yeah but at this point we have already talked about latency uh we talked about concurrency and elasticity uh so let's let's take our api as an example so it is already distributed in several regions it is already it already provides low latency access to to my users it auto scales based on demand uh but what else

should we should we aggregate to this to this architecture so uh it would be interesting if we could make all that using a cost efficient way um so caching really really helps with this job and also helping [Music] helping with the request processing times so you can have like depending on the service mix you use you can reduce those requests by an order of magnitude so caching is kind of tricky on ethereum land so let's talk about that in a

minute and by some observations i could see that most requests will be from 8th calls that requires evm processing and then by the end of the presentation i'm going to talk about almost by the end of i'm going to talk about ways you can optimize that so first question is what can a cache on ethereum right so the easiest part is the well the static the static queries so you can have like protocol version syncing

um net listening if that's relevant uh information you want to expose to your users as long as p account or not version and well chain id chain id will be the same for the the same url and point you're using so you can safely assume that you would be it would never change so you can you can pretty much have a static cache for that something that will kind of never expire so we'll have a really really far in the future expiration date then you have some other uh let's say immutable

you know quote unquote here results so you can have blocks by number block transactions the account of transactions by block hash and then while transaction by hash and then the list goes on i put the quote unquote here for a very particular reason we've been experiencing lots of a lot of discussion regarding minor extracting value and um and all the re-args so it seems like they're

you know groups uh organizing to to create to push for reorgs to extract more value from from the chain and so from running on these things uh so in the event of reorgs you'd have to be really careful with how you're saving your cash keys here so and to have a good way to provide stale cache so you're always providing accurate information to your users so the these methods here they take like one parameter one parameter most then

uh if i'm not mistaken uh some of them will take the block parameters so so that adds lots more complexity but here those uh they take more than one parameter so it starts to get you know um dense in the in the amount of cash that you would save and then here we have the the category of what i cannot cash in ethereum uh so it would be you could even try to cash those things if of course it is possible but uh i wouldn't recommend you to cash

all these calls because those are um they would they are the byproduct of some evm processing and they have they might have well different calculations even for for the same even for the same input that you put on that youth call depending on the block number the eth call might might result in something else so in fact effectively you would not guarantee uh network a consistent response to your users estimate gas so that's basically simulating a transaction that's also cpu bound

there are ways you can try to simulate these estimate gas but it requires you know in advance what is the destination address what is the origin if they have enough balance so if you have an application that your users access estimate gas a lot like for the same token maybe you can even rewrite this method and try to understand instead of uh raising um the all the evm for every request uh you can try to decouple this estimate gas into some of the requests that that would

access let's say uh the origin and the destinations count balances uh check if the destination is a contract uh maybe if the destination is a contract of interest you might already have the typical gas value for that kind of a that kind of operation so you would also take the the method the data being being accessed so the method that that transaction would would access and then you could it could have with with some heuristic you could like just provide a compatible uh gas estimation for those users

well filters are you know stateful uh those are stateful so there some some considerations here uh the first is the filter is a node based so if you have a little bouncer with two ethereum nodes here and you first land on the first one your request lands on the first one to create a filter then you you'd have the filter id the next time you access that node asking for the filter results you would have to land on the same note that would be a good strategy

so there are two considerations on this uh the first one is that you can you can define some rules to have sticky sessions what we call so whenever you you do the second request the application here in the middle we can make a proxy or maybe a load balancer configuration for that it would identify what was the denote that you sent for the creation of that filter then they will route you to the same server to the same ethereum instance over and over again that of course if you have a pretty dynamic rotation of instances

let's say if you if you scale up and down several times during the week and you have applications that consume that data uh on a large period of time so you might want to maybe even abstract the filter functionality so you could save like let's say on a separate data structure uh what is the who's the client uh the user i'm making those requests and storing also the the filter id and all the information regarding that filter so the parameters so should anything happen with that with one specific node you could you

could via a proxy via proxy we can talk about proxy in in submits you could um just get that same information from this data store uh and then you would have information out of those parameters so the address maybe let's say we're talking about uh smart contract events filtering so you can have the contract address the topics you're looking for with the parameters and then the last block you had access so the next time you try to access from let's say an insta your client trying to access that

specific filter id you can look it up and see hey i think i don't have that instance anymore so your application can recreate that filter and then you can start from the block you stopped last time so you can create somewhat resilient filter mechanism using this same thing with with uh block filter painting transaction filter um yeah so these are you know the hardest methods to to to leverage from cash and if you want to really go on a reasonable fairly big scale

you have to treat them kind of one way or another right yeah so we have those states for queries filters uh we can also talk about mempool uh well memphis is shared across several nodes it is not the same every node would have their own version of a mempool and some services even implemented their own mempool so they create a really big shared tool for several users well

it is it is interesting because you can have uh consistent of the data so the same principle as the filters and but what would dictate that for your application is really your your specific needs right yeah so some considerations on caching you got to be really careful with uh stale cache so stereo cache is something that you save and then you fail to update that information and then but users would still request from that

specific cache so then you're basically serving outdated information to your users so that is kind of well that's non-ideal there are some strategies so instead of let's say using um creating one cache key for latest block so whenever people request information from from the latest block you you would just serve that key if you fail to update that you would only present outdated information but then your application can do something like request latest block that's a cheap query inexpensive query uh and then you would

update the uh have the cache key as the block hash that's important uh and let's say some identify like block and then the block hash and then you have all the information regarding that the very block so uh doing two requests might seem uh to your node from your proxy to your node might seem not ideal but it might be really faster in some time so we really got to do some some testing and benchmarks here if you have something like a filter for new blocks and then you

have these as a cache key uh that will be interesting so must handle uh reorg seems seamlessly uh this is of course kind of tricky as well because if you save let's say a cash key for block 1 million but it got changed and those by by a better block with a higher total difficulty then your users again will be seeing um outfit information uh so it is ideal to to use those uh block uh hashes as keys as much as you can instead of numbers

and then you can make those requests so you can even save let's say if you have an elastic cache or yeah if you have an elastic cache you can just put entire jsons of representing those blocks there then you can carry them by by hash or and then you can even have another data structure where you would save the hash to number or the number to hash association so whenever a user wants to to grab like what is the blocker number one million uh anyone to fat from cash you would use this as a secondary index where you would first request

the number to hash data structure and then you request the the actual block so this way you save on on storage for your cache structure uh it makes easy for you to keep consistency so so whenever there is a reorg there you would only update you would only need to update this this index comprising of the number to hash data structure you wouldn't need to update several block jsons and for block information because those be on on block hash keys and that you know

wouldn't make much of a difference so here in service spotlight we can talk about these two services so you have amazon elasticash uses memcached or radis as compatible apis so they they provide you some millisecond response times which means uh they are pretty pretty fast they're fully managed and scalable so you don't have to again worry about managing the software

os patching anything like this and well several applications that are that have a latency as a critical uh is critical to their businesses uh they've been using amazon elastic cash quite a lot so financial services uh crypto exchanges as well uh so people would extract the useful information and sometimes even pre-warm these cache structures on amazon elastic cache and then their their users would redirect it from it uh saving your your notes from exhaustion uh and even saving on uh it'll be providing you cost savings because

it is more inexpensive to make those queries against the servers than your own ethereum node or any other compute node the other here is alabos lambda a pretty interesting service that is a serverless compute unit which means you can just upload your piece of code it could be node.js python go java and then it would uh you can define the entry point and even the the destination of those functions and then they will run on this serverless fashion so it will also take care of provisioning

the the right compute resources for it it is easily scalable you could even have a provisional concurrency so you could you could already have some decent amount of aws lambdas just waiting for more traffic more concurrent access and it's you know again pay only for what you need and for what you use uh so it is built by the millisecond so it kind of incentives you to write you know more performant applications and you would only pay for what is fair for what's being used so there are some patterns here that you

can that you can do so you can use the uh the load balancer that i that i talked before let me just come back here in some slides you could use here like a load bouncer uh and then you could have your lambda function here in between okay cursor here uh you could have an elastic load balancer and then after it you could employ a lambda function uh at this limb the function could act

as as a back end of your public api and then you could even decide what are the cache keys uh send request them from elastic cache and survey requests in in a very performant way so there are some some other optimization uh strategies that worth talking about uh it totally depends on your use case but here i think that we covered all the five main points of the agenda we can talk about more on performance here so when i mentioned here on the east coal method um

one data that's worth sharing is from my observations most of them were to get token balances and on standard token representations the get balance method is just pretty much just reading from storage but then it needs to require us to load the evm and to process that the message by code and run those up codes until they get to you to the actual storage and return it this is more costly so on some experiments i had some interesting results here uh

so we could get like uh using less cpu we could get even over 20 percent more throughput using less cpu which means uh you could even if you have that on at scale um that could can represent significant uh cost and uh and processing savings so the idea of eighth cal optimization for a gap balance is to first identify which are the the contracts you want to optimize and then you could read the uh you could try to

uh or find the their on their source code to do some static analysis there are some tooling for that uh to find what what are the storage position for the get for the balances mapping and then that would that would be the exact uh and then you can calculate what will be the get storage at method that is a storage bound rpc method uh this one third from the bottom uh it does not this get storage it does not load the evm just go straight to the storage of that particular account based on the on the address you provide and the

storage key as well so you can effectively rewrite override the behavior of eth call into a that is an intense and cpu bound operation into a storage bound operation and then you can even use you can even cache get storage at results for a particular block let's say uh with that so if you get let's say in the most optimal scenario you would get a a east call you would translate it to guest storage ad without having access to any any data from the blockchain

then you would just check if you have guest storage ad in your in your cache structure if you have it that's awesome if you don't you would just perform one storage bound query so this way you can optimize it there are other techniques and ways you can achieve just by using these aws lambda as backhand or any other compute just to override some some of these rpc methods uh so one other example is you can create your own rpc methods providing useful information that

the default set of rpc methods do not provide by default so you can have a method called let's say everton underscore get token metadata so i've seen people doing similar things uh and then you can have well just the address as a parameter and then internally you would make the appropriate um the proper calls to the to the ethereum to to the chain and then you can have all the token metadata just like it could be symbol decimals and and the information you want there are other interesting rpc methods

that worth sharing especially if you work with tokens that is the the debug family of methods so you can pretty much have an insight on on exporting all the storage slots for a particular contract address and then you can maybe cache all these balances or maybe just export it for analytics purposes it would require the same kind of calculation for the gas storage at as i mentioned uh these storage slots did for let's say mapping down the low level uh data structures they're not saved

with uh with just the address of that account you want to look to look up it is actually a hash hashing function over the address you're looking for and the storage slot position and each token implementation can have this a different one so requires some knowledge of of some contract some specific contract implementations but it's actually not hard and the benefits can be uh really really good so you could effectively have all balances of a given token on your elastic cache providing like supe millisecond

access to it that might be useful for analytics and accounting might be useful for well for rpc access for end users or you can even instead of elastic cache pull them for for other data structures and other data pipelines there is one undocumented method on geth that is a two methods in fact that is the debug get modified accounts by hash or by number uh this is actually an awesome method i came across it i

recently just recently uh and what it does is provide exactly what are the accounts that got changed in in their account hash so uh storage balance nonce and code so every all of these four could of these four feuds could could incur in an account being modified then you can really more easily do cash eviction if you're saving let's say things on things by information by account uh you can really easily know when that account was changed and then

just performed there so for some cases that that that's just you know let's say for token contracts you have the the event and you know you can easily know whenever you have a an event uh regarding specific uh transfer uh but there are other cases that you don't have it so such as maybe uni swap asset prices on a talking pair or maybe a transaction of ether that landed on a specific account so this way you can easily monitor um and have this small list of accounts that got changed

for for every block that's really powerful or a block range right uh and then you can query this by blocked number or by block hash so those are two ways of doing that uh one consideration is that you to use these two methods you would need to have all those pre-images which are basically the recipe to create a given hash so i mentioned before on the how you can generate those keys based on account and the storage position of a contract

then having these pre-images being saved you can even know and do the reverse kind of the reverse hash to know uh what it had before and and just recreate that more easily so this is a pretty prerequisite for these debug methods so if you're running your lithium nodes well if you're running an archive nodes you probably have it already but if you're running full or snap sync or fast nodes uh you'd have to use a flag to enable pre-images recording you can also export the preimages database from another node so that would just

kind of instantly give these capabilities to regular nodes theoretically uh and let me see so there are several ways you can work with eye scaling your apis and just doing a review here so you can first of all just start with one node and and use it for start with one lithium node and then make use of those um networking services from abs to to have them on on high-speed internet rails so

you would have requests being fulfilled more and more quickly or you can have a multi-region deployment also you can deploy more instances being them elastic compute cloud like the ec2 or amazon manage blockchain for that and you wouldn't even need to manage those servers uh and then you can add more nodes or shut them down depending on your use so this way you can handle concurrency better then you can handle the requests per second for each node by caching or optimizing those rpc calls

so yes i think it would be it for today uh again uh it is a pleasure although i'm not there with you but uh i would really love to but it's a pleasure to you know i have the opportunity to to relay some information regarding it so thank you so much okay

okay okay thank you yeah thank you so much bye you