ZK14: Confidential and Verifiable Telemetry using Zero-Knowledge Proofs - Janwillem Swalens

Transcript

So um I will be presenting the work that we've been doing at Nokia um using zero knowledge proofs for confidential and verifiable telemetry. Uh so today you've probably seen many presentations already with different use cases of uh CKPs maybe in payments identity voting uh privacy preserving analytics ZKML verifiable compute etc. Um and in this talk I want to introduce a new use case of ZKP that we've been exploring which is for telemetry and I will discuss the problems that we encountered and how we uh solve them. So um in general our setup looks at as follows. We have a hardware vendor that manufactures devices and these devices

are installed at the site of a customer. So um the devices are owned by the customer and installed on their premise but the firmware is provided by the hardware vendor. Uh and so in our concrete use case Nokia we manufacture network equipment for example routers that are installed at the our customers which are telecom companies. Um now when an incident occurs um we want to collect telemetry data from these devices to solve the incident. And so that means that we collect lots of telemetry data like log files, uh system states, uh events that happened, other kind of operational data, maybe configuration parameters, etc. And those

are sent to a support engineer at the site of the hardware vendor who can then try to resolve the incident. And yeah, this telemetry data could also be used for security monitoring, maintenance, product improvements, uh etc. Now an issue here is that uh this telemetry data can contain sensitive data that the customer doesn't just want to share with the vendor and it can be sensitive for commercial reasons or for privacy uh reasons. So for example, these LO files can contain IP and MAC addresses which are sensitive or names of subsystems or even if our devices are connected to devices of competitors then we could learn information about

competitors uh which the customer doesn't want. So they want this data they don't want to share uh this kind of data. uh so the customer often defines a policy of acceptable data that can be shared or some transformation that must happen for example IP addresses must be rejected or it's only in case of a critical error that we will share some of the uh data um now to deal with this what we can do is we can have some sanitization algorithm that happens at the site of the customer so we first gather the data on a device of the customer and then we do some reduction ction or sanitization algorithm according to their policy and then the cleaned sanitized data is sent

to uh to the hardware vendor. But yeah, an issue with this is how does then the vendor know that this data was processed correctly here uh in the middle uh because in light of future contract negotiations, price negotiations, there's maybe an incentive for the customer to uh incorrectly process this data. So for the hardware vendor it's important to know that the data is uh authentic. Um and yeah this I'm explaining this in the context of Nokia but uh this is much more widely applicable to all kinds of telemetry um cases. Another thing we looked at in the past is uh energy management in your house. So you might

have solar panels, a home battery and electric car. These devices they uh collect information about their electricity production and consumption and the operator of the electricity grid is interested in this information to optimize the grid and to buy and sell electricity. But there's a a similar concern. Um this data is very detailed and from this data um you can deduce whether you're home or not, how many people are in your household, which kind of devices you own. So you want to sanitize this data before you share it with the operator of the electricity grids. But then on the other hand uh they want to know that the data was not

manipulated. So it might be in your interest to act as if some of the consumption happened during off peak hours instead of peak hours because this is cheaper. So the uh the receiver of the data wants to know that uh this sanitization happened on authentic data. Um so in general uh we are looking at trustworthy telemetry systems where the customer desires confidentiality of their sensitive data and computational integrity. So they define a sanitization algorithm that encodes some policy and this must be executed correctly. From the point of view of the vendor they desire authenticity. So they want to know that the input was not tempered with that the original device data was used and they also desire computational

integrity. they also agreed upon this algorithm and want to know that this exact algorithm was executed. Um now to solve this we designed a small protocol that relies on uh signatures and zero knowledge proofs. So what we do is the devices when they collect their telemetry data they uh sign it with uh signature and then here in the middle when uh the data is sanitized we also verify the signature. So in the ZKP we first hash the inputs we verify the signature and then we do the sanitization algorithm and this is proven with the ZKP which is then sent to the vendor. So here if we verify the ZKP we know that everything happened uh

correctly. Uh so this protocol already guarantees us confidentiality the sensitive data is sanitized and computational integrity and authenticity. So if you would tamper with the input data then the signature doesn't verify. If you temper with the sanitization algorithm or with the output, then the ZKP wouldn't verify. Um, so this already gives us our uh properties we want. But there's a big issue which is a performance. So in our systems, the kind of log files we have, they're really huge. We have some files that were are over 60 megabytes uh large which is more than a million lines of logs with outputs of commands,

diagnostic information, etc. And in general when debugging the idea is the more information you have the easier it is to debug. So we want all of this information. Um and so this is an issue uh when using ZKPS. So in this graph you see an example of the algorithm of the previous slides um implemented using the openVM ZKVM with different input sizes. And so if we have relatively small input sizes for example 100 kilobyte file we can prove this in 21 seconds but as we increase the size of the input the proving time also increases linearly. So for a 10 megabyte file it takes 26

minutes to uh generate the proof. Um and so what we did is uh we looked uh we looked at uh the different phases of um what is happening in the ZKP. So in the ZKP we first have to read the inputs and this bar is also some of the fixed um overheads to setting up uh the ZKP. Then next we need to hash the data. We need to do the signature verification. Then the reduction algorithm and then uh the verifying the output. And so as you see here uh the hash function is actually the most costly uh thing. But uh this is just using uh a normal rust library for uh

calculating a hash in a ZKVM. And actually um in most uh most CKVMs offer optimized instructions for calculating hashes. So if you replace this with the openVM specialized instruction for a SHA 256 hash, it already greatly reduces uh this uh part. And actually this is a very standard optimization. So this bar is the 21 seconds that I was showing on the previous slide. So on the previous slide I already did this optimization. Um then yeah next we see that the signature verification part is very expensive and so there yeah we uh made a small optimization to our um protocol

where we outsource a signature verification to the final verifier. So what we do now in the ZKP is we just hash the input and output it. We don't do very signature verification here and we just forward the signatures to the final verifier who verifies them. And by doing that we don't need to do the signature verification uh in ZK. And so that um just completely eliminates that step um as part of the uh proof generation. Um so now this is for a 100 kilobyte um input file. If you now go to a 1 megabyte file then it looks as follows.

Uh so what we see is that um now yeah the hashing is again becoming more uh expensive. This is the version with the optimized hash with signature verification and this is optimized hash with outsourced signature verification. Uh so actually uh so the hash hashing increases again in size because the input is larger. The signature verification is actually a constant cost between the different uh um uh input sizes because the signature verification is on the hash which is uh the same size and then the reduction now is becoming actually the one of the biggest costs as well as uh reading the inputs and so this is uh something that we're still

working on. So um the redection algorithm we do is uh we use regular expressions to match for example IP addresses and MAC addresses um and there is already uh related work that looks as efficient implementations of regular expressions in ZK. Uh so one easy thing we can do is just by compiling the regular expression ahead of time we already get the speed up. And then currently we're also working on creating specialized instructions in OpenVM for um uh matching regular expressions as well as doing an optimized combination of this with the commitment uh to the input and it can give us another speed up of about 15x of this um red uh part.

Um yeah and so then yeah um I talked about the time it took to generate the proof. Another concern um is memory usage. So the devices we're using often have limited uh memory available and certainly if you want to continue normal operation while generating the proof. So that's also a concern. Uh verification time and proof size is actually less of a concern uh for us. Um okay. So to summarize uh we have been looking at a novel use of CKPS namely for um verifiable and confidential uh telemetry where we guarantee confidentiality of the sensitive data for the customer and authenticity of the data to the hardware vendor and

computational integrity of a predefined sanitization algorithm. And this has required many tricks and optimizations to improve the performance. But we're still looking at more uh tricks and more uh improvements uh as we want to increase uh input sizes. Okay. So that concludes my talk. If there are questions then we do have time for questions. So any questions? Yes. there. >> Um, so you have a private key inside the machine in inside the >> Yes.

>> IoT and then uh and Nokia keeps the list of those. >> Yes. Yeah. >> And then the p public key comes with the thing and then >> Yeah. >> Now you can completely identify which machine was it, right? Because you don't do it in a snark. >> Yes. Yeah. But Yeah. So you you're losing privacy for the efficiency basically. >> Yes. But yeah, we assume that it's um okay for us to know because we know which devices we sold. So this is not really a privacy concern for the customer. Um so actually what they're more concerned about is um their network topology and who which devices are connected to which other devices as well

as then if they are combining from different vendors this kind of information >> and then is a snark that you're using right the ZKVM using a snark >> how come the proof says >> no this is stark based >> this is a snark no >> no stark with a So, >> but I'm just curious why the proof size grows. >> Yeah. So, as the as the trace of the program gets bigger, it generates bigger proof uh proofs. >> Yeah. If that shouldn't happen like at least in grow.

>> No. So, it's not snark. It's start with a T. >> Okay. >> And then the proofs get bigger. And so, OpenVM offers a way to make the proof succeed, but then that's an extra cost which is not worth it for us. I understand. Okay. Thank you. >> Did you uh consider just uh injecting as a hint the NFA path like the nondeterministic? >> This is what we're working on at the moment. Well, >> so instead of like reading reading in the input and then trying to pass it in the thing, you just put in the past thing and then ask it to spit out the >> Yeah. So um yeah because a regular expression can be compiled to a state

machine and to a transition table actually we can this is what we're looking at making this more efficient. Yeah. Okay. Yeah. >> Thank you very much.