Metal3 Meets KubeVirtBMC: Provisioning KubeVirt VMs Like Bare Metal

Transcript

[music] >> So from here we can start from here. Is it audible to everyone? Fine? So yeah, everyone. Thanks a lot for joining me today and it's great to see you all. And I hope you all are doing well. And today I'll be talking about metal 3 minutes keyword BMC provisioning the keyword VMs like a bare metal. So just want to see the quick show of hands of like how many of you here are from bare metal provisioning background? Anyone?

Anyone who are from Kubernetes and container background? Okay. Cool. So talking about myself, I am Anish Kushta and I'm from Nepal and I work as a Kubernetes engineer at Kubrocs Technology and I'm the first and youngest golden keeper from Nepal. So golden keepers are the highest level of honor from Linux Foundation for those who pass 16 certification. And I am also LF X 20 for CNCF incubating project Kubela and uh and you know, apart from the tech, I'm quite in music. I'm drummer. I play a bunch of sports and I really love to go

for the trekking. Mostly the mountain of Nepal and that picture is from Annapurna base camp and this is a QR code you can scan uh to see about my credibility regarding work. And uh um you know, if you want to connect with me in socials, then you can scan this QR code. So just a bit of history of bare metal provisioning. Let's say you know, um you bought a new bare metal server, then if you see the typical life cycle of bare metal uh provisioning, then you start with ragging, cabling, inspection.

You have to do a lot of configuration related to your network. Then eventually you'll install any ways on the top of it. And once it is ready, then you have to operate and maintain it over the time. And if I just talk about, you know, installing OS on the top of any server, you might have, you know, in during your college days, you have Windows laptop, you are you have, you know, installed Linux on the top of it. So, what you'll do is you should turn on that machine and um you probably have any bootable device and you'll start it will start and complete the installation and you'll restart that machine and then the next reboot you try to install the OS >> [clears throat]

>> boot that machine from that install OS. And now then your machine is ready with that ISO image or with that OS. And the problem here is it's requires the physical access to the machine. Now, it's really fine with one machines and two machines. Let's say if you are at cloud or if you are at data center, what if you have you have, you know, 1,000 of bare metal server? And and you would like to install the same OS in 1,000 machine. Then it's not a scalable, right? So, uh so, wouldn't it be cool what if I could automate all of the step remotely without touching uh the, you know, uh physical machine?

Now, uh so, moving forward, I'll just take the example of turn on feature um because you can do a lot of other operation as well. So, I'll just take the example as a reference of power on. So, the question would be how do you turn on this machine remotely? So, comes into picture BMC, uh the one at the middle as put it on the top of it is a BMC. That is kind of mini computer which is one processor and one of the coolest feature of BMC is it's always on and regardless of your state of your bare metal server. >> [snorts] >> And so, one BMC comes in one hardware.

So, if you buy new uh bare metal server, then that's comes with one BMC by default. And the uh the its main function is to let you to control your server remotely. Like, hey, I'd like to turn it on, turn it off, restart the machine and, you know, reboot this machine changing the boot order so on and so forth. Now, uh so, now the problem was uh installing the OS on 1,000 machine. So, people created something called bare metal provisioning tool, which is called Tinkerbell and Metal 3. Uh that talks to IPMI and Redfish interface. That's These are the industry [clears throat] standard protocol that is used to talk to BMC.

And you know, and now BMC is responsible to perform that activity. So, let's say if Metal 3 send a request of power off, it will reach the BMC as kind of operator and that is responsible to turn off that machine. And so, and your provisioning stack will use the BMC to automate that entire life cycle. Now, this is cool like then what's the issue with this bare metal provisioning? So, this I mean, let's say you know, you you would like to test your bare metal provisioning software, let's say Metal 3, against 100 environment. You might have 100 use cases. So, it means you need 100 BMC. And one BMC comes in only one

server. So, technically you need 100 bare metal server. And the cost is going to up. And you might have written the end-to-end test cases for your software, which is Metal 3. And you're running the test cases using GitHub action, which is you know, CI. And how you going to get the real hardware inside the CI? It's not washable right? And it's also you know, slow from the provisioning perspective. Uh and you know, it's the system of time for the bare metal takes more time and you have to deploy your IT team for the stragging and cabling stuff. And just make a note of this. Just remember this point because we're going to solve this problem in upcoming slide.

Now, >> [clears throat] >> entering the virtualization era. Uh you have bare metal server, you can create virtual machines on the top of that using hypervisor. And the problem with this VM is is also doesn't have BMC. Now, what if you'd like to test your bare metal provisioning software against this VM? You can't do it, right? So, the question would be how do you how you provisioning stack will turn on this machine? So, this problem was solved by OpenStack and they created something called virtual BMC and Suzie tool and they export uh Redfish and IPMI that your original BMC used to have and your uh existing workflow uh like metal three will be same. You really don't need to change anything.

So, let's say if metal three send the request of power off, it will reach the virtual BMC and Suzie tool and it already exposed same API that your original BMC used to and it uh intercepts you that API and convert that API call into deliver API call and that's going to perform that power off activity on that target VM. And it's one-to-one mapping. So, if uh there's 100 of VMs, then it will get 100 BMC. So, the all the problem that we're talking about in the previous slide need a real hardware solved because previously you are getting 100 machine. Now, you will just buy a one bare metal server and on the top of it, you will create 100 VMs and using fake VM C, you can enable your existing workflow. So, you are able to save the

cost of 99 VMs, 99 machines. And and you know, like um you can actually run the CI, the provisioning stack, provisioning time is very faster and uh and you know, like you really don't need to worry about um uh you know, that racking and cabling stuff. Now, entering cloud native era and we start talking about containers and to manage those containers, we have Kubernetes. So, >> [clears throat] >> uh so, Kubernetes and now we can actually run the virtual machines on the top of Kubernetes using KubeVirt. So, KubeVirt is a CNCF incubating project that allows us to run the virtual machines on the top of Kubernetes as a pod. And the problem with this KubeVirt uh VM is it doesn't have BMC.

Now, same question. What if you'd like to test your uh bare metal uh provisioning workflow against this virtual machines? You can't do that, right? And the uh and putting the existing solution like virtual BMC or Suzie tool, it's not possible. Uh because, you know, the pod launcher pod uh it it's an instance of your VM uh like KubeVirt machine. And in the same question, how do you turn on this KubeVirt VM? Right? So, we still love bare metals. There won't be any discrimination I mean discrimination in the bare metal I mean virtualization or in cloud native era.

So, introducing KubeVirt VM C. So, KubeVirt VM C is a cloud native VM C emulator that comes with operator and it's actually the middle man. IPMI and Redfish at the front and the same endpoints that your original BMC used to have and it convert that API call into the KubeVirt API call and it was born from SUSE Hackweek 2023. And your provisioning stack already listen to the same endpoint and your KubeVirt VM C expose the same endpoint for your KubeVirt VMs and there won't be any change required in the provisioning side. Your VM will look and act like a bare metal.

And behind the scene like if your provisioning stack send the request for power on or power off, it in back end it call the sub resource API of KubeVirt. Device boot order it pass the VM CR. For virtual media mount it use data volume and hot plug feature. And this is the architecture and if you didn't see the left hand side, that's the custom resource. Once you apply it in the custom resource, it's going to create a deployment. Deployment will create the pod and it also create the service. Now your existing tool will talk to Redfish IPMI protocol and it will reach the KubeVirt VM C pod that acts as a fake BMC and it intercept the request and convert it into the KubeVirt API call in the back end.

And it's going to perform that operation on that target VM. So, this is the demo and if you see the left hand side, that's the the life cycle of bare metal host and so once you install the Metal 3 operator um you have to create a custom resource which is Metal 3 host. It will go for registering state. It will inspect the machine. It will create I don't know collect the hardware related information and once it is available, it will provision that, I know, OS on the target machine and once it is provisioned, your machine will be up and running. And the right hand side, uh this is the so on the left hand side we have metal three.

In between we have keyword VMs as a translator and the right side we have uh keyword VM. And ironic will talk to fake BMC using redfish. API call will convert it into the back end API call and you know, uh for redfish virtual media mount we need data volume. It create data data volume PVC all those stuff for, you know, storage and everything. So and uh this is the uh demo GitHub demo. Uh I mean GitHub link for this demo. You can find all the script. And if I go to the demo part just to start video.

So uh so I'm just running the demo part. So you will get two scripts before you scan this GitHub. First is platform as a service that will you know, set up all those infrastructure related to run this demo. And once you this, you know, runs this script so it's going to run actual demo part. So, you know, it will uh install keyword VM C all those configuration and uh if you see on the right hand side so uh if you see the bare metal host, it is on the registering state. And uh this virtual machines over there, uh this is data volume that will be created by back end in back end and this is the uh pod. So metal three demo VM is a word VM C pod and word launcher pod is a, you

know, uh instance of virtual virtual virtual machines. And this is the logs of uh keyword VM C that is in intercepting that request and convert it into the keyword API call. So now it is the available state. Now it is the going for provisioning state. It is deploying that uh OS on the target VM. See, it collected all the hardware related information. Now it is installing the Ubuntu cloud image on the top of that VM. It got provisioned. Now, and you can see, you know, all those translation, it changed the boot order.

Um you know, and um power on activity, eject media, right? So, that is, you know, uh translated by Kubevirt BMC as a mediator. Now, I'll just run the virtual command and try to assess into that machine. >> [clears throat] >> So, uh Ubuntu is username and password is also username Ubuntu. Now, it is able to uh run, I know, see, this is the Ubuntu ISO, I mean, Ubuntu machine. And you can do all of this like a VM.

And this VM is running as a pod. And who created this VM? Uh the metal 3 uh software created the VM. I didn't do it anything. So, it automate the entire life cycle. So, So, uh just a bit of uh project trajectory. Uh uh the project was started uh back in 2023 at SUSE Hackweek. And we got invitation for Kubevirt representative, if you don't know. No.

Yeah. So, this is the independent project. We are applying for CNCF sandbox now. So um it was, you know, uh it uh invited by Kubevirt representative, and my original was accepted. It was also accepted as a Google Summer of Project. And now, Kubevirt uh adopted a different upstream project. Uh basically, we are about to donate this project to the Red Hat OpenShift team. So, and yeah, this is uh sorry to my co-maintainers, and uh uh both are uh from Taiwan, and uh both are, you know, senior engineer, and were the maintainer for this project, as well.

So, uh this is the QR code, and uh the first one is GitHub link for Kubevirt BMC, and uh CNCF blog, and the documentation. So, yeah, that's all from my side. If you have any questions then >> [music]