Semi-related question... I recently spent an evening trying to familiarize myself with virtualization tooling and my initial impression was that the CLI experience for kvm and hyper-v was rather clunky compared to let's say docker, gcloud, ignite and kubectl. Also a lot of the learning material seems to be oriented towards GUI. It could be that I just haven't spent enough time with kvm and hyper-v. I've spent a LOT of time on those other tools so maybe I'm just more used to them.
Is there good CLI tooling available for virtualization? I would say that I'm looking for the "on-prem" experience where I start by configuring a virtual network, download some unmodified ISOs directly from the operating system vendor, provision my headless virtual machines with a combination of Powershell, Answer files, Bash and Kickstart. I then provide services like DHCP and DNS using my virtual machines rather than the built-in mechanism provided by the virtualization tooling.
You're comparing VM tooling with container tooling, there's going to be substantial differences in level of abstraction. I might suggest you look at the libvirt platform, it's a wrapper around KVM essentially, but there's virsh which I've found reasonably straight forward to work with.
Yes, virsh is really a nice tool to work with. It has multiple times saved my ass when the graphical virt-manager refused to work (for whatever X11-reason).
I had some QEMU VMs running for a while on a home server that were more or less started by command line. You can specify everything as command line parameters.
I wrote a script that pulled some info from a sourced-in bash "config file" for a given VM (such as amount of CPUs, RAM, and where the disks were), executed the appropriate ip commands to create the taps needed for network access (including a private inter-VM network), and then built/ran the long QEMU command with it in a screen session.
Which file is the ISO for the virtual CD ROM is just another QEMU command line parameter.
I had my own bind running in a VM and created a view on the same subnet as the private VM network. Since this was only a few VMs I didn't bother with DHCP, I just statically assigned IPv4s. I used VNC to setup the OS in them.
For small installations or a single host, plain old virsh is probably what you are looking for. Point it to an iso and you're good to go. Not sure if it does any post install provisioning, but that's what kickstart and ansible is for.
Larger virtualization installations will already have a management layer, something like oVirt or vSphere, that has similar tools available.
There's a lot of useful command-line tooling for KVM- and QEMU-based virt. Here's a small selection of them:
• virsh — This[1] is libvirt's shell interface; and gives you access to the rich set of libvirt APIs.
• virt-builder — Use this for rapidly building minimal or customized virtual machines; it's greatly flexible; check out its man page[2]. And here's[3] a quick example that connects both virt-builder and virsh together.
• virt-install — Use this if you don't like the default build of the template images from virt-builder; it lets you create "headless" servers via 'kickstart' and Linux OS trees from the command-line.
• guestfish and libguestfs suite[4] — This rich set of tools help you in a variety of use-cases: repairing your broken disk images, editing, cloning, debugging disk images, and more. It has saved my behind a lot of times.
• qemu-img[5] – This Swiss Army knife lets you powerfully manipulate disk images (QCOW2, raw, et al) offline. Example operations include: create images, backing chains, offline snapshots, disk image merging, ability to convert disk images from one format to another, and more.
In terms of speed, how is virt-builder compare to say Terraform?
Terraform is dog slow sometimes due to cloud provisioning. Would be nice to just be able to build the VM locally then push the AMI/VHD into AWS or Azure.
I don't know how Terraform works, but `virt-builder` will cache the xz-compressed template images locally on your first pull. So for example, if you build a Fedora image:
Then the (xz-compressed) template Fedora 32 image will be cached under ~/.cache/virt-builder. So your subsequent Fedora 32 image provisioning will be much faster.
You might want to give it a whirl and see if it satisfies your needs; `virt-builder` should be available on most major Linux distributions.
virt-builder can usually build out a disk image in 15-60 seconds if the template has already been downloaded and you're running on baremetal. Might be 120 seconds if you have to use nested KVM or TCG.
$ virt-builder fedora-33
[ 1.5] Downloading: http://builder.libguestfs.org/fedora-33.xz
[ 2.3] Planning how to build this image
[ 2.3] Uncompressing
[ 8.6] Opening the new disk
[ 13.8] Setting a random seed
[ 13.8] Setting passwords
virt-builder: Setting random password of root to mZbPJw9d1ZHgowBk
[ 14.8] Finishing off
Output file: fedora-33.img
Output size: 6.0G
Output format: raw
Total usable space: 6.0G
Free space: 4.7G (79%)
virsh and/or lxc is all I ever use. All that cloud stuff, docker, kub-whatever is overkill to run a few VMs or containers at a small scale on-prem. KISS.
I stopped bothering with libvirt and other frontends for local dev environments. Now I use shell scripts to start my VMs. I only create one per month or two so it isn't that much of a hassle.
I second this! I have a shell script to download if necessary Debian, run its installer with setting matching production and then run VM with necessary development directories exposed to it. I tried initially to do that with virsh, but then I gave up. Raw qemu commands in fact rather straightforward indeed.
Yeah, if you have a few virtual machines for specific purposes, there's not much point bothering with some abstraction on top of qemu.
I also have just a few 20 line scripts to run my VMs, and that's all. Works with no changes/hassle for 5 years already or more. No extra SW to learn. Man qemu gives me all the answers. Easy.
I guess if you want to manage some complicated setup and dynamically add/remove disks, network cards, or whatever, all the time, migrate machines, etc., some solution like virsh would be good, but for having everything on one bridge, in one subnet, to run some throwaway VM with Windows/browsers for web testing, the simpler setup is so nice.
I'd recommend Packer and Terraform for this. Packer to provision the machine images from unmodified vendor ISOs and then Terraform to provision the network and VMs.
I include the blog link because there is some nuance in how to get the path right for community Terraform providers that aren't in the Hashicorp registry. The documentation on the GitHub project isn't quite up to date with respect to how the latest versions of Terraform expect the plugin paths to be set up.
I've done this pretty successfully with all the major Linux distros minus Arch, which requires some bootstrapping to get an iso that Packer can work with (no such thing as an answers file for Arch). It's not that big a deal, though. Just find some instructions on how to create and mount a cloud-init iso in addition to the installer iso and use that to add an ssh public key so you can script the installation steps externally. I actually think Packer can do this, but I just haven't gotten it to work yet and have relied on shell scripts.
Hyper-V actually has a very comprehensive PowerShell module that is pretty well documented, by the way: https://docs.microsoft.com/en-us/powershell/module/hyper-v/?.... I've found it pretty easy to use and actually got the Arch auto-provision working on Hyper-V in Windows before I got it working in KVM in Linux.
Another thing is you can just use the cloud images and cloud-init for bootstrapping everything pretty easily, even on-prem. cloud-init has a "no cloud" config option, as mentioned above, where you just mount an iso with the config data as a DVD drive and cloud-init will find it automatically when the distro iso boots.
This guy has a pretty comprehensive example of how to set up a kubernetes homelab entirely using the libvirt Terraform provider from Ubuntu cloud images bootstrapped with cloud-init: https://github.com/zloeber/k8s-lab-terraform-libvirt
I'm not as familiar with Hyper-V but certainly for KVM you would probably interact with it (indirectly) using virsh which I find pretty capable. No sane human should be interacting directly with the qemu command line if they can help it, it's written more for consumption by other software.
I use Hyper-v for years and I rarely create new VMs, this is why I use GUI, it is convenient if you don't do this often enough to remember the syntax and params. Most people that I know are in the same boat, but for mass deployments or automated deployments PowerShell is the way to go.
Sorry for the approximative English, I'm losing my vocabulary with WFH and I'm pretty sleep-deprived.
I've been building a game streaming platform for influencers for the past months. We scaled to 9k users in a month.
We have android x86 running on QEMU-KVM and there are a bunch of issues I need to address.
- virsh (and virt-manager) always add a PS2 mouse to the guest. The issue is PS2 mouses have a relative movement and not an absolute one -> When you use a mouse or the touch screen through VNC you have to move the pointer to a position instead of directly clicking/capturing the mouse
- My VM resolutions are 720x1080 but VNC always launches in 1920x1080. I have black rectangles on both sides of the Screen.
- Streaming the app viewport with VNC has some (small) latency, any way to fix it?
- Can we host an arm android emulator on QEMU? (armeabi-v7 or v8)
- We had a sales presentation from Canonical for their anbox cloud solution, but they never returned to us with a proposal. Is there any other way for us to have the Android VMs in K8S? Or even Lxc if possible, we are flexible. Also, the juju charms for anbox is nowhere to be found
If you want to get in contact, my email is jadiaheno[at]ludexgames.com
I can even show you a demo, we have influencers increasing their revenue with our Beat the boss campaign. We're not VC backed.
Hey I don't know how much help this is, but anbox is a very rough solution. The bootup time for the android container is on the order of minutes, and lits does not work.
You can try to get support in #anbox on freenode, though. There are postmarketOS people that know how to get the system working. I've never seen someone use anbox with qemu, though. It has always been same arch.
Hey Thanks for the reply. I will try to use postmarketOS on qemu. I'm wondering if there is any ec2 machine than can support (qemu-)aarch64, and also if AAA games can run on postmarketOS.
For the PS2 mouse, you can add a USB controller (best is XHCI, as it's more friendly to virtualization and needs little or no CPU usage in the host) and an emulated USB tablet. You can also use the virtio-tablet device if your VM has drivers for it.
Hosting ARM is possible but performance will be ~10 times worse.
- virsh (and virt-manager) always add a PS2 mouse to the guest. The issue is PS2 mouses have a relative movement and not an absolute one -> When you use a mouse or the touch screen through VNC you have to move the pointer to a position instead of directly clicking/capturing the mouse.
I have been using android x86, quite successfully even with touch input. When I did it I didn't use virsh, instead I just used ffmpeg directly via CLI.
>> - Can we host an arm android emulator on QEMU? (armeabi-v7 or v8)
It is very possible to run arm code on a x86 machine, this would however be slower of course.
If you are running on a VPS you might be able to get one with an ARM CPU that supports virtualization.
>> - Streaming the app viewport with VNC has some (small) latency, any way to fix it?
There would be other ways that you can use to remote connect to it such as spice. You might also want to consider running it on an headless X server and then use basically any remote control program.
>> - - My VM resolutions are 720x1080 but VNC always launches in 1920x1080. I have black rectangles on both sides of the Screen.
This sounds as one of the points in the changelog, maybe try to use the newest version?
You're being unfairly downvoted, so allow me to actually answer your first question.
QEMU doesn't have much of a GUI to speak of. It's a virtualization framework with some CLI tooling on top of it. Frontends/management UI's/etc are left as an exercise to the community so you can find one that suits your needs.
A recent Windows build used GTK3 decorations; I was very surprised to see that appear. You are right that it is quite low on functionality, its menu allows powering off and rebooting the VM, and gives access to the serial console as well as a command-line interface for the emulator.
It defaults to the GTK display: https://down.loaded.ie/Lb9RDi3.png. See the -display section in QEMU(1) for details and other options. My favorite is -nographic, which forces VGA output to the terminal; very useful when there is no X session or similar.
Edit: I realised GUI might mean creating and managing virtual machines; virt-manager might be good? Take a look at https://wiki.archlinux.org/index.php/Libvirt for the setup needed. (I think it's easier to learn to use vanilla QEMU, though.)
To pass files into VM I use lsyncd. It synchronizes changes to even big source trees over plain ssh and rsync. Of cause with big tree the initial copy takes time, but there is no security implications of exposing files to potentially untrusted VM.
QEMU does have a quite primitive GUI while it's running that lets you change disks and so on. But really, it's not a graphical emulator. You specify everything via command-line switches. (They're fairly intuitive, mind you. “-hda drive-c.img -m 16 -cpu 486 -soundhw sb16,adlib -vga cirrus” does what it sounds like.)
There are of course tools that provide a friendlier GUI wrapper if the command-line isn't your thing.
There's an app that wraps QEMU called UTM. It's open source but also you can pay $10 and get it on the app store.
I have had a lot of trouble with machines getting corrupted, not working at all, locking up, but I did get a Debian machine working that runs ARM Linux on my Macbook Air M1.
+1 for UTM - The website for mac (https://mac.getutm.app/) has some recipes and pre-built image for Windows and the main Linux distros. I had okay results with emulating Windows -- nothing usable in my day to day, though.
HVF does build with Nix (with an overlay), so the build experience doesn't suck. [1] for the base, then apply [2] to SLIRP if you don't want to use HVF's network adapter (which needs root). I'm happily running FreeBSD and Linux VMs on my Mac Mini with this
Can something like this be used to, say, build emulators for different game consoles? And do the popular emulators use this as a building block? Asking this as someone whose only experience with emulation and virtualization is running games on VisualBoyAdvance and running Ubuntu on VirtualBox
its not really meant for that, but there wouldn't be anything to stop someone from creating the hardware emulation, as far as I know. Still, purpose-built console emulator software is sure to be far better for the task.
Are both original Xbox emulators built off of QEMU (Xemu is a fork of XQEMU). I've only used Xemu, but performance was pretty good for the games I tried on it (it doesn't have a way to upscale rendering yet though).
You could build emulators for consoles, but QEMU is not cycle accurate. Meaning some operations may finish faster or slower than they would on the original console. Some games may rely on that timing for things like physics engines and break in weird ways.
Yes. xbox360 games running on xbox one often run with a custom emulation layer that includes a bunch of monkey patch fixes to make the game work right.
I have heard about QEMU for years. Something about virtualization and Linux, this is what I've started to associate it to.
My Workstation runs on Windows, and up until some years ago I was using VMware Workstation for virtualization, and then moved over to VirtualBox, which I've also started to use on a Windows-based home server. I've always used both of them via the GUI.
Now, if I want to build a beefy Linux server (headless) and run some virtualization on it, would I then use QEMU and get the same thing I've been getting with VirtualBox on Windows?
How do both of these compare performance- and feature-wise? Are they alternatives, or do they server different purposes?
Qemu is a whole-system or CPU emulator, originally for x86 PCs but now encompassing other systems and architectures.
Where the virtualization bit comes in is with kvm. Kvm is a Linux kernel module that lets you take advantage of x86 CPU hardware virtualization. Qemu can use kvm to provide the virtual CPU while it itself emulates the rest of the PC hardware, resulting in a complete virtual system.
That said, qemu can also emulate the CPU itself, allowing for example an x86 OS to run on an ARM system, or vice versa, but that of course is slower.
TSO is Total Store Ordering, which refers to the memory model of x86_64. For Rosetta 2, Apple will switch the M1 processor's memory model when emulating x86_64.
I wish you luck! I've been reading a lot about the MiSTer project lately and it is getting harder and harder to resist taking the plunge. There is some amazing work going on there lately.
For those that missed it, here is a recent article on the project...
Last I played with the coldfire and 68k emulations in qemu to try to get it to boot EmuTOS (an Atari ST operating system) I found a lot of the I/O devices lacking, even just to run headless.
It was sufficient to boot Linux in some configurations but not complete enough of an emulation to run other things. This is in general the story with many things in QEMU.
I didn't keep notes, I suppose I should have. I might revisit this again someday.
It's been probably about 8 years or so since I touched QEMU, but at the time the performance was unusable (on a modest system) compared to a very usable virtualbox or vmware. IIRC, I was just trying to run win XP on Windows 7.
How do things stand these days?
Also, given that QEMU is somewhat platform independent, how well does it run an x86 OS on top of an ARM chip?
If anyone knows how to improve the graphics performance on Windows, please share. Compared with other hypervisors like Virtualbox or ESXI, there’s no graphics driver for Windows VMs with QEMU.
Semi-related question... I recently spent an evening trying to familiarize myself with virtualization tooling and my initial impression was that the CLI experience for kvm and hyper-v was rather clunky compared to let's say docker, gcloud, ignite and kubectl. Also a lot of the learning material seems to be oriented towards GUI. It could be that I just haven't spent enough time with kvm and hyper-v. I've spent a LOT of time on those other tools so maybe I'm just more used to them.
Is there good CLI tooling available for virtualization? I would say that I'm looking for the "on-prem" experience where I start by configuring a virtual network, download some unmodified ISOs directly from the operating system vendor, provision my headless virtual machines with a combination of Powershell, Answer files, Bash and Kickstart. I then provide services like DHCP and DNS using my virtual machines rather than the built-in mechanism provided by the virtualization tooling.
You're comparing VM tooling with container tooling, there's going to be substantial differences in level of abstraction. I might suggest you look at the libvirt platform, it's a wrapper around KVM essentially, but there's virsh which I've found reasonably straight forward to work with.
Yes, virsh is really a nice tool to work with. It has multiple times saved my ass when the graphical virt-manager refused to work (for whatever X11-reason).
Just run qemu-system-blah. I thought the CLI for that was pretty intuitive.
You should try out vagrant, let’s you have a config file for your vm and you can choose different providers like libvirt, virtualbox, etc
You're looking at different abstraction levels. I'd make comparisons like: kvm:unshare, qemu-system:docker, kube:virsh.
If your want high level networking stuff, use virsh or virt-manager GUI. Or maybe proxmox if you want a whole distribution for it.
I had some QEMU VMs running for a while on a home server that were more or less started by command line. You can specify everything as command line parameters.
I wrote a script that pulled some info from a sourced-in bash "config file" for a given VM (such as amount of CPUs, RAM, and where the disks were), executed the appropriate ip commands to create the taps needed for network access (including a private inter-VM network), and then built/ran the long QEMU command with it in a screen session.
Which file is the ISO for the virtual CD ROM is just another QEMU command line parameter.
I had my own bind running in a VM and created a view on the same subnet as the private VM network. Since this was only a few VMs I didn't bother with DHCP, I just statically assigned IPv4s. I used VNC to setup the OS in them.
For small installations or a single host, plain old virsh is probably what you are looking for. Point it to an iso and you're good to go. Not sure if it does any post install provisioning, but that's what kickstart and ansible is for.
Larger virtualization installations will already have a management layer, something like oVirt or vSphere, that has similar tools available.
Look at LXC/LXD you get both VMs and containers with the same CLI.
How do you get VMs with LXC/LXD?
Also LXD's command line tool being
(with c instead of d) while LXC's being
is an abomination. Who wouldn't be confused by that?
you just add --vm to lxc launch
Whoa, crazy, I never saw that option before. Same interface for container and VM is really nice.
There's a lot of useful command-line tooling for KVM- and QEMU-based virt. Here's a small selection of them:
• virsh — This[1] is libvirt's shell interface; and gives you access to the rich set of libvirt APIs.
• virt-builder — Use this for rapidly building minimal or customized virtual machines; it's greatly flexible; check out its man page[2]. And here's[3] a quick example that connects both virt-builder and virsh together.
• virt-install — Use this if you don't like the default build of the template images from virt-builder; it lets you create "headless" servers via 'kickstart' and Linux OS trees from the command-line.
• guestfish and libguestfs suite[4] — This rich set of tools help you in a variety of use-cases: repairing your broken disk images, editing, cloning, debugging disk images, and more. It has saved my behind a lot of times.
• qemu-img[5] – This Swiss Army knife lets you powerfully manipulate disk images (QCOW2, raw, et al) offline. Example operations include: create images, backing chains, offline snapshots, disk image merging, ability to convert disk images from one format to another, and more.
[1] https://libvirt.org/manpages/virsh.html
[2] https://libguestfs.org/virt-builder.1.html
[3] https://developer.fedoraproject.org/tools/virt-builder/about...
[4] http://libguestfs.org/
[5] https://qemu.readthedocs.io/en/latest/tools/qemu-img.html
In terms of speed, how is virt-builder compare to say Terraform?
Terraform is dog slow sometimes due to cloud provisioning. Would be nice to just be able to build the VM locally then push the AMI/VHD into AWS or Azure.
I don't know how Terraform works, but `virt-builder` will cache the xz-compressed template images locally on your first pull. So for example, if you build a Fedora image:
Then the (xz-compressed) template Fedora 32 image will be cached under ~/.cache/virt-builder. So your subsequent Fedora 32 image provisioning will be much faster.
You might want to give it a whirl and see if it satisfies your needs; `virt-builder` should be available on most major Linux distributions.
virt-builder can usually build out a disk image in 15-60 seconds if the template has already been downloaded and you're running on baremetal. Might be 120 seconds if you have to use nested KVM or TCG.
By the way, interesting virt-builder factoid: It uses a STRIPS-based planner to optimize the order of steps when building the image, so it doesn't waste time doing multiple copies. https://en.wikipedia.org/wiki/Stanford_Research_Institute_Pr... https://github.com/rwmjones/guestfs-tools/blob/0cffcbb7848af... https://github.com/libguestfs/libguestfs-common/blob/74bc5c5... https://github.com/libguestfs/libguestfs-common/blob/74bc5c5...
virsh and/or lxc is all I ever use. All that cloud stuff, docker, kub-whatever is overkill to run a few VMs or containers at a small scale on-prem. KISS.
QEMU can be daunting at first (like ffmpeg, also by Fabrice Bellard!). But check out this 2018 guide to using QEMU by Drew Devault:
https://drewdevault.com/2018/09/10/Getting-started-with-qemu...
I stopped bothering with libvirt and other frontends for local dev environments. Now I use shell scripts to start my VMs. I only create one per month or two so it isn't that much of a hassle.
I second this! I have a shell script to download if necessary Debian, run its installer with setting matching production and then run VM with necessary development directories exposed to it. I tried initially to do that with virsh, but then I gave up. Raw qemu commands in fact rather straightforward indeed.
Another nice guide is on ArchWiki: https://wiki.archlinux.org/index.php/QEMU.
Yeah, if you have a few virtual machines for specific purposes, there's not much point bothering with some abstraction on top of qemu.
I also have just a few 20 line scripts to run my VMs, and that's all. Works with no changes/hassle for 5 years already or more. No extra SW to learn. Man qemu gives me all the answers. Easy.
I guess if you want to manage some complicated setup and dynamically add/remove disks, network cards, or whatever, all the time, migrate machines, etc., some solution like virsh would be good, but for having everything on one bridge, in one subnet, to run some throwaway VM with Windows/browsers for web testing, the simpler setup is so nice.
I'd recommend Packer and Terraform for this. Packer to provision the machine images from unmodified vendor ISOs and then Terraform to provision the network and VMs.
https://www.packer.io/docs/builders/qemu
https://github.com/dmacvicar/terraform-provider-libvirt
https://blog.ruanbekker.com/blog/2020/10/08/using-the-libvir...
I include the blog link because there is some nuance in how to get the path right for community Terraform providers that aren't in the Hashicorp registry. The documentation on the GitHub project isn't quite up to date with respect to how the latest versions of Terraform expect the plugin paths to be set up.
I've done this pretty successfully with all the major Linux distros minus Arch, which requires some bootstrapping to get an iso that Packer can work with (no such thing as an answers file for Arch). It's not that big a deal, though. Just find some instructions on how to create and mount a cloud-init iso in addition to the installer iso and use that to add an ssh public key so you can script the installation steps externally. I actually think Packer can do this, but I just haven't gotten it to work yet and have relied on shell scripts.
Hyper-V actually has a very comprehensive PowerShell module that is pretty well documented, by the way: https://docs.microsoft.com/en-us/powershell/module/hyper-v/?.... I've found it pretty easy to use and actually got the Arch auto-provision working on Hyper-V in Windows before I got it working in KVM in Linux.
Another thing is you can just use the cloud images and cloud-init for bootstrapping everything pretty easily, even on-prem. cloud-init has a "no cloud" config option, as mentioned above, where you just mount an iso with the config data as a DVD drive and cloud-init will find it automatically when the distro iso boots.
This guy has a pretty comprehensive example of how to set up a kubernetes homelab entirely using the libvirt Terraform provider from Ubuntu cloud images bootstrapped with cloud-init: https://github.com/zloeber/k8s-lab-terraform-libvirt
I'm not as familiar with Hyper-V but certainly for KVM you would probably interact with it (indirectly) using virsh which I find pretty capable. No sane human should be interacting directly with the qemu command line if they can help it, it's written more for consumption by other software.
For hyper-v you can use PowerShell to manage VM's, see the details here: https://docs.microsoft.com/en-us/virtualization/hyper-v-on-w...
I use Hyper-v for years and I rarely create new VMs, this is why I use GUI, it is convenient if you don't do this often enough to remember the syntax and params. Most people that I know are in the same boat, but for mass deployments or automated deployments PowerShell is the way to go.
Alright HN, I need your help.
Sorry for the approximative English, I'm losing my vocabulary with WFH and I'm pretty sleep-deprived.
I've been building a game streaming platform for influencers for the past months. We scaled to 9k users in a month.
We have android x86 running on QEMU-KVM and there are a bunch of issues I need to address.
- virsh (and virt-manager) always add a PS2 mouse to the guest. The issue is PS2 mouses have a relative movement and not an absolute one -> When you use a mouse or the touch screen through VNC you have to move the pointer to a position instead of directly clicking/capturing the mouse
- My VM resolutions are 720x1080 but VNC always launches in 1920x1080. I have black rectangles on both sides of the Screen.
- Streaming the app viewport with VNC has some (small) latency, any way to fix it?
- Can we host an arm android emulator on QEMU? (armeabi-v7 or v8)
- We had a sales presentation from Canonical for their anbox cloud solution, but they never returned to us with a proposal. Is there any other way for us to have the Android VMs in K8S? Or even Lxc if possible, we are flexible. Also, the juju charms for anbox is nowhere to be found
If you want to get in contact, my email is jadiaheno[at]ludexgames.com I can even show you a demo, we have influencers increasing their revenue with our Beat the boss campaign. We're not VC backed.
Hey I don't know how much help this is, but anbox is a very rough solution. The bootup time for the android container is on the order of minutes, and lits does not work.
You can try to get support in #anbox on freenode, though. There are postmarketOS people that know how to get the system working. I've never seen someone use anbox with qemu, though. It has always been same arch.
Hey Thanks for the reply. I will try to use postmarketOS on qemu. I'm wondering if there is any ec2 machine than can support (qemu-)aarch64, and also if AAA games can run on postmarketOS.
Trying it and i'll let you know.
Well there are native aarch64 EC2 instances! Those might be a good option.
Just used postmarketOS x86 on qemu. Anbox does not run the games, and there are unfortunately a lot of issues.
Will see if i can make some pull requests. Thank you for your help, it's appreciated.
For the PS2 mouse, you can add a USB controller (best is XHCI, as it's more friendly to virtualization and needs little or no CPU usage in the host) and an emulated USB tablet. You can also use the virtio-tablet device if your VM has drivers for it.
Hosting ARM is possible but performance will be ~10 times worse.
Rob herring has made a patchset for AOSP that allows one to run it inside a QEMU/KVM, using virtio device drivers.
https://github.com/robherring/generic_device/wiki
I have personally run this through KVM on arm64.
Hi, I'm in the Anbox Cloud team at Canonical. I'll get in touch via email. https://anbox-cloud.io/docs
Hey Galem, i had your sales rep emailing me. All is well.
- virsh (and virt-manager) always add a PS2 mouse to the guest. The issue is PS2 mouses have a relative movement and not an absolute one -> When you use a mouse or the touch screen through VNC you have to move the pointer to a position instead of directly clicking/capturing the mouse.
I have been using android x86, quite successfully even with touch input. When I did it I didn't use virsh, instead I just used ffmpeg directly via CLI.
>> - Can we host an arm android emulator on QEMU? (armeabi-v7 or v8)
It is very possible to run arm code on a x86 machine, this would however be slower of course.
If you are running on a VPS you might be able to get one with an ARM CPU that supports virtualization.
>> - Streaming the app viewport with VNC has some (small) latency, any way to fix it? There would be other ways that you can use to remote connect to it such as spice. You might also want to consider running it on an headless X server and then use basically any remote control program.
>> - - My VM resolutions are 720x1080 but VNC always launches in 1920x1080. I have black rectangles on both sides of the Screen. This sounds as one of the points in the changelog, maybe try to use the newest version?
Looking forward to testing, VNC: virtio-vga support for scaling resolution based on client window size
Static client window size is SO annoying! This is great news.
Is there any screenshots of the QEMU GUI running? I'm thinking of using it, but I am aghast that there's no screenshots page on the website.
Also: how do you pass files into QEMU? Does it have software similar to Virtualbox's 'guest additions' software?
There's virtio-fs, but it requires a relatively recent Linux kernel.
For Windows, I have samba running on my host.
You're being unfairly downvoted, so allow me to actually answer your first question.
QEMU doesn't have much of a GUI to speak of. It's a virtualization framework with some CLI tooling on top of it. Frontends/management UI's/etc are left as an exercise to the community so you can find one that suits your needs.
For example, a popular one for macOS these days is UTM: https://mac.getutm.app
QtEmu is another: https://qtemu.org
A recent Windows build used GTK3 decorations; I was very surprised to see that appear. You are right that it is quite low on functionality, its menu allows powering off and rebooting the VM, and gives access to the serial console as well as a command-line interface for the emulator.
It defaults to the GTK display: https://down.loaded.ie/Lb9RDi3.png. See the -display section in QEMU(1) for details and other options. My favorite is -nographic, which forces VGA output to the terminal; very useful when there is no X session or similar.
Edit: I realised GUI might mean creating and managing virtual machines; virt-manager might be good? Take a look at https://wiki.archlinux.org/index.php/Libvirt for the setup needed. (I think it's easier to learn to use vanilla QEMU, though.)
To pass files into VM I use lsyncd. It synchronizes changes to even big source trees over plain ssh and rsync. Of cause with big tree the initial copy takes time, but there is no security implications of exposing files to potentially untrusted VM.
> Is there any screenshots of the QEMU GUI running?
You are looking for Virt-manager or Boxes or any other GUI tool based on qemu if you want something similar to the vbox or vmware experience.
> it have software similar to Virtualbox's 'guest additions' software?
Yes, theyre called spice tools. Work for both linux and windows and infact pre installed on popular linux distros.
QEMU does have a quite primitive GUI while it's running that lets you change disks and so on. But really, it's not a graphical emulator. You specify everything via command-line switches. (They're fairly intuitive, mind you. “-hda drive-c.img -m 16 -cpu 486 -soundhw sb16,adlib -vga cirrus” does what it sounds like.)
There are of course tools that provide a friendlier GUI wrapper if the command-line isn't your thing.
Does this support Apple Silicon yet without patches? Can’t seem to see definite info.
Not sure why you're downvoted. QEMU is not on https://isapplesiliconready.com/ either so I'd think it's a valid question?
There's an app that wraps QEMU called UTM. It's open source but also you can pay $10 and get it on the app store.
I have had a lot of trouble with machines getting corrupted, not working at all, locking up, but I did get a Debian machine working that runs ARM Linux on my Macbook Air M1.
https://getutm.app/
+1 for UTM - The website for mac (https://mac.getutm.app/) has some recipes and pre-built image for Windows and the main Linux distros. I had okay results with emulating Windows -- nothing usable in my day to day, though.
Should work for TCG (emulation); not yet for HVF (virtualization).
HVF does build with Nix (with an overlay), so the build experience doesn't suck. [1] for the base, then apply [2] to SLIRP if you don't want to use HVF's network adapter (which needs root). I'm happily running FreeBSD and Linux VMs on my Mac Mini with this
[1]: https://github.com/benpye/nix-config/tree/main/overlays/qemu
[2]: https://gitlab.freedesktop.org/slirp/libslirp/-/commit/72713...
Can something like this be used to, say, build emulators for different game consoles? And do the popular emulators use this as a building block? Asking this as someone whose only experience with emulation and virtualization is running games on VisualBoyAdvance and running Ubuntu on VirtualBox
its not really meant for that, but there wouldn't be anything to stop someone from creating the hardware emulation, as far as I know. Still, purpose-built console emulator software is sure to be far better for the task.
Yes:
https://xemu.app/
https://xqemu.com/
Are both original Xbox emulators built off of QEMU (Xemu is a fork of XQEMU). I've only used Xemu, but performance was pretty good for the games I tried on it (it doesn't have a way to upscale rendering yet though).
Good news! Upscaling is in work but buggy right now.
https://github.com/mborgerson/xemu/tree/feat/surf-scale
You could build emulators for consoles, but QEMU is not cycle accurate. Meaning some operations may finish faster or slower than they would on the original console. Some games may rely on that timing for things like physics engines and break in weird ways.
Is that also true for Xbox games?
Yes. xbox360 games running on xbox one often run with a custom emulation layer that includes a bunch of monkey patch fixes to make the game work right.
You're looking for MESS: http://mess.redump.net/
I have heard about QEMU for years. Something about virtualization and Linux, this is what I've started to associate it to.
My Workstation runs on Windows, and up until some years ago I was using VMware Workstation for virtualization, and then moved over to VirtualBox, which I've also started to use on a Windows-based home server. I've always used both of them via the GUI.
Now, if I want to build a beefy Linux server (headless) and run some virtualization on it, would I then use QEMU and get the same thing I've been getting with VirtualBox on Windows?
How do both of these compare performance- and feature-wise? Are they alternatives, or do they server different purposes?
You should look into Proxmox PVE.
It offers a nice Web UI for QEMU and LXC and makes the whole experience quite good.
I came from VirtualBox too and never looked back.
Qemu is a whole-system or CPU emulator, originally for x86 PCs but now encompassing other systems and architectures.
Where the virtualization bit comes in is with kvm. Kvm is a Linux kernel module that lets you take advantage of x86 CPU hardware virtualization. Qemu can use kvm to provide the virtual CPU while it itself emulates the rest of the PC hardware, resulting in a complete virtual system.
That said, qemu can also emulate the CPU itself, allowing for example an x86 OS to run on an ARM system, or vice versa, but that of course is slower.
Does QEMU support TSO when used with Apple Silicon?
Doesn't that require installing custom kernel extensions to enable, as there is no API to enable it?
Setting ACTLR_EL1 in a virtual machine (through Hypervisor.framework) is an option.
Do you mean TSG? From https://wiki.qemu.org/ChangeLog/6.0#TCG
> TCG > Added support for Apple Silicon hosts (macOS)
TSO is Total Store Ordering, which refers to the memory model of x86_64. For Rosetta 2, Apple will switch the M1 processor's memory model when emulating x86_64.
Any documentation for this?
Does this mean you can run an x86 VM faster than a snails pace?
I tried an WinXP VM on my M1 Air just for fun but got the performance of an early Pentium. I know emulation is slow but was hoping for a little more.
Are there any option for emulation of 68k machines ?
https://wiki.qemu.org/ChangeLog/6.0#68k
This is a video of the maintainer talking about the new virtual 68k 'machine'. https://www.youtube.com/watch?v=s_ve0bCC9q4 I'm also experimenting with hybrid emulation on the mister minimig - i.e. using qemu as the CPU and the FPGA for the Amiga hardware. https://misterfpga.org/viewtopic.php?f=4&t=2397
I wish you luck! I've been reading a lot about the MiSTer project lately and it is getting harder and harder to resist taking the plunge. There is some amazing work going on there lately.
For those that missed it, here is a recent article on the project...
https://www.pressreader.com/australia/edge/20210422/28424953...
yeah I was able to run Debian's m68k version in QEMU quite successfully with almost no hassle.
Last I played with the coldfire and 68k emulations in qemu to try to get it to boot EmuTOS (an Atari ST operating system) I found a lot of the I/O devices lacking, even just to run headless.
It was sufficient to boot Linux in some configurations but not complete enough of an emulation to run other things. This is in general the story with many things in QEMU.
I didn't keep notes, I suppose I should have. I might revisit this again someday.
m68k emulation in QEMU has matured quite a bit in the past years thanks to the efforts of Laurent Vivier.
It's been probably about 8 years or so since I touched QEMU, but at the time the performance was unusable (on a modest system) compared to a very usable virtualbox or vmware. IIRC, I was just trying to run win XP on Windows 7.
How do things stand these days?
Also, given that QEMU is somewhat platform independent, how well does it run an x86 OS on top of an ARM chip?
Oh yeah so for performance you would want to look into kvm which is used by QEMU.
KVM is a Linux kernel module. ineedasername says they were using QEMU on Windows 7.
Windows has a very similar hypervisor accelerator that QEMU supports called whp.
If anyone knows how to improve the graphics performance on Windows, please share. Compared with other hypervisors like Virtualbox or ESXI, there’s no graphics driver for Windows VMs with QEMU.
> and lots more…
Would be nice if they linked a changelist with all of the commits instead of this^.
These seem to be most of the interesting past threads? Additions welcome:
QEMU Internals - https://news.ycombinator.com/item?id=26941744 - April 2021 (33 comments)
Show HN: QEMU front end for M1 and Intel Macs - https://news.ycombinator.com/item?id=26260390 - Feb 2021 (122 comments)
How to run FreeBSD 13 in QEMU on Apple Silicon Mac - https://news.ycombinator.com/item?id=26053983 - Feb 2021 (39 comments)
QEMU Advent Calendar 2020 - https://news.ycombinator.com/item?id=25262608 - Dec 2020 (30 comments)
Changed-block tracking and differential backups in QEMU - https://news.ycombinator.com/item?id=25189078 - Nov 2020 (5 comments)
Booting a macOS Apple Silicon Kernel in QEMU - https://news.ycombinator.com/item?id=25064593 - Nov 2020 (70 comments)
QEMU 680x0 support [video] - https://news.ycombinator.com/item?id=25027213 - Nov 2020 (13 comments)
QEMU should move from C to Rust - https://news.ycombinator.com/item?id=24133128 - Aug 2020 (209 comments) (god help us)
macOS in QEMU in Docker - https://news.ycombinator.com/item?id=23419101 - June 2020 (186 comments)
Xnu-QEMU-Arm64: iOS on QEMU - https://news.ycombinator.com/item?id=22870905 - April 2020 (34 comments)
QEMU for iOS - https://news.ycombinator.com/item?id=22385370 - Feb 2020 (145 comments)
QEMU VM Escape - https://news.ycombinator.com/item?id=20796446 - Aug 2019 (57 comments)
QEMU 4.1 released - https://news.ycombinator.com/item?id=20713932 - Aug 2019 (14 comments)
QEMU v4.0.0 released - https://news.ycombinator.com/item?id=19736309 - April 2019 (159 comments)
QEMU Advent Calendar: A surprise disk image each day until Christmas - https://news.ycombinator.com/item?id=18588043 - Dec 2018 (34 comments)
Emulate Mac OS 9 with QEMU - https://news.ycombinator.com/item?id=18585157 - Dec 2018 (64 comments)
From VNC to reverse shell - https://news.ycombinator.com/item?id=18051264 - Sept 2018 (9 comments)
Osx-kvm: Run macOS on QEMU/KVM - https://news.ycombinator.com/item?id=17763855 - Aug 2018 (106 comments)
Almost booting an iOS kernel in QEMU - https://news.ycombinator.com/item?id=17544689 - July 2018 (19 comments)
Running Amiga-like OSes on QEMU - https://news.ycombinator.com/item?id=17531490 - July 2018 (12 comments)
Accelerating QEMU on Windows with HAXM - https://news.ycombinator.com/item?id=15759080 - Nov 2017 (24 comments)
VM Escape: QEMU Case Study - https://news.ycombinator.com/item?id=14218600 - April 2017 (44 comments)
QEMU: user-to-root privesc inside VM via bad translation caching - https://news.ycombinator.com/item?id=13928675 - March 2017 (28 comments)
Multi-threaded emulation for QEMU - https://news.ycombinator.com/item?id=13759283 - Feb 2017 (9 comments)
QEMU: virtfs permits guest to access entire host filesystem - https://news.ycombinator.com/item?id=13753950 - Feb 2017 (43 comments)
QEMU Advent Calendar - https://news.ycombinator.com/item?id=13118291 - Dec 2016 (37 comments)
OS X-KVM: Running Mac OS X El Capitan on KVM and QEMU - https://news.ycombinator.com/item?id=12556609 - Sept 2016 (89 comments)
Announcing qboot, a minimal x86 firmware for QEMU - https://news.ycombinator.com/item?id=9585631 - May 2015 (43 comments)
QEMU Advent Calendar 2014 - https://news.ycombinator.com/item?id=8712998 - Dec 2014 (16 comments)
QEMU 2.0.0 Released - https://news.ycombinator.com/item?id=7604916 - April 2014 (74 comments)
Virgil3D - a virtual 3D GPU for qemu - https://news.ycombinator.com/item?id=7042299 - Jan 2014 (1 comment)
QEMU 1.7.0 is now available - https://news.ycombinator.com/item?id=6883988 - Dec 2013 (16 comments)
QEMU 1.6.0 is now available - https://news.ycombinator.com/item?id=6220565 - Aug 2013 (18 comments)
QEMU 1.4.0 released with 95% of native IO performance - https://news.ycombinator.com/item?id=5231545 - Feb 2013 (51 comments)
QEMU 1.0 released - https://news.ycombinator.com/item?id=3303958 - Dec 2011 (19 comments)
Qemu + KVM is the future of open source virtualization - https://news.ycombinator.com/item?id=1226598 - March 2010 (2 comments)
Rip IDE.
Also readconfig/writeconfig
It sucks that Qemu has drifted so far from normal users
IDE support is still there, https://qemu-project.gitlab.io/qemu/system/removed-features....