Virtual Machine Manager (VMM)

[Hedon James]

Okay, I've finally taken SB's advice and started tinkering with VMM in attempt to replace Virtualbox (VB).  I do like the idea of a "native" Linux virtualization solution, and VMM seems to be WAY overkill for my limited use-case scenario.  That's okay though, as I'd rather "grow into" software than "grow out of" it.

 

There isn't much documentation regarding VMM, especially compared to VB; and the documentation available seems to be way over my head, using concepts and terminology that sound greek to me.  But a little background first...  I have intentions of migrating away from Lubuntu and their 3-year LTS support cycle.  To that end, I'm looking at jumping over to Debian Stable, with their approximately 5 year support window.  I've assembled a Debian 10 LXQT VM in VB, but replacing VB with VMM.  I've taken a Refracta Snapshot, copied to usb stick with "dd", and successfully installed to testbed laptop with Refracta Installer.  So far, so good, and I'm pretty happy with everything. 

 

The last piece of my "beta test" is to install Windows VMs, but using VMM in place of VB (VB is no longer supported by Debian 10; VMM is the recommended replacement).  I'm noticing that VMM has MANY more options for VM customization than VB.  So many that it's a little overwhelming...I thought I knew a little about VMs from being able to manipulate my VB VMs to solve numerous VM issues over the years....from driver issues, to expanded disk issues, to formatting issues, to snapshots, etc....  So I was comfortable (perhaps cocky?) to jump over to VMM and give it a go.

 

I've created a Test-Win7VM using VMMs recommended defaults, installing from a Win7.iso installation disk.  Everything worked as expected, although it can't be updated (discontinued support), the monitor resolution isn't native (not a big deal, but would like to resolve), and no sound (incorrect default driver?).  We'll come back to this later and see what I can learn by "fixing" these issues, but let me continue to the REAL concern I need help with.

 

With a successful creation of Windows VM in VMM, I could "load up" the VM with software that I use in Windows, such as Appraisal Dictionary, appraisal software, Quickbooks, etc...  But ideally, I already have a Win7 VM and a Win10 VM in VB, both with vdi (or maybe vmdk?) disks.  I copied over my Win7 VB disk and created a Win7 VM in VMM, using the existing Win7.vdi, just like I have done numerous times before in VB.  However, when starting the VMM machine, it "hangs" on the Win7 boot splash image and freezes.  I can only "force close" from the VMM menu option.  Googling suggests this is a very common occurrence with Win VMs and I need to choose a "VNC viewer" rather than "Spice",  and I should choose "cirrus" as VGA driver rather than QXL.  WTH are they talking about?  Looking through the VMM options, I can see the Display categories for Spice vs VNC servers; and I can see the Video categories for "QXL" vs "VGA" vs "Virtio".  But I see nothing for a selection of "cirrus".  I've also seen some convoluted references to BIOS vs UEFI and that when booting from BIOS, "SeaBIOS" is preferable.  I've made no specific selections, but I did note that VMM is booting with "SeaBIOS".

 

While I can install Win VMs from scatch, then load them up with the softwares I need, this seems counter-intuitive to the point of a VM.  I'd really like to just boot a Win VM with my CURRENTLY EXISTING windows disk...like pulling a metal disk from computer A and connecting it to computer B, and continuing with business as usual.  I could save TENS of man-hours if I can do this in the manner described.

 

So, as the resident KVM/Qemu/Virt-Machine Manager expert here at BATL....what am I doing wrong SB?  Can I do what I'd like to do, as described?  If so, how?  Or am I relegated to creating virgin WinVMs and "loading them up" all over again with needed software?

[Hedon James]

Update:

 

I've learned that VMM is a GUI front end for qemu & kvm tools.  The native disk format for VB is *.vdi and *.vmdk, although I've learned that *.vmdk is actually a VMWare format; but VB is fine with it.  The native format for the fully open source qemu/kvm software stack is *.qcow2 (qemu copy on write, gen 2).  On a hunch (see last paragraph), I'm wondering if the file format is my issue.  On a fact-finding mission:

 

- I've converted my WindowsXP vdi disk to an img format with VBoxManage tools, then converted the img to a qcow2 format; still boots to BSOD

- I've converted my WindowsXP vdi disk to a RAW img with VBoxManage, then converted the IMG to qcow2; still boots to BSOD

- I've converted my WindowsXP vdi disk to a qcow2 format with qemu-img tools; still boots to BSOD

- as a "check", the VB WinXP VM still works fine in VB, so the source vdi is not corrupted

 

The BSOD message is to the effect that "problem was detected.  to prevent damage to disk, boot was stopped.  check for virus, remove last device, and/or run chkdsk /f".  I can do none of the above with VMM and, as mentioned above, disk image is fine in VB.

 

While my WinXP VM is more of a "legacy" machine that I rarely use, it's nice to have around, as there IS software on there that is so old it can't be replaced anymore, but runs fine.  Things like CD labels and jewel case designers; a DRM "stripper" for music; a lifetime version of PlayOn media server software; a very old version of WinTotal appraisal software, etc...  So I'd like to get it converted successfully, but I can't remember the last time I "needed" it.  My critical VMs to convert are Win7 and Win10.

 

When I try to repeat the WinXP conversion process for my Win7 disk, the Win7 disk is in *.vmdk format.  Both VBoxManage and qemu-img conversions throw an error about "invalid header" and won't convert.  So I haven't even been able to attempt a run in VMM.  As an alternative, I tried to spin up a Win7.vmdk from a different machine (no conversion), and it starts to boot in VMM before it freezes/hangs on the Win7 boot splash.  That's when I attempted to "new install" a Win7VM in WMM, which turned out fine; which lead me to conclude that MAYBE it's a "native format" issue?  But after the above results, I'm stumped...

 

Can I get some fresh eyes on this?  SB, you're the only person on here i KNOW uses qemu-kvm...do you also use the Virt-Manager GUI (VMM)?  Any thoughts?

Edited January 20, 2020 by Hedon James

[Hedon James]

I think I solved my WinXP VM issue.  Solution found here:

 

http://slashsda.blogspot.com/2012/03/migrate-from-virtualbox-to-kvm.html

 

I only had to install the "patch", from within VB, and attached the WinXP.vmdk disk to a VMM-created VM.  Booted up as expected, and everything seems to be working fine.  Had to "reactivate" due to the significant hardware change, but it's WinXP and it's just going through the motions, I think.  Activation successful!  Now to figure out how to duplicate VBs "internal network only" functionality and get this VM off the internet!  LOL.

 

As a side note, I've wasted numerous man hours converted vmdk and vdi formats to qcow2, before failing.  I thought I had to convert *vmdk and *vdi images to native qcow2 format, but turns out qemu reads those formats just fine, as well as others.  Right now, VMM is reading & writing to the disk in vmdk format.  Nevertheless, once I'm 100% satisfied, I'll probably convert to qemu's native qcow2 format to avoid any future unforseeable issues.  Until then, seems like verifying that VMM boots a vmdk or vdi image is a real time saver before converting.  FWIW...

 

Stay tuned for Win7 and Win10 VMs.  Looks like a completely different solution, involving virtio drivers and a virtio.iso image.  On to the next one!

 

Edit #2:  spoke too soon.  had to do the "patch" AND the Hiren's boot Cd fix HDD controller, AND the suggested registry hack.  WinXP is booting again, but insists on looking for drivers for newly found hardware (qemu hw vs. VB hardware).  Hoping the virtio drivers iso image will fix that, but at least it's working again (for now)!

Edited January 24, 2020 by Hedon James

[V.T. Eric Layton]

COOL!

 

I solved my VM issues years ago by deciding to never do anything VM on my machines. Worked like a champ!

[Hedon James]

Okay, think I'm done "hacking" Win XP to work in VMM/qemu.

 

Solved my "new hardware found, looking for drivers" issues by following along here:

http://www.zeta.systems/blog/2018/07/03/Installing-Virtio-Drivers-In-Windows-On-KVM/

 

Was relatively straightforward and painless, and notifications went away, and system seemed to run better.  But I couldn't leave "well enough" alone.  Avast was severely outdated, tried to update itself, failed & crashed.  So I ripped it off and installed AVG.  I still had IE8 as a web browser and decided to add a more modern XP-compatible browser.  You know what a challenge it is to find a reasonably modern 32-bit XP-compatible web browser?  What a rabbit hole that was!!!  Ended up installing Slimjet 10 and all is good on the internet side.

 

Noticed several unnecessary services were autostarting, so disabled them.  WinXP was running pretty lean & mean in a qemu VM!  Then I decided to uninstall "unnecessary software" like Adobe Flash, an old version of Quickbooks, and VirtualBox Guest Additions, and clean up the registry, etc... with CCleaner.  Long story short, removing VB Guest Addition drivers was a BIG MISTAKE and the BSOD re-appeared.  After several more Safe-Mode boots and registry hacks, I've got it working again, but my brain is JELLO!  I hope I don't ever have to duplicate this process, or help anyone else with it, as I'm not sure I'll remember all the steps.  It sure wasn't as simple and clean as all the online tutorials...

 

Learnings from this exercise:

- converting an existing Win VM to qemu is way more convoluted than converting the disk image to a native qemu format

- getting WinXP running in a VM format is ridiculously challenging, IMO.  I only pressed on for the learnings, and to keep really old legacy software functioning and viable.

- if Win7 and Win10 migrations are this challenging, it's probably more time-efficient to do a fresh install and load the VM with desired software; that wasn't an option with my WinXP software, but I still have the disks, licenses/registrations for Win7 & Win10 software, so I have no qualms performing a virgin install and building them backup.

- I hate proprietary software; it was a true PITA to import/convert proprietary *vdi and *vmdk formats into the open source *qcow2 format and I don't know for certain, but I'm willing to bet that the open-source format converts into the proprietary format with no issues whatsoever.  Flashbacks of converting *.doc reports into *odt files and the nightmare of formatting/pagination/table issues; but odt pulls into doc format just fine. 

And now that I know this about vdi and vmdk formats, I will do whatever it takes to eradicate that on MY systems. 

 

I stripped DRM off my entire music collection (over 7,000 songs) once I learned it was there, and put everything into a universally compatible mp3 format (would've preferred ogg, but ogg wasn't universal for me).  I converted my entire library of appraisal reports & spreadsheets (over 20 years' worth) to open document formats.  And now I'm on the warpath against vdi and vmdk.  You 2 jokers are next for termination...

[Hedon James]

Almost final update for the next guy who wants to attempt a similar migration from VirtualBox (VB) to VirtualMachineManager (VMM):

 

- despite my previously hard-earned knowledge gleaned from my WinXP migration, Win7 did not go so well.  Win7 would not boot.  Troubleshooting with the Win7 wizard, as well as disk repairs from the install disk, all failed.  Googling the failure code lead to many proposed solutions, none of which worked for me.  It all boiled down to a detected hardware change that I just couldn't remedy, nor circumvent.  Even Hirens Boot Repair couldn't get it done.  I know I spent 10-12 man-hours on this over the entire weekend.  Finally gave up, threw in the Win7 installation CD and installed Win7 fresh on the disk image.  Had a VMM Win7 VM in less than 15 minutes; probably spent another 30 minutes re-installing software on the Win7 VM; then let the VM update itself from SP1 to finish overnight.  Woke up this morning to about 115 successfully installed updates.  Still had 4-5 "stragglers" that updated after reboot.  But my Win7 VM has been re-constructed on VMM "hardware" and drivers.  This wasn't an option for my WinXP machine, as I no longer have the WinXP disks or activation codes, nor the legacy software I was trying to keep.  While Win7 was a supreme PITA, if I had to fail at one, I'm glad it was Win7.  I have the disks, the codes, and the software to reconstruct that machine.  And it was much quicker (and probably much "cleaner") to reinstall from scratch.

 

- Win10 was a piece of cake.  The only hiccup was that Win10 was not an option in the VMM drop-down list for OS selection.  So I chose Win8.1, and built the VMM machine with defaults; attached the Win10 disk for storage, and it fired right up, with no issues whatsoever.  Win10 has always been a testbed for me, with no production software.  But now that Win7 support is over, I'll load up the Win10 VM with my windows-only software and start using Win10 for those purposes.  I'll keep Win7 VM as an emergency backup, cuz you never know; and I'll keep WinXP VM for those RARE occasions where I need my legacy software.

 

The only thing left to do is to convert my VM disks from native *vdi and *vmdk formats to qemu *cow2 formats.  But based on previous tinkerings, that should be a piece of cake for an already functioning storage image.  Once that's done, and everything is backed up, I'll be migrating my host machine (my main work rig) from Lubuntu 16.04 to my customized Debian 10 LXQt OS.  I'm a little nervous to convert my CURRENT work rig, as I've usually gotten/built a new rig and brought it up to speed, running side-by-side before the switchover.  But for the first time since I've needed an upgrade of my main rig, my hardware is still relevant and I just can't talk myself into a new machine, even though it would be "safer".  But I did buy a new SSD and I WILL pull my current SSD with current Lubuntu intact, just in case I need to revert quickly.  For the price of new SSDs, it's worth it to me for the safety net of having my machine back up and running as it did before, in the event of unforseen issues.

 

I've planned ahead (or so I think?!), so it'll probably be wasted money.  But if I embrace the risk under the assumption that I know what I'm doing and everything will be fine, I can tell you EXACTLY how that scenario almost always plays out... 

[V.T. Eric Layton]

A very informative thread. If I ever go "virtual", I'll definitely be searching for this on Scot's. And no, this is not meant to be a sarcastic statement; just thought I'd verify that because everyone who knows me would assume it was from the very get-go.

[Hedon James]

2 hours ago, V.T. Eric Layton said:

A very informative thread. If I ever go "virtual", I'll definitely be searching for this on Scot's. And no, this is not meant to be a sarcastic statement; just thought I'd verify that because everyone who knows me would assume it was from the very get-go.

If you're not converting existing VMs, you won't need this thread.  Much simpler, cleaner, & time-efficient to create from scratch.  And VMM is actually quite simple.  It is capable of so much more, and it's overkill for my simple VM scenarios; but I like solutions you grow INTO, rather than OUT of.  And there's a lot in VMM to grow into.

 

If you decide to give it a whirl, I'm glad to help however I can.  Helpful hint...Windows 10 is free if you join Insider Preview.  So it might be fun for the free knowledge!  Only catch is you can't activate.  But the only thing you're missing from activation is ability to customize themes, wallpaper, icons, fonts, etc...  And I don't care what windows looks like!  LOL!

 

[V.T. Eric Layton]

Heh... I've been running an unlicensed Win 7 Enterprise for about three years now. The commercial license (from a local college I attended) expired back then. I can't change my theme or desktop. Oh, darn. I only use the thing to play games. It hasn't been connected to the Internet since I installed it on this system.

[Hedon James]

okay, final update, i think.

 

conversion from *vdi and *vmdk to native qemu *cow2 formats was completed successfully and VMs now have suspend/resume functionality, just like VB.  Tinkered with the sound driver settings in each VM and I also have sound in each now!  So I'd say mission accomplished with the critical Windows VMs and I'm in good shape for host machine OS migration.  Just need a break in the action for 2-3 days to ENSURE the production rig has any potential bugs/glitches ironed out and/or workarounds figured out in advance.

 

I think the only thing I'm missing in VMM that I'd really like to re-implement is the ability to re-order and/or "group" my VMs.  Right now, it displays my VMs in a reverse chronological order...newest installation first, oldest installation last.  In VirtualBox, I grouped my VMs in "Windows", "Ubuntu", "Remastered OS", and "General Linux" categories, and sorted them alphabetically.  I really need to cut down on my VMs, and I will, but I'd still like another way to organize/display them, other than the default chronological order.

 

Where you at SB?  I undertook this project based on your recommendation, and track record of solid advice, but haven't heard from ya once in this thread?!  Anything to share regarding this minor (but still kinda important) issue?

 

Otherwise, I hope someone else finds this thread useful someday, without chuckling too much at my learnings.  Fait Accompli!

[raymac46]

What about Guest Additions ? Do you need anything like that to get full screen/native resolution?

 

[Hedon James]

I've been installing qemu guest additions, according to this article:

http://www.zeta.systems/blog/2018/07/03/Installing-Virtio-Drivers-In-Windows-On-KVM/

 

This was "mission critical" in my WinXP migration, but wasn't necessary in my clean Win7 install or Win10 migration.  So truth be told, I'm not sure they're NECESSARY for native resolution; and VMM has a "full screen" option that I haven't messed with, cuz I'm not a fan of the full screen appearance.  But the guest additions exist, and presumably add something to the experience.  Maybe SB knows more.  Then again, SB advocates for qemu/KVM virtual machines and VMM is only a gui front-end for gemu...so maybe SB doesn't know much about VMM.  He's been conspicuously absent in this thread...

[securitybreach]

Sorry, some how i missed this post. Let me read through it now .

[securitybreach]

Hold on, why are you converting anything? Just simply fire up VMM.

 

Click File, New Virtual Machine and then choose Local install media.

Next choose the ISO location, choose the OS name or set custom name , click forward and set Memory amount and CPU cores.

Then either choose choose an existing disk image or create a new one with the wizard.

Then click next. Finally make sure you click the box that says Customize Configuration Before Install.

This allows you to customize the hardware. I only change two things: Display Spice to VNC server and the Video settings from QXL to VGA.

Then I start the installation like normal.

 

Sorry for missing your post but if it happens again, just message me. I feel bad about missing it but I have so much going on that I missed the post.

[securitybreach]

If you need me to, I can record a video of me running through the Wizard.

[securitybreach]

Here is a video I made of creating a new vm: https://my.pcloud.com/publink/show?code=XZOr7skZhFOmP8wXf2FY1pY6wCMdTBt6hxpX

[securitybreach]

Funny enough, I didn't even know that it was called Virtual Machine Manager (VMM) as I always seen it referred to as virt-manager. I have been using it for a few years now.

[Hedon James]

11 hours ago, securitybreach said:

Hold on, why are you converting anything? Just simply fire up VMM.

 

Click File, New Virtual Machine and then choose Local install media.

Next choose the ISO location, choose the OS name or set custom name , click forward and set Memory amount and CPU cores.

Then either choose choose an existing disk image or create a new one with the wizard.

Then click next. Finally make sure you click the box that says Customize Configuration Before Install.

This allows you to customize the hardware. I only change two things: Display Spice to VNC server and the Video settings from QXL to VGA.

Then I start the installation like normal.

 

Sorry for missing your post but if it happens again, just message me. I feel bad about missing it but I have so much going on that I missed the post.

 

That is how I created VMM appliances, using existing Windows VM disk images, but using default settings (presumably obtained from "type of OS to be installed").  When I started messing with customization settings, nothing was working.  Came to the conclusion that maybe the *vdi and *vmdk file formats were the issue (they weren't) and learned how to convert to native *cow2 format.  That didn't work either, so went back to the clones of original vdi and vmdk file formats once I learned that qemu was fine with that.

 

But WinXP disk image would NOT boot.  Went down the rabbit hole of troubleshooting and was able to get it out of the BSOD boot, and WinXP was good to go.  See above.

 

And I couldn't get Win7 to boot no matter what I did.  Went through all the WinXP gyrations.  Web surfed for several other Win7 proposed solutions.  NOTHING worked.  Put the disk back in a VirtualBox VM and booted right up.  I'm fairly certain the issue was a "significant hardware change" that Win7 didn't like, as troubleshooting errors seemed to suggest PCI drivers as culprit (new motherboard?)  Finally gave in and installed Win7 from scratch, overwriting the previous Win7.vdi image.  Everything is fine with that now.

 

Win10 fired up with no issues whatsoever.  The way I hoped WinXP and Win7 would've behaved.

 

Once I got those three machines migrated into VMM and working, I realized I couldn't suspend them, as the native cow2 format is required for snapshots.  So I converted the working vdi and vmdk formats to cow2 and, tada, suspend/resume snapshots are working.

 

So there you go...summarized in a few brief paragraphs...you're completely up-to-speed now!  LOL!  Only thing I'm still looking for in VMM is ability to "group" VMs, or to rearrange/sort them in a manner of my own preference, rather than default order.  Got any nuggets of knowledge for that?

Edited February 11, 2020 by Hedon James

[securitybreach]

3 minutes ago, Hedon James said:

 

So there you go...summarized in a few brief paragraphs...you're completely up-to-speed now!  LOL!  Only thing I'm still looking for in VMM is ability to "group" VMs, or to rearrange/sort them in a manner of my own preference, rather than default order.  Got any nuggets of knowledge for that?

 

I will have to wait till I get home to see if/how you rearrange them but I have never tried grouping them before. Let me see what I can find on that.

[Hedon James]

1 hour ago, securitybreach said:

Funny enough, I didn't even know that it was called Virtual Machine Manager (VMM) as I always seen it referred to as virt-manager. I have been using it for a few years now.

 

In all fairness, that is the CLI command to launch it!

[securitybreach]

1 minute ago, Hedon James said:

 

In all fairness, that is the CLI command to launch it!

 

Well you can also interact with them using the virsh command as well:

 

Quote

 

virsh list (this will print list of your VMs)

virsh dominfo your_vm_name (get info about selected VM)

virsh autostart your_vm_name (set autostart of selected VM)

 

 

 

[Hedon James]

This is what I did with VB.  Ideally, I'd like to implement a similar system.  The default newest-first to oldest-last arrangement isn't going to work for me!

 

[securitybreach]

Lots more listed here: https://computingforgeeks.com/virsh-commands-cheatsheet/

[securitybreach]

Just now, Hedon James said:

This is what I did with VB.  Ideally, I'd like to implement a similar system.  The default newest-first to oldest-last arrangement isn't going to work for me!

 

 

Wow, that is an impressive list

[securitybreach]

Check this out:

virt-install, virt-clone, virt-convert a set of command line tools for provisioning new VMs from install media, existing VMs and appliances, respectively. See also 'virsh' command line shell with libvirt.

 

As well as other frontends for working with QEMU:

 

https://wiki.qemu.org/Links

[Bookmem]

What packages are needed to run VMM in Debian?  (Mint 19.3)

[securitybreach]

18 minutes ago, Bookmem said:

What packages are needed to run VMM in Debian?  (Mint 19.3)

 

Not for sure but here is the link to install it on LinuxMint: https://community.linuxmint.com/software/view/virt-manager

[Bookmem]

11 minutes ago, securitybreach said:

 

Not for sure but here is the link to install it on LinuxMint: https://community.linuxmint.com/software/view/virt-manager

I've got that installed already along with Qemu/kvm.  But when I run it I get an error message saying the libvertd daemon isn't running. If I try to install libvirtd via apt, it can't locate the package.

 

[Hedon James]

31 minutes ago, Bookmem said:

What packages are needed to run VMM in Debian?  (Mint 19.3)

 

Can't remember exactly, but I think when I installed virt-manager

sudo apt install virt-manager

the virt-manager pulled in all the requisite dependency packages.

 

If VMM doesn't work, or anything is missing, here is the underlying Qemu/KVM page for Debian.  Virt-Manager is just a GUI front-end for qemu & kvm.

https://wiki.debian.org/KVM

[securitybreach]

2 minutes ago, Bookmem said:

I've got that installed already along with Qemu/kvm.  But when I run it I get an error message saying the libvertd daemon isn't running. If I try to install libvirtd via apt, it can't locate the package.

 

 

Oh, you need to start the daemon first:

 

sudo systemctl start libvirtd