As per wikipedia, google compute engine uses KVM as hypervisor. I can see mention about vcpu while creating an instance.
Why KVM? Why not VMware OR Xen?
I mean what is the specific reason to choose KVM as a Hypervisor choice?
PS:
Even Xen is a Open source product.
There were a number of factors in the decision, you might not be surprised to learn. :-)
One important factor was compatibility between KVM and existing isolation/scaling processes at Google. (cgroups aka "containers") This allows Google to reuse the same mechanisms that it uses to ensure performance of applications like websearch and gmail to provide consistent performance between VMs scheduled on the machine. This helps GCE avoid noisy neighbor problems.
As you're probably aware, Google has had a long history of Linux kernel development; using KVM allows Google to leverage that talent for GCE. In addition, the hypervisor/hardware emulation split in KVM (where the hypervisor implemented by KVM only emulates a few low-level devices/features, and defers the remaining emulation the the process that opens /dev/kvm) allows for development of virtual devices that have access to the full range of user-space software, including infrastructure like Colossus and BigTable where needed.
Xen, VMware, and HyperV are also great hypervisors and machine emulators, but hopefully that gives you a glimpse into some of the reasons that KVM was a good fit for Google.
Related
As far as I understand, ...
virtualization, although commonly used to refer to server virtualization, refers to creating virtual versions of any IT component, such as networking and storage
although containerization is commonly contrasted to virtualization, it is technically a form of server virtualization that takes place on the OS level
although virtual machines (VMs) commonly refer to the output of hardware-level server virtualization (system VMs), they can also refer to the output of application virtualization (process VMs), such as JVM
Bearing the above in mind, I am trying to wrap my head around the difference between containers and process VMs (NOT system VMs). In other words, what is the difference between OS-level server virtualization and application virtualization?
Don't both technically refer to one and the same thing: a platform-independent software execution environment that is created using software that abstract the environment’s underlying OS?
Although some say that the isolation achieved by container is a key difference, it is also stated that a system VM "is limited to the resources and abstractions provided by the virtual machine"
I have created a graphic representation for you, it is easier (for me) to explain the differences like this, I hope it helps.
OS-level virtualization aims to run unmodified application for a particular OS. Application can communicate with external world only through OS API, therefore a virtualization component put on that API allows to present different image of external world (e.g. amount of memory, network configuration, process list) to applications running in different virtualization context (container). Generally application runs on "real" CPU (if not already virtualized) and does not need (and sometimes have) to know that environment presented by OS is somehow filtered. It is not platform-independent software execution environment.
On the other hand, application VM aims to run applications that are prepared specially for that VM. For example, a Java VM interpretes a bytecode compiled for a "processor" which has little common with a real CPU. There are CPUs which can run some Java byte code natively, but the general concept is to provide a bytecode effective for software interpretation on different "real" OS platforms. For it to work, JVM has to provide some so called native code to interface with OS API calls it is run on. You can run your program on Sparc, ARM, Intel etc. provided that you have OS-specific intepreter application and your bytecode is conformant to specification.
Does the 'softmmu' mean that the virtual machine has a single linear address space available to machine and user mode? Or does it have some virtual memory capabilities that are implemented via software and not the underlying processor? Or maybe it means something different entirely?
-softmmu as a suffix in QEMU target names means "complete system emulation including an emulated MMU, for running entire guest OSes or bare metal programs". It is opposed to QEMU's -linux-user mode, which means "emulates a single Linux binary only, translating syscalls it makes into syscalls on the host". Building the foo-softmmu target will give you a qemu-system-foo executable; building foo-linux-user will give you a qemu-foo executable.
So a CPU emulated by -softmmu should provide all the facilities that the real guest CPU's hardware MMU provides, which usually means multiple address spaces which can be configured via the guest code setting up page tables and enabling the MMU.
Hypervisors isolate different OS running on the same physical machine from each other. Within this definition, non-volatile memory (like hard-drives or flash) separation exists as well.
When thinking on Type-2 hypervisors, it is easy to understand how they separate non-volatile memory because they just use the file system implementation of the underlying OS to allocate different "hard-drive files" to each VM.
But than, when i come to think about Type-1 hypervisors, the problem becomes harder. They can use IOMMU to isolate different hardware interfaces, but in the case of just one non-volatile memory interface in the system I don't see how it helps.
So one way to implement it will be to separate one device into 2 "partitions", and make the hypervisor interpret calls from the VMs and decide whether the calls are legit or not. I'm not keen on communication protocols to non-volatile interfaces but the hypervisor will have to be be familiar with those protocols in order to make the verdict, which sounds (maybe) like an overkill.
Are there other ways to implement this kind of isolation?
Yes you are right, hypervisor will have to be be familiar with those protocols in order to make the isolation possible.
The overhead is mostly dependent on protocol. Like NVMe based SSD basically works on PCIe and some NVMe devices support SR-IOV which greatly reduces the effort but some don't leaving the burden on the hyperviosr.
Mostly this support is configured at build time, like how much memory will be given to each guest, command privilege for each guest etc, and when a guest send a command, hypervisor verifies its bounds and forwards them accordingly.
So why there is not any support like MMU or IOMMU in this case?
There are hundreds of types of such devices with different protocols, NVMe, AHCI etc, and if the vendor came to support these to allow better virtualization, he will end up with a huge chip that's not gonna fit in.
I am new to qemu simulator.I want to emulate our existing pure c h264(video decoder)code in arm platform(cortex-a9) using qemu in ubuntu 12.04 and I had done it successfully from the links available in the internet.
Also we are having multithreading(pthreads) code in our application to speed up the process.If we enable multithreading we are getting the same performance (i.e)single thread(without multithreading).
Eg. single thread 9.75sec
Multithread 9.76sec
Since qemu will support parallel processing we are not able to get the performance.
steps done are as follows
1.compile the code using arm-linux-gnueabi-toolchain
2.Execute the code
qemu-arm -L executable
3.qemu version 1.6.1
Is there any option or settings has to be done in qemu if we want measure the performance in multi threading because we want to get the difference between single thread and multithread using qemu since we are not having any arm board with us.
Moreover,multithreading application hangs if we run for third time or fourth time i.e inconsistent behaviour in qemu.
whether we can rely on this qemu simulator or not since it is not cycle accurate.
You will not be able to use QEMU to estimate real hardware speed.
Also QEMU currently supports SMP running in a single thread... this means your guest OS will see multiple CPUs but will not recieve adicional cycles since all the emulation is occuring in a single thread.
Note that IO is delegated to separate threads... so usually if your VM is doing cpu and IO work you will see at least 1.5+ cores on the host being used.
There has been alot of research into parallelizing the cpu emulation in qemu but without much sucess. I suggest you buy some real hardware and run it there especially consiering that coretex-a9 hardware is cheap these days.
Let's say I have a piece of code that runs fine on an OS. Now, if I install that OS on a virtual machine (server virtualization), and run that code on that, is it possible that the code behaves differently?
If so, what are the prerequisites for that? For example, does it have to be compiled machine code (in other words, are interpreted languages safe?)? Does it have to be certain OS instructions? Specific virtualization technology (Xen, KVM, VMware..)?
Also, what are the possible different behaviors?
Yes. Like any machine, the virtual machine is just another computer (implemented in software instead of hardware).
For one, lots of commercial apps will blow up when you run them on a VM due to:
copy protection detecting the VM
copy protection rigging your hardware, using undocumented features of BIOS/Kernel/hardware
Secondly, a VM is just another computer consisting of hardware implemented in assembly instead of circuits/dye/microcode/magic. This means the VM must provide the emulated hardware either through pass-through or emulation. The fact that hardware is very diverse can cause all kinds of different behavior. Also note the possible lack of drivers for or acceleration of the emulated hardware.
But of course a typical business application for example isn't nearly as likely to rely on any hardware details as all it does is call some GUI API.
Interpreted languages are only safe from this to the extent that they are "interpreted", if the interpreted language calls out to some native code, all this is possible again.
For an example of something detecting that it's running under a VM, check this, it's just one of the literally thousands of ways to detect the VM.
In theory the program should run exactly the same as on a physical machine.
In practice however, there may be differences due to
Machine\OS configuration and drivers
Load of the virtual machine host.
Differences in machine configuration are similar to difference you would see between any difference physical machine. Depending on how critical you application is to the end user, you should run the same set of tests that you would a physical box to determine whether the environment is acceptable for use.
Depending on the virtualisation technology, the host may not have the ability to guarantee the client resources at specific times. This can lead to weird behavior on the client. Potentially you would see more occurrences of application errors due to IO timeouts an starvation of memory.
To successfully virtualise an application for production use you need to do a bit of work to understand the resource profile of the application\client and virtual host.