Qemu Virt Machine from flash device image - qemu

Question

I have been developing the OS for a prototype device using hardware. Unfortunately, it's a very manual and buggy process to flash the OS each time and then debug the issues.
I'd like to switch to developing the OS in QEMU, so that I can be sure that the OS is loading correctly before going through the faff of programming the device for real. This will come in handy later for Continuous Integration work.
I have a full copy of the NVM device that is generated from my build process. This is a known working image I'd like to run in QEMU as a start point. This is ready to then be JTAG'd onto the device. The partition layout is:
P0 - loader - Flash of IDBLoader from rockchip loader binaries
P1 - Uboot - Flash of Uboot
P2 - trust - Flash of Trust image for rockchip specific loader
P3 - / - Root partition with Debian based image and packages required for application
P4 - data partition - Application Data
I have not changed anything with the Rockchip partitions (P0 - P2) apart from the serial console settings. When trying to boot the image though, nothing happens. There is no output at all, but the VM shows as still running. I use the following command to run it:
qemu-system-aarch64 -machine virt -cpu cortex-a53 \
-kernel u-boot-nodtb.bin \
-drive format=raw,file=image.img \
-boot c -serial stdio
I have no error information to go on to understand what is going on with it, where can I get more information or debug?

Answer 1

QEMU cannot not emulate arbitrary hardware. You will have to compile U-Boot to match the hardware that QEMU emulates, e.g. using make qemu_arm64_defconfig. The OS must also provide drivers for QEMU's emulated hardware.
If you want to emulate the complete hardware to debug drivers, Renode (https://renode.io/) is a good choice.

Answer 2

For anyone else trying to figure this out, I found good resources here:
https://translatedcode.wordpress.com/2017/07/24/installing-debian-on-qemus-64-bit-arm-virt-board/
and
https://azeria-labs.com/emulate-raspberry-pi-with-qemu/
Looking at the information though, you need to extract the kernel from your image and provide that to the qemu command line as an argument. You'll also need to append an argument telling the system which partition to use as a root drive.
My final command line for starting the machine looks like this:
qemu-system-aarch64 -machine virt -cpu cortex-a53 \
-drive format=raw,file=image.img,id=hd \
-boot c -serial stdio
-kernel <kernelextracted> -append "root=fe04"

Answer 3

Different Arm boards can be significantly different from one another in where they put their hardware, including where they put basic hardware required for bootup (UART, RAM, interrupt controller, etc). It is therefore pretty much expected that if you take a piece of low-level software like u-boot or a Linux kernel that was compiled to run on one board and try to run it on a different one that it will fail to boot. Generally it won't be able to output anything because it won't even have been able to find the UART. (Linux kernels can be compiled to be generic and include drivers for a wider variety of hardware, so if you have that sort of kernel it can be booted on a different board type: it will use a device tree blob, provided either by you or autogenerated by QEMU for the 'virt' board, to figure out what hardware it's running on and adapt to it. But kernels compiled for a specific embedded target are often built with only the device drivers they need, and that kind of kernel can't boot on a different system.)
There are broadly speaking two paths you can take:
(1) build the guest for the board you're emulating (here the 'virt' one). u-boot and Linux both have support for QEMU's 'virt' board. This may or may not be useful for what you're trying to do -- you would be able to test any userspace level code that doesn't care what hardware it runs on, but obviously not anything that's specific to the real hardware you're targeting.
(2) In theory you could add emulation support to QEMU for the hardware you're trying to run on. However this is generally a fair amount of work and isn't trivial if you're not already familiar with QEMU internals. I usually ballpark estimate it as "about as much work as it would take to port the kernel to the hardware"; though it depends a bit on how much functionality you/your guest need to have working, and whether QEMU already has a model of the SoC your hardware is using.
To answer the general "what's the best way to debug a guest image that doesn't boot", the best thing is usually to connect an arm-aware gdb to QEMU's gdbstub. This gives you debug access broadly similar to a JTAG debug connection to real hardware and is probably sufficient to let you find out where the guest is crashing. QEMU also has some debug logging options under the '-d' option, although the logging is partially intended to assist in debugging problems within QEMU itself and can be a bit tricky to interpret.