Xen Hypervisor

From ErikaWiki

(Difference between revisions)
Jump to: navigation, search
(Creating a working system)
(Compiling and installing the bootloader)
Line 125: Line 125:
# dd if=uboot.img of=/dev/mmcblk0 bs=1024 seek=40
# dd if=uboot.img of=/dev/mmcblk0 bs=1024 seek=40
</pre>
</pre>
 +
 +
== Creating the root file system ==
 +
This section covers the creation of a single partition which will be formatted with an ext4 file system.
 +
 +
After that, an [http://archlinuxarm.org/platforms/armv7/allwinner/cubieboard2 Arch Linux file system] will be used for the partition (on a Debian host [http://linux-sunxi.org/Bootable_SD_card#Rootfs you can use <tt>debootstrap</tt> as an alternative to the download of a preconfigured root file system]).
 +
 +
<pre>
 +
# fdisk /dev/mmcblk0
 +
</pre>
 +
 +
Then, you need to create a Linux partition. Please note that the first sector of the first partition in the SD card must be in a position after an offset of 40 KB (for the U-Boot installation) plus the size of the U-Boot image.
 +
 +
<pre>
 +
# mkfs.ext4 /dev/mmcblk0p1
 +
# mount /dev/mmcblk0p1 /mnt
 +
# wget http://archlinuxarm.org/os/ArchLinuxARM-sun7i-latest.tar.gz
 +
# tar -xf ArchLinuxARM-sun7i-latest.tar.gz -C mnt
 +
# umount /mnt
 +
</pre>
 +
 +
== Compiling and installing of the dom0 kernel ==
 +
Download the sunxi-devel branch from the official sunxi repository. That branch contains the drivers for the SD, USB ed ethernet.
 +
 +
<pre>
 +
# mkdir sunxi-devel
 +
# cd sunxi-devel
 +
# git init
 +
# git remote add -t sunxi-devel -f origin https://github.com/linux-sunxi/linux-sunxi.git
 +
# git checkout sunxi-devel
 +
</pre>
 +
 +
After that, please go on with teh kernel compilation. In the default configuration for the architecture and platform you need to enable the Xen options, as well as the peripherals and drivers that we want to use.
 +
 +
A complete list of the needed configuration options is available on the [http://openmirage.org/wiki/xenoncubieboard2 Mirage OS wiki].
 +
 +
Since ours is a minimal configuration, we disabled the support for dynamic loading modules.
 +
<pre>
 +
# make ARCH=arm multi_v7_defconfig
 +
# make ARCH=arm menuconfig
 +
# make ARCH=arm CROSS_COMPILE=armv7a-hardfloat-linux-gnueabi- zImage dtbs -j4
 +
</pre>
 +
 +
The <tt>dtbs</tt> target is needed to compile the device tree in the binary format used during boot. Once the compilation is terminated, the Linux zImage can be found as <tt>arch/arm/boot/zImage</tt>, whereas the binary device tree to be used is <tt>arch/arm/boot/dts/sun7i-a20-cubieboard2.dtb</tt>.
 +
 +
You need to copy both the zImage and the binary device tree  in the root of the file system you created on the SD card containing the environment we are creating.
= Publications @ CloudCon =
= Publications @ CloudCon =

Revision as of 18:24, 12 January 2015

Contents

Introduction

This page describes the current support of ERIKA Enterprise for the Xen Hypervisor.

In particular, ERIKA can be run as a guest operating system inside a Xen domU.

Requirements for the working environment

The AllwinnerA20 CPU available in the cubieboard2 is directly supported by Xen; The Xen Wiki contains most of the documentation needed for building the software components.

This section summarizes the basic requirements of the development environment needed to start ERIKA Enterprise as a domU over a Linux dom0.

Firmware Requirements

The Hypervisor boot must happen in Hypervisor mode (EL2) (non secure).

Dupport for the EL2 boot mode is in the official repository since January 2014.

Moreover, to correctly init both cores of an SMP Platform, the bootloader needs to correctly handle the Power State Coordination Interface (PSCI); This support is not currently available in the official U-Boot repository, but a first working version is available in the development branch wip/psci of this repository.

dom0 requirements

Device tree.

All the platform features must be enumerated in a device tree, which must be compiled in the device tree binary format (also called device tree blob).

Starting with kernel 3.14, it is no more needed the creation of a dedicated device tree for the Linux boot on top of an hypervisor: it is in fact sufficient to [http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/arch/arm/boot/dts/sun7ia20cubieboard2. dts use the device tree provided together with Linux]

In other words, the bootloader must provide to Xen a dtb for the dom0 kernel, plus an optional one for initramfs (in case the kernel needs it). The bootloader also needs to load the kernel and initramfs in memory, and also to note their location in the device tree blob; That information is included into a new node of the device tree, named chosen, that contains:

  • the image location and a classification per compatibility
  • additional parameters to be passed to the hypervisor or to the dom0 during boot.

The chosen node must be defined in the following way (as specified in docs/misc/arm/devicetree/booting.txt:

chosen {
  bootargs = "$xen_cmdline";
  modules {
    module@0 { /* dom0 kernel */
      bootargs = "$dom0_cmdline";
      compatible = "xen,linuxzimage", "xen,multibootmodule";
      reg = <$kernel_addr $kernel_size>
    };
    module@1 { /* initramfs */
      compatible = "xen,linuxinitrd", "xen,multibootmodule";
      reg = <$initramfs_addr $initramfs_size>;
    };
  };
};

The chosen node can be manually added to the device tree (e.g., by changing the sources and recompiling), or can be created dynamically at boot time using the U-Boot command fdt.

domU requirements

domU executable format.

Xen uses the zImage format to load a ARM kernel. However, the image should not be compressed; the only needed operation is the compilation of the kernel source code with a zImage header.

That header must have an offset 0x24 inside the zImage and must contain:

  1. a magic number (0x016f2818) that identifies the file as zImage;
  2. the starting address of the zImage;
  3. The end address of the zImage.

The following is an header example:

_start:
  @ zImage header
.rept 8
  mov r0, r0
.endr
  b _start_cubieboard2
  .word 0x016f2818 @ Magic numbers to help the loader
  .word _start_0 @ absolute load/run zImage address
  .word _end _
  start @ zImage size
  @ end of zImage header
_start_cubieboard2:

Please remember the zImage should not be compressed. To obtain the executable in the required format you can use the command objcopy as done during the compilation of a normal application for Cortex Ax (the hypothesys is that the objcopy path is for the ARMv7 toolchain is in the environment variable EE_OBJCOPY):

$(EE_OBJCOPY) -O binary myProject_master.elf myProject_master.bin

base address for the executable image of the domU

Th base address for the adress space reserved to a domU is 0x80000000. The [xen/include/public/archarm.h:372 linker script] used for the compilation of the ERIKA source code must be adapted to that value.

Privileged Instructions

A non-privileged domain cannot access to the System Control Register to enable the memory management unit (by setting bit M); If done, a domU exception is raised.

Moreover, the hypervisor protects register CP15 c15 to non privileged guest domains. This because on some ARM architectures using that register is possible to have a dump of caches L1/L2: having free access to that register would then bring memory isolation problems.

IMPORTANT: Commit to be reverted

If you compile a Xen Repository including commit 878ff4fe1816d7f808f11254b555b9e9c2f121fa, this will provoke a critical shutdown of the ERIKA Enterprise domU at boot time. We did not yet understand why this happens. the only solution is currently to revert the commit.

Creating a working system

The following steps will guide through the creation of a working system including a Linux dom0 and an ERIKA Enterprise domU that will be executed under the Xen hypervisor.

The hypothesis is that a armv7a (hardfloat) toolchain is installed in the system. That toolchain must include at least a gcc compiler, the ld linker as well as the objcopy tool. The hypothesis is that the executables will have a name starting with armv7a-hardfloat-linux-gnueabi- (Please note that this prefix is typical for the Gentoo distributions. Debian packages typically have the prefix arm-linux-gnueabihf-.).

The following sections describe how the environment can be installed into an SD card. Installing it over a USB flash drive is similar. This page does not contain instructions on using tftp or netboot.

Preparing the toolchain

Debian distributions as well as other distributions typically provide pre-compiled toolchains (on Ubuntu the package name is gcc-arm-linux-gnueabi).

However, the suggested procedure in our case is to use the crossdev application to generate a toolchain for the specific architecture:

crossdev -t armv7a-hardfloat-linux-gnueabi

Compiling and installing the bootloader

Use the following command to download and compile the U-Boot repository with support for PSCI:

# git clone https://git.kernel.org/pub/scm/linux/kernel/git/maz/uboot.git u-boot-psci
# cd u-boot-psci
# git checkout origin/wip/psci
# make CROSS_COMPILE=armv7a-hardfloat-linux-gnueabi- Cubieboard2_config
# make CROSS_COMPILE=armv7a-hardfloat-linux-gnueabi-

The bootloader should be stored at an offset of 40KB, whereas the symbol table at an offset of 8KB.

Supposing we are using a Linux host, and that the SD card can be accessed using the device file /dev/mmcblk0, you can use the following commands:

# dd if=spl/sunxispl.bin of=/dev/mmcblk0 bs=1024 seek=8
# dd if=uboot.img of=/dev/mmcblk0 bs=1024 seek=40

Creating the root file system

This section covers the creation of a single partition which will be formatted with an ext4 file system.

After that, an Arch Linux file system will be used for the partition (on a Debian host you can use debootstrap as an alternative to the download of a preconfigured root file system).

# fdisk /dev/mmcblk0

Then, you need to create a Linux partition. Please note that the first sector of the first partition in the SD card must be in a position after an offset of 40 KB (for the U-Boot installation) plus the size of the U-Boot image.

# mkfs.ext4 /dev/mmcblk0p1
# mount /dev/mmcblk0p1 /mnt
# wget http://archlinuxarm.org/os/ArchLinuxARM-sun7i-latest.tar.gz
# tar -xf ArchLinuxARM-sun7i-latest.tar.gz -C mnt
# umount /mnt

Compiling and installing of the dom0 kernel

Download the sunxi-devel branch from the official sunxi repository. That branch contains the drivers for the SD, USB ed ethernet.

# mkdir sunxi-devel
# cd sunxi-devel
# git init
# git remote add -t sunxi-devel -f origin https://github.com/linux-sunxi/linux-sunxi.git
# git checkout sunxi-devel

After that, please go on with teh kernel compilation. In the default configuration for the architecture and platform you need to enable the Xen options, as well as the peripherals and drivers that we want to use.

A complete list of the needed configuration options is available on the Mirage OS wiki.

Since ours is a minimal configuration, we disabled the support for dynamic loading modules.

# make ARCH=arm multi_v7_defconfig
# make ARCH=arm menuconfig
# make ARCH=arm CROSS_COMPILE=armv7a-hardfloat-linux-gnueabi- zImage dtbs -j4

The dtbs target is needed to compile the device tree in the binary format used during boot. Once the compilation is terminated, the Linux zImage can be found as arch/arm/boot/zImage, whereas the binary device tree to be used is arch/arm/boot/dts/sun7i-a20-cubieboard2.dtb.

You need to copy both the zImage and the binary device tree in the root of the file system you created on the SD card containing the environment we are creating.

Publications @ CloudCon

Arianna Avanzini, Integrating Linux and the Real-Time ERIKA OS Through the Xen Hypervisor - Arianna Avanzini, CloudCon 2014


Acknowledgements

This work has been done thanks to Arianna Avanzini as part of her Master Thesis done under the supervision of Prof. Paolo Valente of the University Modena Reggio Emilia. This page is mainly a translation of instructions coming from Arianna.

Special Thanks also to Bruno Morelli who provided the initial iMX6 support, as well as the integration of the complete package inside the ERIKA Enterprise virtual machine.

Personal tools