Xen Hypervisor

From ErikaWiki

(Difference between revisions)
Jump to: navigation, search
(Creating a working system)
Line 96: Line 96:
The following sections describe how the environment can be installed into an SD card. Installing it over a USB flash drive is similar. This page does not contain instructions on using <tt>tftp</tt> or <tt>netboot</tt>.
The following sections describe how the environment can be installed into an SD card. Installing it over a USB flash drive is similar. This page does not contain instructions on using <tt>tftp</tt> or <tt>netboot</tt>.
 +
 +
== Preparing the toolchain ==
 +
Debian distributions as well as other distributions typically provide pre-compiled toolchains (on Ubuntu the package name is <tt>gcc-arm-linux-gnueabi</tt>).
 +
 +
However, the suggested procedure in our case is to use the <tt>crossdev</tt> application to generate a toolchain for the specific architecture:
 +
 +
<pre>
 +
crossdev -t armv7a-hardfloat-linux-gnueabi
 +
</pre>
 +
 +
== Compiling and installing the bootloader ==
 +
Use the following command to download and compile the [http://linuxsunxi.org/Bootable_SD_card#Bootloader U-Boot repository] with support for PSCI:
 +
 +
<pre>
 +
# git clone https://git.kernel.org/pub/scm/linux/kernel/git/maz/uboot.git u-boot-psci
 +
# cd u-boot-psci
 +
# git checkout origin/wip/psci
 +
# make CROSS_COMPILE=armv7a-hardfloat-linux-gnueabi- Cubieboard2_config
 +
# make CROSS_COMPILE=armv7a-hardfloat-linux-gnueabi-
 +
</pre>
 +
 +
The bootloader should be stored at an offset of 40KB, whereas the symbol table at an offset of 8KB.
 +
 +
Supposing we are using a Linux host, and that the SD card can be accessed using the device file <tt>/dev/mmcblk0</tt>, you can use the following commands:
 +
 +
<pre>
 +
# dd if=spl/sunxispl.bin of=/dev/mmcblk0 bs=1024 seek=8
 +
# dd if=uboot.img of=/dev/mmcblk0 bs=1024 seek=40
 +
</pre>
= Publications @ CloudCon =
= Publications @ CloudCon =

Revision as of 18:06, 12 January 2015

Contents

Introduction

This page describes the current support of ERIKA Enterprise for the Xen Hypervisor.

In particular, ERIKA can be run as a guest operating system inside a Xen domU.

Requirements for the working environment

The AllwinnerA20 CPU available in the cubieboard2 is directly supported by Xen; The Xen Wiki contains most of the documentation needed for building the software components.

This section summarizes the basic requirements of the development environment needed to start ERIKA Enterprise as a domU over a Linux dom0.

Firmware Requirements

The Hypervisor boot must happen in Hypervisor mode (EL2) (non secure).

Dupport for the EL2 boot mode is in the official repository since January 2014.

Moreover, to correctly init both cores of an SMP Platform, the bootloader needs to correctly handle the Power State Coordination Interface (PSCI); This support is not currently available in the official U-Boot repository, but a first working version is available in the development branch wip/psci of this repository.

dom0 requirements

Device tree.

All the platform features must be enumerated in a device tree, which must be compiled in the device tree binary format (also called device tree blob).

Starting with kernel 3.14, it is no more needed the creation of a dedicated device tree for the Linux boot on top of an hypervisor: it is in fact sufficient to [http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/arch/arm/boot/dts/sun7ia20cubieboard2. dts use the device tree provided together with Linux]

In other words, the bootloader must provide to Xen a dtb for the dom0 kernel, plus an optional one for initramfs (in case the kernel needs it). The bootloader also needs to load the kernel and initramfs in memory, and also to note their location in the device tree blob; That information is included into a new node of the device tree, named chosen, that contains:

  • the image location and a classification per compatibility
  • additional parameters to be passed to the hypervisor or to the dom0 during boot.

The chosen node must be defined in the following way (as specified in docs/misc/arm/devicetree/booting.txt:

chosen {
  bootargs = "$xen_cmdline";
  modules {
    module@0 { /* dom0 kernel */
      bootargs = "$dom0_cmdline";
      compatible = "xen,linuxzimage", "xen,multibootmodule";
      reg = <$kernel_addr $kernel_size>
    };
    module@1 { /* initramfs */
      compatible = "xen,linuxinitrd", "xen,multibootmodule";
      reg = <$initramfs_addr $initramfs_size>;
    };
  };
};

The chosen node can be manually added to the device tree (e.g., by changing the sources and recompiling), or can be created dynamically at boot time using the U-Boot command fdt.

domU requirements

domU executable format.

Xen uses the zImage format to load a ARM kernel. However, the image should not be compressed; the only needed operation is the compilation of the kernel source code with a zImage header.

That header must have an offset 0x24 inside the zImage and must contain:

  1. a magic number (0x016f2818) that identifies the file as zImage;
  2. the starting address of the zImage;
  3. The end address of the zImage.

The following is an header example:

_start:
  @ zImage header
.rept 8
  mov r0, r0
.endr
  b _start_cubieboard2
  .word 0x016f2818 @ Magic numbers to help the loader
  .word _start_0 @ absolute load/run zImage address
  .word _end _
  start @ zImage size
  @ end of zImage header
_start_cubieboard2:

Please remember the zImage should not be compressed. To obtain the executable in the required format you can use the command objcopy as done during the compilation of a normal application for Cortex Ax (the hypothesys is that the objcopy path is for the ARMv7 toolchain is in the environment variable EE_OBJCOPY):

$(EE_OBJCOPY) -O binary myProject_master.elf myProject_master.bin

base address for the executable image of the domU

Th base address for the adress space reserved to a domU is 0x80000000. The [xen/include/public/archarm.h:372 linker script] used for the compilation of the ERIKA source code must be adapted to that value.

Privileged Instructions

A non-privileged domain cannot access to the System Control Register to enable the memory management unit (by setting bit M); If done, a domU exception is raised.

Moreover, the hypervisor protects register CP15 c15 to non privileged guest domains. This because on some ARM architectures using that register is possible to have a dump of caches L1/L2: having free access to that register would then bring memory isolation problems.

IMPORTANT: Commit to be reverted

If you compile a Xen Repository including commit 878ff4fe1816d7f808f11254b555b9e9c2f121fa, this will provoke a critical shutdown of the ERIKA Enterprise domU at boot time. We did not yet understand why this happens. the only solution is currently to revert the commit.

Creating a working system

The following steps will guide through the creation of a working system including a Linux dom0 and an ERIKA Enterprise domU that will be executed under the Xen hypervisor.

The hypothesis is that a armv7a (hardfloat) toolchain is installed in the system. That toolchain must include at least a gcc compiler, the ld linker as well as the objcopy tool. The hypothesis is that the executables will have a name starting with armv7a-hardfloat-linux-gnueabi- (Please note that this prefix is typical for the Gentoo distributions. Debian packages typically have the prefix arm-linux-gnueabihf-.).

The following sections describe how the environment can be installed into an SD card. Installing it over a USB flash drive is similar. This page does not contain instructions on using tftp or netboot.

Preparing the toolchain

Debian distributions as well as other distributions typically provide pre-compiled toolchains (on Ubuntu the package name is gcc-arm-linux-gnueabi).

However, the suggested procedure in our case is to use the crossdev application to generate a toolchain for the specific architecture:

crossdev -t armv7a-hardfloat-linux-gnueabi

Compiling and installing the bootloader

Use the following command to download and compile the U-Boot repository with support for PSCI:

# git clone https://git.kernel.org/pub/scm/linux/kernel/git/maz/uboot.git u-boot-psci
# cd u-boot-psci
# git checkout origin/wip/psci
# make CROSS_COMPILE=armv7a-hardfloat-linux-gnueabi- Cubieboard2_config
# make CROSS_COMPILE=armv7a-hardfloat-linux-gnueabi-

The bootloader should be stored at an offset of 40KB, whereas the symbol table at an offset of 8KB.

Supposing we are using a Linux host, and that the SD card can be accessed using the device file /dev/mmcblk0, you can use the following commands:

# dd if=spl/sunxispl.bin of=/dev/mmcblk0 bs=1024 seek=8
# dd if=uboot.img of=/dev/mmcblk0 bs=1024 seek=40

Publications @ CloudCon

Arianna Avanzini, Integrating Linux and the Real-Time ERIKA OS Through the Xen Hypervisor - Arianna Avanzini, CloudCon 2014


Acknowledgements

This work has been done thanks to Arianna Avanzini as part of her Master Thesis done under the supervision of Prof. Paolo Valente of the University Modena Reggio Emilia. This page is mainly a translation of instructions coming from Arianna.

Special Thanks also to Bruno Morelli who provided the initial iMX6 support, as well as the integration of the complete package inside the ERIKA Enterprise virtual machine.

Personal tools