I'm a happy Debian user since 1997 and my friends are too, so I've never really looked at other distributions.

Debian Xen comes with good instructions including a config file, for installing a DomU/guest system. So until now, I just followed the instructions and never found out the nitty gritty of how an image is actually booted.

When a client asked me if they could have 'OpenSUSE 11.0 x64' installed on their DomU, I thought "sure. How hard can it be - just get an OpenSUSE install image, and modify some parameters in the config file to make it boot another image..."

Well, it is not actually that easy if neither the meaninig of these parameters, nor the basics of the whole process are properly documented anywhere. Google was giving me nothing but unintelligible half-documented stuff which is probably really clear if you already know the internals of the Xen boot process. The only 'understandable' documentation for installing an OpenSUSE DomU was... from SUSE itself, and that went "Start this graphical SUSE application and click this button". Nope, not applicable in my case.

What I found out after hours, days, of hit & miss experimenting:

General stuff

1)

Xen does not usually/by default use kernels on the DomU's file system to boot its DomU. If you just specify 'kernel' & 'ramdisk' parameters, those represent files which will be read from the Dom0's native file system.

2)

If you want to start a kernel & ramdisk that resides on the DomU's file system, there are programs for that. Debian (and most other systems, I think) ships with 'pygrub', which is kind of a grub replacement that 'extracts' the kernel and ramdisk from the DomU file system and 'feeds' it to Xen, when it starts up a DomU.

I'm not up to date on the gory technical details. I just found out that if you specify parameter "bootloader=/path/to/pygrub" and then 2 parameters "kernel" & "ramdisk" representing paths on the DomU's file system, the DomU boots that kernel/ramdisk. And if you don't specify the "kernel" & "ramdisk" parameters, pygrub will try and read a 'grub menu list' from your DomU file system and present you with choices.

In other words: if you start your DomU and see a 'grub screen', it is not your DomU running yet. (It can't be; Xen does not boot 'a virtual computer' for you, it loads and boots a kernel. So your DomU can never run machine code off your file system's boot sector.) It is pygrub (a python program running on your Dom0), emulating grub behaviour for you.

But I digress Only the first paragraph is important for booting OpenSUSE, in itself.

OpenSUSE stuff

OpenSUSE 11.0 does not have a 'text mode' installation procedure. Despite what the ambiguous info on the internetz says, that text mode probably existed up tp 10.x... but not in 11 anymore.

There is (in my case) one good kernel/ramdisk install image on the OpenSUSE ISO install media. When I configured my Xen config file with "bootloader=/path/to/pygrub / kernel=/boot/x64_86/vmlinuz / ramdisk=/boot/x64_86/vmlinuz" (or something like that), the DomU booted, albeit with a few kernel errors. And then seemed to totally 'hang'.
(And I lost hours trying to trying to recreate install images, a process which had some hurdles when doing that on Debian1, but in the end I had made my own images, which... also seemed to hang.)

The solution: forget text mode. Configure your xen .cfg file with another option: "vfb = [ 'type=vnc' ]". Then if you (after booting) connect a vnc client to 127.0.0.1:5900, you see... an install screen!

I have no idea what I have actually done next, and whether the things that went wrong next are really wrong, or whether I've just overlooked some things. Anyway...
I had configured the "disk" parameter in the xen DomU .cfg file to have two images: one being the install media (which in my case was a DVD ISO image on disk), one being the 'empty disk' that OpenSUSE should be installed on. But the installer did not recognise any 'CD/DVD media', at all, no matter what arguments I tried for the "disk" parameter. So I copied the DVD image over into an ext3 formatted loopback file system, specified that to the DomU as an 'xvd' disk, and tried the "install from hard disk partition" option in OpenSUSE installer. That did not work for about 4 times, and worked the 5th time- I can't tell you why, or the exact location of the correct menu entry.

But in the end, I was installing an OpenSUSE system on a fresh DomU on my Debian Lenny server. Yay!

So the troubles were over...?

Ofcourse not.

After I edited the Xen config to boot the now-installed OpenSUSE image, I got a nice (py)grub screen, the kernel booted, and... init segfaulted. How nice.

After uttering some profanities and leaving it alone for a day, I downloaded and installed OpenSUSE 11.1 (instead of 11.0). That worked without a problem. The system booted up perfectly.

A day later, my client came back saying the DomU did not boot anymore. He had updated his system (using normal OpenSUSE update mechanisms that I don't know about), and now his system was unresponsive.

It turns out that the kernel that was installed, could not recognise the 'xvd' file system. So apparently, after the kernel boots, it needs to load the initrd image from disk... and this kernel was unable to do that. (And my client, or the SUSE update system, had deleted the 'old' working kernel already.)
Again, I don't know the internal details of linux kernel booting well enough. It turned out that when I configured the "kernel" and "ramdisk" parameters in the Xen DomU's .cfg file (instead of booting a kernel through the pygrub selection screen), the kernel and initrd were loaded correctly through pybrub, and the system started.

My client and I decided to leave it at that. Even more time investment was not deemed necessary at this point. So now they're running a machine with a hardcoded kernel/ramdisk path in a config file that they need to edit when booting a new kernel. It would be nice to have a (py)grub screen instead, when rebooting... but for now, we can live with it.

Conclusion

Installing and booting a Xen DomU from another distro (i.e. a combination which has not been thoroughly tested) works. Most of the time. With recent stock kernels. If you know what you're doing and are not discouraged by a few failures and frustrations at first.

I included my config file just for reference. The first part is the stock Debian config with (mostly) stock Debian comments; the last part is mine - and not everything is properly documented, but at least it's more than nothing.

• 1. primarily because Debian 'rpm' cannot read OpenSUSE 11 .rpm files. They're compressed, so 'rpm' does not work on them; one needs to do 'cat XXX.rpm | rpm2cpio | lzma -d | cpio -ivd'