Justin Santa Barbara’s blog

Databases, Clouds and More

Creating an OpenStack Image

Here’s how I create an OpenStack disk image. Everyone has their own tweaks here; the most unusual thing I do is that I don’t use cloud-init, because I don’t want to rely on DHCP; I use OpenStack’s ‘configuration drive’ instead. I think that cloud-init has support for config-drive in its very latest versions, so I may revisit this in future.

A few comments:

  • I think that Grub is crazily complicated, requiring too many UUIDs to match up with each other; I use extlinux instread.
  • Some techniques inspired by vmdebootstrap
  • vmbuilder can only create ubuntu images, but we really want Debian server images to be an option (and RedHat etc)
  • Installing a (local) apt-cacher makes our life much more pleasant
  • I’ll probably bundle this into a script sometime soon!

The Host Machine

This documentation is for a Debian machine. Either use a real Debian machine, or use a Debian guest image, or install Debian in a guest!

In particular, if you’re using Ubuntu Oneirc, it won’t work until the fix is in for this extlinux bug.

Just use Debian for this!

Pre-reqs

1
2
3
4
5
6
7
8

mkdir -p images
cd images

sudo apt-get update
sudo apt-get install --yes debootstrap curl qemu-kvm mbr extlinux parted kpartx
# An apt cache makes subsequent image-building much faster
sudo apt-get install --yes apt-cacher-ng
sudo apt-get upgrade

Image Creation

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55

# Sadly, most of this needs to be done as root..
sudo bash

# Choose your image OS
OS=squeeze
KERNEL=linux-image-2.6-amd64


# Create a (sparse) 8 gig image
dd if=/dev/null bs=1M seek=8192 of=disk.raw

#Create partitions
parted -s disk.raw mklabel msdos
parted -s disk.raw mkpart primary 0% 100%
parted -s disk.raw set 1 boot on

# Install Master Boot Record
install-mbr disk.raw

# Mount the partitions
modprobe dm_mod
kpartx -av disk.raw
# Hopefully it’s loop0p1..
LOOPBACK_PARTITION=/dev/mapper/loop0p1

# Format filesystem
yes | mkfs.ext4 ${LOOPBACK_PARTITION}

# Don’t force a check based on dates
tune2fs -i 0 /dev/mapper/loop0p1

# Mount the disk image
mkdir mnt
mount ${LOOPBACK_PARTITION} mnt/

# Create root partition using debootstrap
http_proxy="http://127.0.0.1:3142" debootstrap \
      --include=openssh-server,${KERNEL} ${OS} mnt/

# Prepare for chroot
mount -t proc proc mnt/proc/

# Use chroot to fix up a few things (locales, mostly)
chroot mnt/ apt-get update
chroot mnt/ apt-get install locales
chroot mnt/ locale-gen en_US.utf8

chroot mnt/ /bin/bash -c "DEBIAN_FRONTEND=noninteractive dpkg-reconfigure locales"

# Finishing the image
chroot mnt/ apt-get upgrade
chroot mnt/ apt-get clean

# Remove persistent device names so that eth0 comes up as eth0
rm mnt/etc/udev/rules.d/70-persistent-net.rules

Set up /etc/fstab

1
2
3
4
5
6
7
8
9
10

PARTITION_UUID=`blkid -o value -s UUID ${LOOPBACK_PARTITION}`

echo -e "# /etc/fstab: static file system information." > mnt/etc/fstab
echo -e "proc\\t/proc\\tproc\\tnodev,noexec,nosuid\\t0\\t0" >> mnt/etc/fstab
echo -e "/dev/sda1\\t/\\text4\\terrors=remount-ro\\t0\\t1" >> mnt/etc/fstab
echo -e "" >> mnt/etc/fstab
# TODO: Swap

#Check it looks good
more mnt/etc/fstab

Use config drive

I highly recommend using config drive. There’s a little init script I contributed which can apply the network configuration from the config drive, located in contrib/openstack-config in the source tree of nova.

1
2
3
4
5

# Install the init script for config drive
curl https://raw.github.com/openstack/nova/master/contrib/openstack-config > mnt/etc/init.d/openstack-config
chown root:root mnt/etc/init.d/openstack-config
chmod 755 mnt/etc/init.d/openstack-config
chroot mnt/ /usr/sbin/update-rc.d openstack-config defaults

Set up extlinux

1
2
3
4
5
6

VMLINUZ=`chroot mnt find boot/ -name "vmlinuz-*"`
INITRD=`chroot mnt find boot/ -name "initrd*"`
echo -e "default linux\ntimeout 1\n\nlabel linux\nkernel ${VMLINUZ}\nappend initrd=${INITRD} root=UUID=${PARTITION_UUID} ro quiet" > mnt/extlinux.conf
cat mnt/extlinux.conf

extlinux --install mnt/

Unmount & convert to a qcow image

1
2
3
4
5
6
7
8
9
10
11

# Sync and unmount (with paranoid levels of sync-ing)
sync; sleep 2; sync
umount mnt/proc
umount mnt/
sync
kpartx -d disk.raw
sync

# Convert to qcow2 (which means that the raw image size isn’t too important)
rm disk.qcow2
qemu-img convert -f raw -O qcow2 disk.raw disk.qcow2

Upload the image into glance

1
2
3
4
5
6
7
8
9
10
11
12
13

# We don't need to be root any more...
exit

RAW_SIZE=`cat disk.qcow2 | wc -c`
echo "RAW_SIZE=${RAW_SIZE}"

# Change these parameters to the correct values
export OS_AUTH_USER=`whoami`
export OS_AUTH_KEY="secret"
export OS_TENANT_NAME=privatecloud
export OS_AUTH_URL=http://192.168.191.1:5000/v2.0/

glance add name=DebianSqueeze is_public=True disk_format=qcow2 container_format=bare image_size="${RAW_SIZE}" < disk.qcow2