----
OpenNebula 4.8 With Ceph Support on Debian Wheezy
// Ceph

A quick howto to install OpenNebula 4.8 with support for Ceph on Debian Wheezy.

1  2  3  4  5  6  7  8  9  10  11  12

$ onedatastore show cephds  DATASTORE 101 INFORMATION  ID : 101  NAME : cephds  USER : oneadmin  GROUP : oneadmin  CLUSTER : -  TYPE : IMAGE  DS_MAD : ceph  TM_MAD : ceph  BASE PATH : /var/lib/one//datastores/101  DISK_TYPE : RBD

OpenNebula Installation

OpenNebula Frontend

Install OpenNebula repo

wget -q -O- http://downloads.opennebula.org/repo/Ubuntu/repo.key | apt-key add -  echo "deb http://downloads.opennebula.org/repo/4.8/Debian/7/ stable opennebula" > /etc/apt/sources.list.d/opennebula.list

Download packages

apt-get update  apt-get install opennebula opennebula-sunstone nfs-kernel-server

Configure and start the service

sed -i -e 's/:host: 127.0.0.1/:host: 0.0.0.0/g' /etc/one/sunstone-server.conf  /etc/init.d/opennebula-sunstone restart

Export NFS

echo "/var/lib/one/ *(rw,sync,no_subtree_check,root_squash)" >> /etc/exports  service nfs-kernel-server restart

Configure SSH Public Key

su - oneadmin  $ cp ~/.ssh/id_rsa.pub ~/.ssh/authorized_keys      $ cat << EOT > ~/.ssh/config  Host *      StrictHostKeyChecking no      UserKnownHostsFile /dev/null  EOT  $ chmod 600 ~/.ssh/config

OpenNebula Compute Nodes

Install OpenNebula repo

wget -q -O- http://downloads.opennebula.org/repo/Ubuntu/repo.key | apt-key add -  echo "deb http://downloads.opennebula.org/repo/4.8/Debian/7/ stable opennebula" > /etc/apt/sources.list.d/opennebula.list

Download packages

apt-get update  apt-get install opennebula-node nfs-common bridge-utils

Network config

vim /etc/network/interfaces      ...      bridge_ports eth0      bridge_fd 9      bridge_hello 2      bridge_maxage 12      bridge_stp off

Fstab

vim /etc/fstab  10.2.0.130:/var/lib/one/  /var/lib/one/  nfs   soft,intr,rsize=8192,wsize=8192,noauto  mount /var/lib/one/

Qemu config

cat << EOT > /etc/libvirt/qemu.conf  user  = "oneadmin"  group = "oneadmin"  dynamic_ownership = 0  EOT

Ceph configuration

I suppose that you have already a Ceph cluster running.

Now create a pool "one" for Opennebula and create auth key.

ceph osd pool create one 128 128  ceph osd pool set one crush_ruleset 2  ceph auth get-or-create client.one mon 'allow r' osd 'allow rwx pool=one'  [client.one]      key = AQCfTjVUeOPqIhAAiCAiBIgYd85fuMFT0dXVpA==

Add Ceph firefly for support rbd on libvirt + qemu

Before, you need compile rbd support for libvirt and qemu. You can have a look to this post : http://cephnotes.ksperis.com/blog/2013/09/12/using-ceph-rbd-with-libvirt-on-debian-wheezy (If you do not want to compile qemu you can directluy download those packages here : http://ksperis.com/files/qemu-kvm_1.1.2+dfsg-6_amd64.deb, http://ksperis.com/files/qemu-kvm_1.1.2+dfsg-6_amd64.deb)

On each node, add Ceph firefly repo, and reinstall libvirt and qemu packages

apt-get install lsb-release  wget -q -O- 'https://ceph.com/git/?p=ceph.git;a=blob_plain;f=keys/release.asc' | sudo apt-key add -  echo deb http://ceph.com/debian-firefly/ $(lsb_release -sc) main | sudo tee /etc/apt/sources.list.d/ceph.list  apt-get update  apt-get install librados2 librbd1    dpkg -i qemu-kvm_1.1.2+dfsg-6+deb7u4_amd64.deb qemu-utils_1.1.2+dfsg-6a+deb7u4_amd64.deb

Create the secret for libvirt :

echo "  <secret ephemeral='no' private='no'>     <usage type='ceph'>       <name>client.one secret</name>     </usage>  </secret>" > secret.xml      service libvirt reload

Add ceph datastore on OpenNebula

$ su - oneadmin  $ vim ceph.one  NAME         = cephds  DS_MAD       = ceph  TM_MAD       = ceph  DISK_TYPE    = RBD  POOL_NAME    = one  BRIDGE_LIST  ="onenode1 onenode2 onenode3"  CEPH_HOST    ="192.168.0.1:6789 192.168.0.2:6789 192.168.0.3:6789"  CEPH_SECRET  ="26a8b4d7-eb24-bf85-396d-fbf0x252e402"  CEPH_USER    ="one"    $ onedatastore create ceph.one  ID: 101

Example config :

$ onedatastore show cephds  DATASTORE 101 INFORMATION                                                         ID             : 101                   NAME           : cephds                USER           : oneadmin              GROUP          : oneadmin              CLUSTER        : -                     TYPE           : IMAGE                 DS_MAD         : ceph                  TM_MAD         : ceph                  BASE PATH      : /var/lib/one//datastores/101  DISK_TYPE      : RBD                     DATASTORE CAPACITY                                                                TOTAL:         : 15.9T                 FREE:          : 13.4T                 USED:          : 2.5T                  LIMIT:         : -                       PERMISSIONS                                                                       OWNER          : um-                   GROUP          : u--                   OTHER          : ---                     DATASTORE TEMPLATE                                                                BASE_PATH="/var/lib/one//datastores/"  BRIDGE_LIST="onenode1 onenode2 onenode3"  CEPH_HOST="192.168.0.1:6789 192.168.0.2:6789 192.168.0.3:6789"  CEPH_SECRET="26a8b4d7-eb24-bf85-396d-fbf0x252e402"  CEPH_USER="one"  CLONE_TARGET="SELF"  DISK_TYPE="RBD"  DS_MAD="ceph"  LN_TARGET="NONE"  POOL_NAME="one"  TM_MAD="ceph"  TYPE="IMAGE_DS"    IMAGES           9                10               11               12               16               21               24               27               28               29               31               32

Try import image in new datastore :

$ oneimage create      --name "CentOS-6.5_x86_64"      --path "http://appliances.c12g.com/CentOS-6.5/centos6.5.qcow2.gz"      --driver qcow2      --datastore cephds

More details here :

http://ceph.com/community/ceph-support-in-opennebula-4-0/

http://archives.opennebula.org/documentation:archives:rel4.0:ceph_ds

----

Shared via my feedly reader

Sent from my iPhone