The purpose of this guide is to provide you with an easy way to get OpenStack hands-on practice in a multi-domain environment for both CLI (OpenStack unified client) and Horizon. The default installation guides provided by OpenStack documentation allow only a single domain “default”.

This can be of big help for COA (OpenStack Foundation Certified OpenStack Administrator) exam takers.


Lab Requirement:

  • 6 Gig of RAM available for the VM
  • CPU supporting HW virtualisation
  • Hypervisor: VMware workstation or KVM
  • Centos DVD image: CentOS-7-x86_64-DVD.iso

Using either Vmware workstation or KVM create a virtual machine with the following minimum settings:

  • 6 GB RAM
  • 20 GB HDD (100 GB if you plan to test ubuntu, centos or other distribution cloud images)
  • 1 Network adapter (bridged to a host network accessible to the Internet)


Centos installation

Install Centos on the machine from CentOS-7-x86_64-DVD.iso image. The installation is straight forward. Make sure to disable SELinux and set a power user and root user passwords.

Afte rebooting the VM, set an appropriate hostname before starting OpenStack RDO installation.

As an example:

echo " rdo-newton rdo-newton.localdomain" >> /etc/hosts
hostnamectl set-hostname rdo-newton


Snapshot time: Just in case the installation of OpenStack goes wrong it is a good idea to take a snapshot of the just installed OS ex: “just installed centos”

OpenStack Installation

We will install the latest stable OpenStack release named Newton, even though, at the time of this writing, the exam is based on Liberty, it is about testing core knowledge of OpenStack administration not specificities of distributions (1).

From “The Operators’ View on the Certified OpenStack Administrator Exam” Video at 04:17, I quote the speaker:

“…that’s actually ok because you’re really

only being tested on the core stuff

that’s in OpenStack that’s really gonna

not change much of anything across the

different revisions …”

Prepare the host & install RDO all-in-one OpenStack

Installing RDO itself is very well documented and pretty straight forward (2).

Disable firewalld and NetworkManager

sudo systemctl disable firewalld
sudo systemctl stop firewalld
sudo systemctl disable NetworkManager
sudo systemctl stop NetworkManager
sudo systemctl enable network
sudo systemctl start network

Install repositories

sudo yum install -y centos-release-openstack-newton
sudo yum install -y
sudo yum update -y

Install packstack installer

sudo yum install -y openstack-packstack

Run the installation

sudo packstack --allinone

Sit back and relax, the installation will take ~ [5-10]min


Snapshot time: At this point you should have a default working OpenStack system, just in case you can take another snapshot of the just installed Openstack name it something like “just installed OpenStack”.

Enable multi-domain support

Now that OpenStack is installed, we need to enable multi-domain support for Horizon and use identity version 3 because, by default, OpenStack client doesn’t support “domain” command. Same for Horizon which doesn’t support domain-specific login and configuration.

openstack CLI client

Look for admin token in keystone configuration file /etc/keystone/keystone.conf

cat /etc/keystone/keystone.conf | grep ^admin_token

Here is an example of result:

admin_token = c5e7fd765e184219a7f46308320f9b72

Load the following environmental variable (3)

export OS_TOKEN=c5e7fd765e184219a7f46308320f9b72
export OS_URL=http://rdo-newton:35357/v3



Configure Horizon to use keystone v3 and multi-domain support

Edit the file at /usr/share/openstack-dashboard/openstack_dashboard/local/

'identity': 3,

Restart httpd

systemctl restart httpd

Now, in the login screen, you should have an additional input field for the domain:


New Identity policy

By default, Keystone uses /etc/keystone/policy.json which is a basic example of API protection and does not assume any particular use of domains.

A more advanced policy file named policy.v3cloudsample.json is available and more appropriate for multi-domain authentication (4).

cd /etc/keystone/

Backup the default policy file

cp policy.json policy.json.back

Replace it with the new one

cp policy.v3cloudsample.json policy.json

Domain management:

By default, only domain “default” is created, so let’s create a second domain with an admin user and a regular one:

Create a domain “domain1”:

openstack domain create domain1 --description "domain 1"

Create a project “domain1_project1” within the domain “domain1”:

openstack project create domain1_project1 --domain domain1

Create a user “domain1_admin”:

openstack user create domain1_admin --email "" --domain domain1 --password-prompt

Assign domain scope admin role to “domain1_admin”

openstack role add --user domain1_admin --domain domain1 admin

Create a regular user within the domain “domain1”

openstack user create domain1_user1 --email "" --domain domain1 --password-prompt

and here is what we get on domain-basis:


Connecting the cloud virtual machines to the outside world

One last thing, let’s prepare OpenStack to allow virtual machines to access outside world (5).

Openstack RDO All-in-one deployment uses as “public” network with default gateway configured on routers connected to the public network.

Note: What is qualified as “public” network is still from the rfc1918, the word is used to differentiate between the networks used in the cloud between VMs and networks outside the cloud.

In my case, the VM interface “eno16777736″ is bridged to my physical network, used by my host to reach the Internet, note that br-ex is DOWN by default and has no IP configuration.


Configure the interface eno16777736 with a static IP from your public network:

cat /etc/sysconfig/network-scripts/ifcfg-eno16777736 

Check your configuration:

[root@rdo-newton ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eno16777736: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:7c:03:26 brd ff:ff:ff:ff:ff:ff
inet brd scope global eno16777736
valid_lft forever preferred_lft forever
inet6 fe80::20c:29ff:fe7c:326/64 scope link
valid_lft forever preferred_lft forever

3: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
link/ether a2:c7:57:f9:19:f4 brd ff:ff:ff:ff:ff:ff
4: br-ex: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noop state DOWN
link/ether 12:9e:8b:fc:67:47 brd ff:ff:ff:ff:ff:ff

5: br-int: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
link/ether 7e:b8:3d:f1:a2:4f brd ff:ff:ff:ff:ff:ff
6: br-tun: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
link/ether ca:62:29:fe:77:43 brd ff:ff:ff:ff:ff:ff

The default gateway, mentioned above, is by default not configured on any interface, so all traffic attempting to go out will fail.

This can be observed during traffic capture of br-ex:

$tcpdump -i br-ex

tcpdump: WARNING: br-ex: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on br-ex, link-type EN10MB (Ethernet), capture size 65535 bytes
10:59:50.418966 ARP, Request who-has tell, length 28
10:59:51.420156 ARP, Request who-has tell, length 28
10:59:52.422163 ARP, Request who-has tell, length 28
10:59:54.421406 ARP, Request who-has tell, length 28
10:59:55.422171 ARP, Request who-has tell, length 28
10:59:56.424161 ARP, Request who-has tell, length 28

So, the solution is to :

  1. assign to br-ex
  2. enable forwarding
  3. configure NAT translation (MASQUERADE) to allow all traffic from cloud instances to appear as generated from eno16777736 IP address

Temporary settings

Enable routing on the host

sysctl -w net.ipv4.ip_forward = 1

Brint br-ex interface up and set its ip to router GW

ifconfig br-ex up
ip a a dev br-ex

Configure the host to perform NAT address translation on br-ex

iptables -I FORWARD -i br-ex -j ACCEPT
iptables -I FORWARD -o br-ex -j ACCEPT
iptables -t nat -I POSTROUTING -s ! -d -j MASQUERADE

Persistent settings

Create a script file accessoutside:

sysctl -w net.ipv4.ip_forward = 1
ifconfig br-ex up
ip a a dev br-ex
iptables -I FORWARD -i br-ex -j ACCEPT
iptables -I FORWARD -o br-ex -j ACCEPT
iptables -t nat -I POSTROUTING -s ! -d -j MASQUERADE

Make the script start at boot

cp accessoutside /etc/init.d/accessoutside
echo "/etc/init.d/accessoutside" >> /etc/rc.local

Now, from the CLI or from Horizon, you can create a private network, a key pair and a router, connect the private subnet to the router, create an instance connected to that subnet and start it:

Test a ping to



Snapshot time: If everything works as expected, make a snapshot and you can mess with the configuration.

That’s it folks!

Enjoy your COA preparation!







Looking for team training?