Basic Flow

Obtain an URL for the web based spice console

openstack console url show –spice UUID

Stick the above url into a browser which will load spice console written in javascript, connecting back to spicehtml5proxy, which in turn forward traffics to the corresponding HV.

Obtain access url and token

request path

Client → Nova API → RPC (get_spice_console)→ Cell → RPC (get_spice_console) → HV

response path

Client ←(url+Token) ←  Nova API  ← Cell (Save token → connection info  into Console Auth Store with a TTL) ←  (host, port, url+token, token )  ← HV

On browser:

URL allows loading app

round 1.

browser → (url with token) → vip → spicehtml5proxy ( from default.web)

browser  ← vip ← (spice web based client) ← spicehtml5proxy

2.

browser ← → (websocket based spice traffics + token) ← →vip  ← →  spicehtml5proxy (obtain connection info by token from Console Auth) ←  (forward traffics)  → HV

* python >=2.74 is required on controller

Nova Controller setting

Hypervisor setting

HV will return connection details like where to connect back from proxy including host and port.

Token is also generated on HV, and cell manager add it into console Auth later, so is full access url with token, so html5proxy_base_url is required on every HV as well.

Console-auth & token store backend

By default Console-auth save everything in local dictionary. So each spicehtml5proxy or cell manager ( when authorize a new token) may talk to different console-auth which maintains it’s own independent token cache.

Fortunately token store is implemented with oslo_cache library  with caching back-end configurable.

In production a memcachd should be used so that console-auth can scale.

more details regarding  oslo cache configuration

https://docs.openstack.org/oslo.cache/latest/configuration/index.html

3 VIP/LB

Last, but not the least, Vip/LB has to  support either websocket, if doing L7, or just plain TCP,  since spice traffics requires upgrading http to websocket.

Live migration set up

block based migration or leaving it with default

# /etc/nova/nova.conf

block_migration_flag=VIR_MIGRATE_UNDEFINE_SOURCE,VIR_MIGRATE_PEER2PEER,VIR_MIGRATE_LIVE,VIR_MIGRATE_TUNNELLED,VIR_MIGRATE_NON_SHARED_INC,VIR_MIGRATE_AUTO_CONVERGE

====TCP based======

/etc/nova/nova.conf

——————

..

[libvirt]

live_migration_uri = “qemu+tcp://%s/system”

..

/etc/libvirt/libvirtd.conf

——————

listen_tls = 0

listen_tcp = 1

unix_sock_group = “libvirt”

unix_sock_ro_perms = “0777”

unix_sock_rw

_perms = “0770”

auth_unix_ro = “none”

auth_unix_rw = “none”

auth_tcp = “none”

/etc/default/libvirtd

——————————

start_libvirtd=”yes”

libvirtd_opts=”-l”

*restart both libvirt-bin & nova-compute *

testing

virsh -c qemu+tcp://hostname_peer/system hostname

====TCP+SASL ======

install

apt update

apt install sasl2-bin

/etc/nova/nova.conf

——————

..

[libvirt]

live_migration_uri = “qemu+tcp://%s/system”

..

/etc/libvirt/libvirtd.conf

——————

listen_tls = 0

listen_tcp = 1

unix_sock_group = “libvirt”

unix_sock_ro_perms = “0777”

unix_sock_rw

_perms = “0770”

auth_unix_ro = “none”

auth_unix_rw = “none”

auth_tcp = “sasl”

/etc/default/libvirtd

——————————

start_libvirtd=”yes”

libvirtd_opts=”-l”

/etc/sasl2/libvirt.conf

—————————————

mech_list: digest-md5

sasldb_path: /etc/sasldb2

create user test and give nova as password

———

saslpasswd2 -a libvirt test

sasldblistusers2 -f /etc/sasldb2

enable libvirt client auto authenticate without prompt

——

/etc/libvirt/auth.conf

——————————

[credentials-defgrp]

authname=test

password=nova

[auth-libvirt-default]

credentials=defgrp

*restart both libvirt-bin & nova-compute *

test, should just work, with not prompt for user&password

virsh -c qemu+tcp://hostname_peer/system hostname

=====================

====TLS full validation ======

key tools

apt-get install gnutls-bin

/etc/nova/nova.conf

——————

..

[libvirt]

live_migration_uri = “qemu+tls://%s/system”

..

/etc/libvirt/libvirtd.conf

——————

listen_tls = 1

tls_no_verify_certificate = 0

tls_no_verify_address = 0

listen_tcp = 0

unix_sock_group = “libvirt”

unix_sock_ro_perms = “0777”

unix_sock_rw

_perms = “0770”

auth_unix_ro = “none”

auth_unix_rw = “none”

auth_tls = “none”

/etc/default/libvirtd

——————————

start_libvirtd=”yes”

libvirtd_opts=”-l”

generate key pairs & certs following

https://wiki.libvirt.org/page/TLSSetup

service nova-compute restart

virsh -c qemu+tls://venus-2/system hostname

virsh -c qemu+tls://venus-6/system hostname

*restart both libvirt-bin & nova-compute *

test, should just work

virsh -c qemu+tls://hostname_peer/system hostname

======================

===TLS no validation ======

key tools

apt-get install gnutls-bin

/etc/nova/nova.conf

——————

..

[libvirt]

live_migration_uri = “qemu+tls://%s/system”

..

/etc/libvirt/libvirtd.conf

——————

listen_tls = 1

tls_no_verify_certificate = 1

tls_no_verify_address = 1

listen_tcp = 0

unix_sock_group = “libvirt”

unix_sock_ro_perms = “0777”

unix_sock_rw

_perms = “0770”

auth_unix_ro = “none”

auth_unix_rw = “none”

auth_tls = “none”

/etc/default/libvirtd

——————————

start_libvirtd=”yes”

libvirtd_opts=”-l”

generate key pairs & certs following

https://wiki.libvirt.org/page/TLSSetup

service libvirt-bin restart

service nova-compute restart

virsh -c qemu+tls://host-1/system hostname

virsh -c qemu+tls://host-2/system hostname

*restart both libvirt-bin & nova-compute *

testing

virsh -c “qemu+tls://host-1/system?no_verify=1” hostname

virsh -c “qemu+tls://host-2/system?no_verify=1” hostname

 

========TLS no validation + SASL========

following TLS no validation

a few changes

/etc/libvirt/libvirtd.conf

——————

listen_tls = 1

tls_no_verify_certificate = 1

tls_no_verify_address = 1

listen_tcp = 0

unix_sock_group = “libvirt”

unix_sock_ro_perms = “0777”

unix_sock_rw

_perms = “0770”

auth_unix_ro = “none”

auth_unix_rw = “none”

auth_tls = “sasl”

/etc/sasl2/libvirt.conf

—————————————

mech_list: digest-md5

sasldb_path: /etc/sasldb2

• scram-sha-1 requires properly signed certs

create user test and give nova as password

———

saslpasswd2 -a libvirt test

sasldblistusers2 -f /etc/sasldb2

libvirt client auto authenticate without prompt

——

/etc/libvirt/auth.conf

——————————

[credentials-defgrp]

authname=test

password=nova

[auth-libvirt-default]

credentials=defgrp

*restart both libvirt-bin & nova-compute *

testing

virsh -c “qemu+tls://host-1/system?no_verify=1” hostname

====ssh tunneling ======

No libvirt change required

only nova.conf and generate keys for user nova

nova.conf

live_migration_uri = “qemu+ssh://nova@%s/system?no_verify=1&keyfile=/var/lib/nova/.ssh/id_rsa”

see user nova  and path to its private key file is specified in the url

add public keys to authenticated hosts files.

Note: nova user login shell required.. further research required to limit set of binaries..