LXD

Contents

Summary

Orabuntu-LXC v7.0.0-beta AMIDE (Amazon Mult-I-host LXD Docker Enterprise) is the first edition of Orabuntu-LXC which offers support for LXD and for LXD Clusters.

OpenvSwitch Design in Orabuntu-LXC version 6

Orabuntu-LXC in version 6 introduced the multi-host automated OpenvSwitch GRE tunnel building feature which allowed users to span a network of LXC containers across multiple physical hosts and/or virtual machines (or mixtures of physical hosts and virtual machines) using GRE tunnels over a Multi-Host OpenvSwitch network. In version 6, Orabuntu-LXC still used the "server-side" scripts to implement the VLAN scheme for the LXC containers. More information on how networking and VLAN was implemented in Orabuntu-LXC version 6 can be found here.

In particular, Orabuntu-LXC in version 6 used scripts in the LXC config file to accomplish the attachment to OpenvSwitch for example as shown below.

# Networking
lxc.network.type = veth
lxc.network.flags = up
lxc.network.script.up = /etc/network/if-up.d/openvswitch/xxxxxxxx-pub-ifup-sw10
lxc.network.script.down = /etc/network/if-down.d/openvswitch/xxxxxxxx-pub-ifdown-sw10
lxc.network.veth.pair = xxxxxxxx
lxc.network.name = eth0
lxc.network.mtu = 1500

lxc.network.hwaddr = 00:16:3e:c7:48:d8

Below is an example of the script that was used in Orabuntu-LXC version 6.

#!/bin/bash
ovsBr='sw10'
ovs-vsctl add-port ${ovsBr} $5
ovs-vsctl set port $5 tag=12

And here is what the /etc/network/if-up.d/openvswitch/xxxxxxxx-pub-ifup.sw11 file looks like as shown below.

#!/bin/bash
ovsBr='sw11'
ovs-vsctl add-port ${ovsBr} $5
ovs-vsctl set port $5 tag=13

OpenvSwitch Design in Orabuntu-LXC version 7

The changes to OpenvSwitch design in Orabuntu-LXC version 7 were inspired by a post here. This method allows Orabuntu-LXC version 7 to preserve the VLAN scheme of Orabuntu-LXC version 6, but to also eliminated the server-side scripts mentioned above. Note that this also requires a version of LXC itself that supports OpenvSwitch in the config file of the container (i.e. the "lxc.network.link" parameter). Therefore, additional work was done on the code that will be released as Orabuntu-LXC version 7 to deploy minimum version LXC 3.0.4 on all linux distros that Orabuntu-LXC supports (currently Oracle Linux, Ubuntu, RedHat, CentOS, and Fedora).

In Orabuntu-LXC version 7 the VLAN scheme is preserved, not by the server-side scripts, but by new additional ovs bridges attached to the "core" ovs bridges which carry the vlan information, for example as shown below in bold from the crt_ovs_sw1.sh script.

ovs-vsctl --may-exist add-br sw1
ovs-vsctl --may-exist add-br sw1a sw1 10

The sw1a bridge provides the vlan information, while the sw1 switch handles the iptables rule-based connection to the physical internet-connected interface and the GRE tunnel connections to other interfaces on physical hosts or virtual machines to create the GRE tunnel network. The VLAN information is bestowed on LXC containers that connect to the sw1a vlan-tagged switch. Bridge sw1 has "trunks=10, ..." setting which allows the desired traffic to traverse the sw1 switch.

Currently Orabuntu-LXC version 7 has not yet been released (as of this writing on February 25, 2021) because work is still being done to determine how to best put the LXD containers on these new vlan-tagged switches such as "sw1a." So for now, it is not yet coded into Orabuntu-LXC how to put LXD containers on the OpenvSwitch network. Nevertheless, the networking changes were made to utilize this new layer of "vlan-tagged-switches" such as sw1a and sx1a so that the "lxc.network.scripts" parameter and "lxc.net.0.network.scripts" parameters in the config files for the LXC containers could finally be retired and removed from the code while still preserving the existing VLAN scheme of Orabuntu-LXC. This design change was a significant improvement to Orabuntu-LXC version 6, and, it is hoped, lays the foundation for expansion of Orabuntu-LXC to completely support LXD containers and LXD clusters. It is the use of the new vlan-tagged-switches for use with LXD that is still under development. In the meantime, LXD containers can be created on the lxdbr0 as usual, with the Orabuntu-LXC OpenvSwitch network providing the LXD Cluster endpoints, and can be managed as usual leveraging the OpenvSwitch network.

LXD Clusters on Orabuntu-LXC version 7

However, what is already available for use in Orabuntu-LXC version 7 is the ability to create LXD clusters automatically. At this point, this feature is only available for Ubuntu Linux 20.04 and that code is already available for use in Orabuntu-LXC version 6.13.24-beta AMIDE and also in Master. However, it is "switched off" and so to use it one needs to edit the

anylinux-services.HUB.HOST.sh file; and

anylinux-services.GRE.HOST.sh file

and also needs to create the required zpools for LXD storage (and the code checks and exits if you have forgotten to pre-create the zfs storage pools). The program expects you to create a pool on the LXD cluster master, and a pool on the LXD Cluster second server, shown below respectively, as:

olxc-001

olxc-002

Currently, there is no flexibility in the code to name your zpools to custom user-selectable names, but this is a relatively easy feature to introduce, so it will be possible to use custom zfs storage pool names when Orabuntu-LXC version 7 is released. For now, in the 6.13.24-beta AMIDE "preview" one needs to create the zpools with those names.

The cluster is created fully automatically, simply by setting the required install parameters in the anylinux-services.[HUB|GRE].HOST.sh files. Edit the files and set this section to "Y" as shown below in bold.

################ LXD Cluster Settings ######################

### Ubuntu Linux LXD Storage (optional)

StoragePoolName=olxc-002        # Relevant only if LXDCluster=Y
StorageDriver=zfs               # Relevant only if LXDCluster=Y

### Oracle Linux LXD Storage (optional)

BtrfsLun="\/dev\/sdXn"          # Relevant only if LXDCluster=Y (e.g. Set to /dev/sdb1)
LXD=N                           # This value is currently unused. Leave set to N.

LXDCluster=N                    # Default value
PreSeed=N                       # Default value

if   [ $LinuxFlavor = 'Ubuntu' ] && [ $UbuntuMajorVersion -ge 20 ]
then
        echo ''
        echo "=============================================="
        echo "Display Optional LXD Cluster Values...        "
        echo "=============================================="
        echo ''

        LXDCluster=N    # Set to Y for automated LXD Cluster creation (optional).
        PreSeed=N       # Set to Y for automated LXD Cluster creation (optional).

        echo 'LXDCluster = '$LXDCluster
        echo 'PreSeed    = '$PreSeed

        echo ''
        echo "=============================================="
        echo "Done: Display LXD Cluster Values.             "
        echo "=============================================="
        echo ''

        sleep 5

        clear

        if [ $LXDCluster = 'Y' ]
        then
                echo ''
                echo "=============================================="
                echo "Check ZFS Storage Pool Exists...              "
                echo "=============================================="
                echo ''

                function CheckZpoolExist {
                        sudo zpool list $StoragePoolName | grep ONLINE | wc -l
                }
                ZpoolExist=$(CheckZpoolExist)

                if [ $ZpoolExist -eq 1 ]
                then
                        echo "ZFS $StoragePoolName exists."
                else
                        echo "ZFS $StoragePoolName does not exist."

                        echo "ZFS $StoragePoolName must be created before running Orabuntu-LXC in LXD Cluster Mode."

                        echo "Orabuntu-LXC Exiting."
                        exit
                fi

                echo ''
                echo "=============================================="
                echo "Done: Check ZFS Storage Pool Exists.          "
                echo "=============================================="
                echo ''

                sleep 5

                clear
        fi
fi

if [ $LinuxFlavor = 'Oracle' ] && [ $Release -eq 8 ]
then
        LXDCluster=N
        PreSeed=N

        if [ $LXDCluster = 'Y' ]
        then
                echo ''
                echo "=============================================="
                echo "                WARNING !!                    "
                echo "=============================================="
                echo ''
                echo "=============================================="
                echo "LXD Cluster will RE-FORMAT $BtrfsLun as a     "
                echo "BTRFS file system for LXD.                    "

                echo "                                              "

                echo "If you do NOT want to use /dev/sdXn for this "
                echo "purpose, hit CTRL+c now to exit.              "
                echo "=============================================="
                echo ''

                sleep 20
        fi
fi

################## LXD Cluster Settings END #########################

As shown above, work is also underway to provide automated LXD cluster creation for Oracle Linux 8 hosts as well, so that in Orabuntu-LXC version 7 there will be the ability to create LXD cluster automatically on at least one Debian-family Linux (Ubuntu 20.04) and at least one RedHat-family Linux (Oracle Linux 8). Work is also planned to offer LXD clusters on Fedora 33 as well.

At this point in development, if using Orabuntu-LXC 6.13.24-beta AMIDE to create and use LXD, create the required olxc-001/olxc-002 zfs storage pools on the HUB host and GRE host, respectively, and then set the switches shown above to "Y" and the cluster will be created automatically by Orabuntu-LXC which puts the LXD cluster on the OpenvSwitch network utilizing the GRE tunnels that Orabuntu-LXC creates as part of the multi-host option. When the install is complete on Ubuntu Linux 20.04, create LXD containers on the lxdbr0 interface. The LXD containers created in that way will be available in the usual expected way across all Orabuntu-LXC hosts in the multi-host OpenvSwitch network.

Comments

	Search this site