5. Deploying CloudFoundry with BOSH Director on OpenStack¶

Todo

Draft: to be deleted after the documentation is ready:

Configure OpenStack Domain,Project, User, Network for the deployment - Done in Section “4. Configure OpenStack”

2. Validate the OpenStack configuration using: https://github.com/eniware-org/cf-openstack-validator :

cf-openstack-validator installation done in section 5.2. CF-OpenStack-Validator installation

3. Setup the OpenStack projects for the BOSH and CloudFoundry installation using TerraForm modules from here: https://github.com/eniware-org/bosh-openstack-environment-templates

4. Install BOSH: https://bosh.io/docs/init-openstack/#deploy

Prepare and upload cloud-config.yml to BOSH to finilize the cloud configuration.
Deploy CloudFoundry.

In previous section we’ve configured OpenStack for use within a production environment. Various types of clients were installed for different OpenStack operations. We have set environment variables, external network, flavors, domain, project and user.

In this section we’ll create an environment that consists of a BOSH Director and Cloudfoudry deployment that it orchestrates.

5.1. Prerequisites¶

To be able to proceed, you need to the following:

Working OpenStack environment using both Juju and MAAS.
A user able to create/delete resource in this environment.
Flavors with properly configured names and settings.

5.2. CF-OpenStack-Validator installation¶

CF OpenStack Validator is an extension that verifies whether the OpenStack installation is ready to run BOSH and install Cloud Foundry. The intended place to run the validator is a VM within your OpenStack.

5.2.1. Prerequisites for CF-OpenStack-Validator¶

OpenStack configuration requirements are as follows:

Keystone v.2/v.3 Juju charm installed.
Created OpenStack project.
Created user with access to the previously created project (ideally you don’t want to run as admin).
Created network - connect the network with a router to your external network.
Allocated a floating IP.
Allowed ssh access in the default security group - create a key pair by executing:

$ ssh-keygen -t rsa -b 4096 -N "" -f cf-validator.rsa_id
Upload the generated public key to OpenStack as cf-validator.

A public image available in Glance.

The validator runs on Linux. Please ensure that the following packages are installed on your system:

ruby 2.4.x or newer
make
gcc
zlib1g-dev
libssl-dev
ssh

5.2.2. Installation of CF-OpenStack-Validator¶

To clone the CF-OpenStack-Validator repository:

git clone https://github.com/eniware-org/cf-openstack-validator

Navigate to the cf-openstack-validator folder:

cd cf-openstack-validator

Copy the generated private key into the cf-openstack-validator folder.

Copy validator.template.yml to validator.yml and replace occurrences of <replace-me> with appropriate values (see prerequisites):

If using Keystone v.3, ensure there are values for domain and project.

If using Keystone v.2, remove domain and project, and ensure there is a value for tenant. Also use the Keystone v.2 URL as auth_url.

$ cp validator.template.yml validator.yml

Download a stemcell from OpenStack stemcells bosh.io:

$ wget --content-disposition https://bosh.io/d/stemcells/bosh-openstack-kvm-ubuntu-trusty-go_agent

Install the following dependencies:

$ gem install bundler
$ bundle install

5.2.3. Additional configurations¶

CPI:

Validator downloads CPI release from the URL specified in the validator configuration. You can override this by specifying the --cpi-release command line option with the path to a CPI release tarball.

If you already have a CPI compiled, you can specify the path to the executable in the environment variable OPENSTACK_CPI_BIN. This is used when no CPI release is specified on the command line. It overrides the setting in the validator configuration file.

Extensions:

You can extend the validator with custom tests. For a detailed description and examples, please have a look at the extension documentation.

The eniware-org repository already contains some extensions. Each extension has its own documentation which can be found in the corresponding extension folder.

To learn about available cf-validator options run the following command:

$ ./validate --help

You can find more additional OpenStack related configuration options for possible solutions here.

5.3. Validate the OpenStack configuration¶

Before deploying Cloud Foundry, make sure to successfully run the CF-OpenStack-Validator against your project:

Make sure you have the required flavors on OpenStack by enabling the flavors extension with the flavors.yml file in this directory. Flavor names need to match those specified in the cloud config.
If you plan using the Swift ops file to enable Swift as blobstore for the Cloud Controller, you should also run the Swift extension.

To start the validation process type the following command:

$ ./validate --stemcell bosh-stemcell-<xxx>-openstack-kvm-ubuntu-trusty-go_agent.tgz --config validator.yml

5.4. Prepare OpenStack environment for BOSH and Cloud Foundry via Terraform¶

You can use a Terraform environment template to configure your OpenStack project automatically. You will need to create a terraform.tfvars file with information about the environment.

Important

The terraform scripts will output the OpenStack resource information required for the BOSH manifest. Make sure to treat the created terraform.tfstate files with care.

Hint

Instead of using Terraform, you can prepare an OpenStack environment manually as described here.

5.4.1. Install Terraform module¶

Make sure you have updated package database and installed unzip package:

sudo apt-get update
sudo apt-get install -y git unzip

To install the Terraform module:

git clone https://github.com/eniware-org/bosh-openstack-environment-templates.git
wget https://releases.hashicorp.com/terraform/0.11.11/terraform_0.11.11_linux_amd64.zip
unzip terraform_0.11.11_linux_amd64.zip
chmod +x terraform
sudo mv terraform /usr/local/bin/

5.4.2. OpenStack environment for BOSH¶

Setup an OpenStack project to install BOSH:¶

To setup an OpenStack project to install BOSH please use the following Terraform module. Adapt terraform.tfvars.template to your needs.

Create a working folder:
mkdir tmp
Copy the template file:
cp bosh-openstack-environment-templates/bosh-init-tf/terraform.tfvars.template tmp/terraform.tfvars
Generate a key pair executing the following script:
sh bosh-openstack-environment-templates/bosh-init-cf/generate_ssh_keypair.sh
Move the generated key pair to your working folder tmp:
mv bosh.* tmp/
Navigate to the working folder tmp:
cd tmp
Configure the Terraform environment template terraform.tfvars.

Run the following commands:
terraform init ../bosh-openstack-environment-templates/bosh-init-tf/
terraform apply ../bosh-openstack-environment-templates/bosh-init-tf/
Save the terraform.tfvars and terraform.tfstate files for bosh-init-tf:
mv terraform.tfvars bosh_terraform.tfvars
mv terraform.tfstate bosh_terraform.tfstate

Terraform tempalte file configuration for BOSH:¶

The content of the terraform tempalte file terraform.tfvars for BOSH is as follows:

auth_url = "<auth_url>"
domain_name = "<domain_name>"
user_name = "<ostack_user>"
password = "<ostack_pw>"
tenant_name = "<ostack_tenant_name>"
region_name = "<region_name>"
availability_zone = "<availability_zone>"

ext_net_name = "<ext_net_name>"
ext_net_id = "<ext_net_id>"

# in case your OpenStack needs custom nameservers
# dns_nameservers = 8.8.8.8

# Disambiguate keypairs with this suffix
# keypair_suffix = "<keypair_suffix>"

# Disambiguate security groups with this suffix
# security_group_suffix = "<security_group_suffix>"

# in case of self signed certificate select one of the following options
# cacert_file = "<path-to-certificate>"
# insecure = "true"

To edit the terraform.tfvars for BOSH using the described in this documentation scenario:

nano terraform.tfvars

Enter the following settings:

auth_url="http://192.168.40.228:5000/v3"
domain_name="cf_domain"
user_name="eniware"
password= <your_password>
tenant_name="cloudfoundry"
region_name="RegionOne"
availability_zone="nova"
ext_net_name="ext_net"
ext_net_id="db178716-7d8a-444b-854a-685feb5bf7ea"

auth_url is the URL of the Keystone service, which is http://192.168.40.228:5000/v3 in our case (it can be retrieved by using juju status | grep keystone/0 command).
The created domain cf_domain, with project cloudfondry and user eniware are set in the template in the domain_name, user_name, password, and tenant_name fields.
region_name can be retrieved when editing the Neutron config file or from here.
The defined external network is set in ext_net_name filed.
The ext_name_id identificator can be retrieved from the OpenStack web UI (go to Project > Network > Networks, click on ext_net and go to Overview tab) or by using the command openstack network list.

5.4.3. OpenStack environment for Cloud Foundry¶

Setup an OpenStack project to install Cloud Foundry:¶

To setup the project to install Cloud Foundry please use the following Terraform module. Adapt terraform.tfvars.template to your needs. Variable bosh_router_id is output of the previous BOSH terraform module.

Copy the template file terraform.tfvars file for cf-deployment-tf:
cp ../bosh-openstack-environment-templates/cf-deployment-tf/terraform.tfvars.template ./terraform.tfvars
Configure the Terraform environment template terraform.tfvars for cf-deployment-tf:

Run the following commands:
terraform init ../bosh-openstack-environment-templates/cf-deployment-tf/
terraform apply ../bosh-openstack-environment-templates/cf-deployment-tf/

Terraform tempalte file configuration for Cloud Foundry:¶

The content of the terraform tempalte file terraform.tfvars for Cloud Foundry is as follows:

auth_url = "<auth-url>"
domain_name = "<domain>"
user_name = "<user>"
password = "<password>"
project_name = "<project-name>"
region_name = "<region-name>"
availability_zones = ["<az-1>","<az-2>","<az-3>"]
ext_net_name = "<external-network-name>"

# the OpenStack router id which can be used to access the BOSH network
bosh_router_id = "<bosh-router-id>"

# in case Openstack has its own DNS servers
dns_nameservers = ["<dns-server-1>","<dns-server-2>"]

# does BOSH use a local blobstore? Set to 'false', if your BOSH Director uses e.g. S3 to store its blobs
use_local_blobstore = "<true or false>" #default is true

# enable TCP routing setup
use_tcp_router = "<true or false>" #default is true
num_tcp_ports = <number> #default is 100, needs to be > 0

# in case of self signed certificate select one of the following options
# cacert_file = "<path-to-certificate>"
# insecure = "true"

To edit the terraform.tfvars for Cloud Foundry using the described in this documentation scenario:

nano terraform.tfvars

Enter the following settings:

auth_url="http://192.168.40.228:5000/v3"
domain_name="cf_domain"
user_name="eniware"
password= <your_password>
tenant_name="cloudfoundry"
region_name="RegionOne"
availability_zones = ["nova", "nova", "nova"]
bosh_router_id = ""
dns_nameservers = ["8.8.8.8"]
use_local_blobstore = "true"
use_tcp_router = "true"
num_tcp_ports = 100

auth_url, domain_name, user_name, password, tenant_name, and region_name are the same as in terraform.tfvars template file for BOSH.
bosh_router_id can be retrieved from the output of the previous terraform script for BOSH.

5.5. Install BOSH¶

To install the BOSH director please follow the instructions on section 6. Isntall BOSH of this documentation.

For additional information you can visit bosh.io.

Make sure the BOSH director is accessible through the BOSH cli, by following the instructions on bosh.io. Use this mechanism in all BOSH cli examples in this documentation.

5.6. Cloud Config¶

After the BOSH director has been installed, you can prepare and upload a cloud config based on the cloud-config.yml file.

Take the variables and outputs from the Terraform run of cf-deployment-tf to finalize the cloud config.

Use the following command to upload the cloud config.

bosh update-cloud-config \
     -v availability_zone1="<az-1>" \
     -v availability_zone2="<az-2>" \
     -v availability_zone3="<az-3>" \
     -v network_id1="<cf-network-id-1>" \
     -v network_id2="<cf-network-id-2>" \
     -v network_id3="<cf-network-id-3>" \
     cf-deployment/iaas-support/openstack/cloud-config.yml

5.7. Deploy Cloud Foundry¶

To deploy Cloud Foundry run the following command filling in the necessary variables. system_domain is the user facing domain name of your Cloud Foundry installation.

bosh -d cf deploy cf-deployment/cf-deployment.yml \
     -o cf-deployment/operations/use-compiled-releases.yml \
     -o cf-deployment/operations/openstack.yml \
     -v system_domain="<system-domain>"

With Swift as Blobstore

Create four containers in Swift, which are used to store the artifacts for buildpacks, app-packages, droplets, and additional resources, respectively. The container names need to be passed in as variables in the below command snippet
Set a Temporary URL Key for your Swift account

Add the following lines to the deploy cmd:

-o cf-deployment/operations/use-swift-blobstore.yml \
-v auth_url="<auth-url>" \
-v openstack_project="<project-name>" \
-v openstack_domain="<domain>" \
-v openstack_username="<user>" \
-v openstack_password="<password>" \
-v openstack_temp_url_key="<temp-url-key>" \
-v app_package_directory_key="<app-package-directory-key>" \
-v buildpack_directory_key="<buildpack-directory-key>" \
-v droplet_directory_key="<droplet-directory-key>" \
-v resource_directory_key="<resource-directory-key>" \