Bootstrap a full stack installation¶
The command yorc bootstrap
can be used to bootstrap the full stack, from Alien4Cloud
to Yorc and its dependencies, over different types of infrastructures, on a single node
or distributed on several nodes.
Prerequisites¶
Hosts¶
The local host from where the command yorc bootstrap
will be run, as well as
remote hosts where the full stack will be deployed, should be Linux x86_64 systems
operating with at least 2 CPUs and 4 Go of RAM.
The bootstrap was validated on:
- CentOS 7,
- Red Hat Enterprise Linux 7.5,
- Ubuntu 19.04 (which is installing python3 by default, see bootstrap configuration file example below for specific Ansible configuration settings needed for remote hosts using python3).
Packages¶
The bootstrap operation will install a basic Yorc setup on the local host.
It requires the following packages to be installed on the local host:
- python and python-pip (or python3/python3-pip)
- zip/unzip
- openssh-client
- openssl (to generate certificates when they are not provided)
- wget
This basic installation on the local host will attempt to install without sudo privileges python ansible module 2.7.9 if needed as well as python packages MarkupSafe, jinja2, PyYAML, six, cryptography, setuptools. So if this ansible module or python packages are not yet installed on the local host, you could either add them yourself, with sudo privileges, running for example:
sudo pip install ansible==2.7.9
Or you could create a python virtual environment, and let the Yorc bootstrap command install the ansible module within this virtual environment (operation which doesn’t require sudo privileges).
You can run these commands to create a virtual environment, here a virtual
environment called yorcenv
:
sudo pip install virtualenv
virtualenv yorcenv
source yorcenv/bin/activate
You are now ready to download Yorc binary, running:
wget https://github.com/ystia/yorc/releases/download/v4.0.1/yorc-4.0.1.tgz tar xzf yorc-4.0.1.tgz ./yorc bootstrap --help
Define firewall rules on your Cloud Provider infrastructure¶
To access Alien4Cloud UI from the local host, you may need to define firewall rules before attempting to bootstrap the full stack on a cloud provider infrastructure.
For example on Google Cloud, you could define this firewall rule for the port 8088
used by the UI, and associate it to a tag (here a4c
) :
$ gcloud compute firewall-rules create a4c-rule \
--allow tcp:8088 --target-tags a4c
You could then specify this tag a4c
in the compute instance to create by the
bootstrap deployment, as it is done in Google Configuration file example.
This way the created compute instance where Alien4Cloud will be deployed will
have its port 8088 open.
Bootstrap process overview¶
The command yorc bootstrap
will configure on the local host a basic setup with
Yorc and a Consul data store.
This basic setup will then be used to bootstrap the full stack Alien4Cloud/Yorc and its dependencies over a selected location.
You can deploy the full stack either on a single node (by default), or distributed
on several nodes as described in Run Yorc in HA mode, using yorc bootstrap
command line option --deployment_type HA
described below.
When flag --insecure
is not specified, a secured installation will performed:
- TLS with mutual authentication between components will be configured,
- a Vault will be installed and used to store location credentials.
Configuration values can be provided by the user:
- in interactive mode,
- through a configuration file,
- using
yorc bootstrap
command line options- using environment variables.
You can combine these modes, yorc bootstrap
will check in any case if
required configuration values are missing, and will ask for missing values.
These configuration values will allow you to specify:
- optional Alien4Cloud configuration values
- Yorc configuration values, all optional, except from:
- the path to a ssh private key that will be used by the local orchestrator to connect to the bootstrapped setup
- the Certificate authority private key passphrase to use in the default secure mode (while the other properties, Certificate Authority private key and PEM-encoded Certificate Authority, are optional. If not provided, they will be generated, and the generated Certificate Authority at
work/bootstrapResources/ca.pem
can then be imported in your Web browser as a trusted Certificate Authority)- Location configuration with required configuration values depending on the infrastructure type, as described at Locations Configuration
- Configuration of compute Nodes to create on demand,
- User used to connect to these compute nodes,
- Configuration of the connection to public network created on demand.
Details of these on-demand resources configuration values are provided in the Alien4Cloud
Yorc plugin Documentation at https://yorc-a4c-plugin.readthedocs.io/en/latest/location.html.
For example, in the Google Configuration file example, you can see on-demand compute
and address
configuration values.
Once configuration settings are provided, yorc bootstrap
will proceed to the
full stack deployment, showing deployment steps progress (by default, but you can see
deployment logs instead trough the option --follow logs
described below).
Once the deployment is finished, the orchestrator on the local host is still running,
so you can perform commands like ./yorc deployments list
, ./yorc deployments logs -b
, etc…
Or perform any deployment troubleshooting if needed.
To undeploy a bootstrapped setup, you can also use the CLI, running ./yorc deployments undeploy <deployment id>
.
To clean the local host setup, run:
./yorc bootstrap cleanup
This will only clean the local host environment, it won’t undeploy the bootstrapped setup installed on remote hosts. It stops the local yorc and consul servers, cleans files in working directory except from downloaded bundles, on purpose as some of them take time to be downloaded.
Bootstrapping the setup in interactive mode¶
You can bootstrap the setup in interactive mode running:
./yorc bootstrap [--review]
You will have then to select the infrastructure type (Google Cloud, AWS, OpenStack, Hosts Pool) and provide a name to the location on which you want to deploy the full stack, then you will be asked to provide configuration values depending on the infrastructure type.
The command line option --review
allows to review and update all configuration
values before proceeding to the deployment, opening the editor specified in the
environment variable EDITOR
if defined or using vi or vim if available.
Bootstrapping the setup using command line options¶
The following yorc bootstrap
option are available:
--alien4cloud_download_url
Alien4Cloud download URL (defaults to the Alien4Cloud version compatible with this Yorc, under https://www.portaildulibre.fr/nexus/repository/opensource-releases/alien4cloud/alien4cloud-premium-dist/)--alien4cloud_password
Alien4Cloud password (default, admin)--alien4cloud_port
Alien4Cloud port (default 8088)--alien4cloud_user
Alien4Cloud user (default, admin)--ansible_extra_package_repository_url
URL of package indexes where to find the ansible package, instead of the default Python Package repository--ansible_use_openssh
Prefer OpenSSH over Paramiko, python implementation of SSH--ansible_version
Ansible version (default 2.7.9)--config_only
Makes the bootstrapping abort right after exporting the inputs--consul_download_url
Consul download URL (default, Consul version compatible with this Yorc, under https://releases.hashicorp.com/consul/)--consul_encrypt_key
16-bytes, Base64 encoded value of an encryption key used to encrypt Consul network traffic--consul_port
Consul port (default 8543)--credentials_user
User Yorc uses to connect to Compute Nodes--deployment_name
Name of the deployment. If not specified deployment name is based on time.--deployment_type
Define deployment type: single_node or HA (default, single_node)--follow
Follow bootstrap deployment steps, logs, or none (default, steps)--infrastructure
Define the type of infrastructure where to deploy Yorc: google, openstack, aws, hostspool--insecure
Insecure mode - no TLS configuration, no Vault to store secrets--jdk_download_url
Java Development Kit download URL (default, JDK downloaded from https://edelivery.oracle.com/otn-pub/java/jdk/)--jdk_version
Java Development Kit version (default 1.8.0-131-b11)--location
Name identifying the location where to deploy Yorc--resources_zip
Path to bootstrap resources zip file (default, zip bundled within Yorc)--review
Review and update input values before starting the bootstrap--terraform_download_url
Terraform download URL (default, Terraform version compatible with this Yorc, under https://releases.hashicorp.com/terraform/)--terraform_plugins_download_urls
Terraform plugins download URLs (default, Terraform plugins compatible with this Yorc, under https://releases.hashicorp.com/terraform-provider-xxx/)--values
Path to file containing input values--vault_download_url
Hashicorp Vault download URL (default “https://releases.hashicorp.com/vault/1.0.3/vault_1.0.3_linux_amd64.zip”)--vault_port
Vault port (default 8200)--working_directory
Working directory where to place deployment files (default, work)--yorc_ca_key_file
Path to Certificate Authority private key, accessible locally--yorc_ca_passphrase
Bootstrapped Yorc Home directory (default, /var/yorc)--yorc_ca_pem_file
Path to PEM-encoded Certificate Authority, accessible locally--yorc_data_dir
Bootstrapped Yorc Home directory (default, /var/yorc)--yorc_download_url
Yorc download URL (default, current Yorc release under https://github.com/ystia/yorc/releases/)--yorc_plugin_download_url
Yorc plugin download URL--yorc_port
Yorc HTTP REST API port (default 8800)--yorc_private_key_file
Path to ssh private key accessible locally--yorc_workers_number
Number of Yorc workers handling bootstrap deployment tasks (default 30)
The option --resources_zip
is an advanced usage option allowing you to change
the bootstrap deployment description. You need to clone first the Yorc source code repository at
https://github.com/ystia/yorc, go into to directory commands
, change deployment
description files under bootstrap/resources/topology
, then zip the content of bootstrap/resources/
so that this zip will be used to perform the bootstrap deployment.
Bootstrapping the setup using environment variables¶
Similarly to the configuration of yorc server
through environment variables
described at Yorc Server Configuration, the bootstrap configuration can be provided
through environment variables following the same naming rules, for example:
YORC_ALIEN4CLOUD_PORT
allows to define the Alien4Cloud port
Once these environment variables are defined, you can bootstrap the setup running : .. parsed-literal:
./yorc bootstrap [--review]
Bootstrapping the setup using a configuration file¶
You can bootstrap the setup using a configuration file running:
./yorc bootstrap --values <path to configuration file> [--review]
Similarly to the configuration of yorc server
through a configuration file,
described at Yorc Server Configuration, the bootstrap configuration can be provided
in a configuration file following the same naming rules for configuration variables,
for example :
alien4cloud:
user: admin
port: 8088
locations:
- name: myLocation
type: openstack
properties:
auth_url: http://10.197.135.201:5000/v2.0
The bootstrap configuration file can be also be used to define Ansible Inventory configuration parameters. This is needed for example if remote hosts have python3 installed by default and not python, like on Ubuntu 18+.
In this case, you can add in the bootstrap configuration file, a section allowing to configure an Ansible behavioral inventory parameter that will allow to specify which python interpreter could be used by Ansible on remote hosts, as described in Ansible Inventory Configuration section.
This would give for example in the bootstrap configuration file:
ansible:
inventory:
"target_hosts:vars":
- ansible_python_interpreter=/usr/bin/python3
See later below a full example of bootstrap configuration file defining such a parameter.
Sections below provide examples of configuration files define a location for each infrastructure type.
Example of a Google Cloud deployment configuration file¶
yorc:
# Path to private key file on local host
# used to connect to hosts on the bootstrapped setup
private_key_file: /home/myuser/.ssh/yorc.pem
# Path to Certificate Authority private key, accessible locally
# If no key ile provided, one will be generated
ca_key_file: /home/myuser//ca-key.pem
# Certificate authority private key passphrase
ca_passphrase: changeme
# Path to PEM-encoded Certificate Authority, accessible locally
# If not provided, a Certifcate Authority will be generated
ca_pem_file: /home/myuser/ca.pem
locations:
- name: firstGoogleLocation
type: google
properties:
# Path on local host to file containing Google service account private keys
application_credentials: /home/myuser/gcp/myproject-a90a&bf599ef.json
project: myproject
address:
region: europe-west1
compute:
image_project: centos-cloud
image_family: centos-7
machine_type: n1-standard-2
zone: europe-west1-b
# User and public key to define on created compute instance
metadata: "ssh-keys=user1:ssh-ed25519 AAAABCd/gV/C+b3h3r5K011evEELMD72S4..."
tags: a4c
credentials:
# User on compute instance created on demand
user: user1
Example of a Google Cloud deployment configuration with Ubuntu 19.04 on-demand compute¶
In this example, on-demand compute instances run Ubuntu 19.04 on which python3
is installed by default (and not python).
In this case, a specific Ansible behavioral inventory parameter
ansible_python_interpreter
must be defined so that Ansible is able to find
this python interpreter on the remote hosts.
yorc:
# Path to private key file on local host
# used to connect to hosts on the bootstrapped setup
private_key_file: /home/myuser/.ssh/yorc.pem
# Path to Certificate Authority private key, accessible locally
# If no key ile provided, one will be generated
ca_key_file: /home/myuser//ca-key.pem
# Certificate authority private key passphrase
ca_passphrase: changeme
# Path to PEM-encoded Certificate Authority, accessible locally
# If not provided, a Certifcate Authority will be generated
ca_pem_file: /home/myuser/ca.pem
locations:
- name: firstGoogleLocation
type: google
properties:
# Path on local host to file containing Google service account private keys
application_credentials: /home/myuser/gcp/myproject-a90a&bf599ef.json
project: myproject
ansible:
inventory:
# Remote host run Ubuntu 19.04, using python3.
# Defining here the Ansible behavioral inventory parameter ansible_python_interpreter
# pointing to python3.
# This is required or Ansible will attempt to use python on the remote host
# which will fail as python is not installed by default.
"target_hosts:vars":
- ansible_python_interpreter=/usr/bin/python3
address:
region: europe-west1
compute:
image_project: ubuntu-os-cloud
image_family: ubuntu-1904
machine_type: n1-standard-2
zone: europe-west1-b
# User and public key to define on created compute instance
metadata: "ssh-keys=user1:ssh-ed25519 AAAABCd/gV/C+b3h3r5K011evEELMD72S4..."
tags: a4c
credentials:
# User on compute instance created on demand
user: user1
Example of an AWS deployment configuration file¶
yorc:
# Path to private key file on local host
# used to connect to hosts on the bootstrapped setup
private_key_file: /home/myuser/.ssh/yorc.pem
# Path to Certificate Authority private key, accessible locally
# If no key ile provided, one will be generated
ca_key_file: /home/myuser//ca-key.pem
# Certificate authority private key passphrase
ca_passphrase: changeme
# Path to PEM-encoded Certificate Authority, accessible locally
# If not provided, a Certifcate Authority will be generated
ca_pem_file: /home/myuser/ca.pem
locations:
- name: firstAWSLocation
type: aws
properties:
region: us-east-2
access_key: ABCDEFABCDEFABCD12DA
secret_key: aabcdxYxABC/a1bcdef
address:
ip_version: 4
compute:
image_id: ami-18f8df7d
instance_type: t2.large
key_name: key-yorc
security_groups: janus-securityGroup
delete_volume_on_termination: true
credentials:
# User on compute instance created on demand
user: user1
Example of an OpenStack deployment configuration file¶
yorc:
# Path to private key file on local host
# used to connect to hosts on the bootstrapped setup
private_key_file: /home/myuser/.ssh/yorc.pem
# Path to Certificate Authority private key, accessible locally
# If no key ile provided, one will be generated
ca_key_file: /home/myuser//ca-key.pem
# Certificate authority private key passphrase
ca_passphrase: changeme
# Path to PEM-encoded Certificate Authority, accessible locally
# If not provided, a Certificate Authority will be generated
ca_pem_file: /home/myuser/ca.pem
locations:
- name: firstOpenStackLocation
type: openstack
properties:
auth_url: http://10.1.2.3:5000/v2.0
default_security_groups:
- secgroup1
- secgroup2
password: mypasswd
private_network_name: private-test
region: RegionOne
tenant_name: mytenant
user_name: myuser
address:
floating_network_name: mypublic-net
compute:
image: "7d9bd308-d9c1-4952-123-95b761672499"
flavor: 3
key_pair: yorc
credentials:
# User on compute instance created on demand
user: user1
Example of a Hosts Pool deployment configuration file¶
yorc:
# Path to private key file on local host
# used to connect to hosts on the bootstrapped setup
private_key_file: /home/myuser/.ssh/yorc.pem
# Path to Certificate Authority private key, accessible locally
# If no key ile provided, one will be generated
ca_key_file: /home/myuser//ca-key.pem
# Certificate authority private key passphrase
ca_passphrase: changeme
# Path to PEM-encoded Certificate Authority, accessible locally
# If not provided, a Certificate Authority will be generated
ca_pem_file: /home/myuser/ca.pem
compute:
shareable: "false"
hosts:
- name: host1
connection:
user: user1
host: 10.129.1.10
port: 22
labels:
host.cpu_frequency: 3 GHz
host.disk_size: 40 GB
host.mem_size: 4GB
host.num_cpus: "2"
os.architecture: x86_64
os.distribution: centos
os.type: linux
os.version: "7.3.1611"
private_address: "10.0.0.10"
public_address: "10.129.1.10"
- name: host2
connection:
user: user1
host: 10.129.1.11
port: 22
labels:
environment: dev
host.cpu_frequency: 3 GHz
host.disk_size: 40 GB
host.mem_size: 4GB
host.num_cpus: "2"
os.architecture: x86_64
os.distribution: centos
os.type: linux
os.version: "7.3.1611"
private_address: "10.0.0.11"
public_address: "10.129.1.11"
Exporting and loading an interactive configuration file¶
When deploying, the final configuration of the bootstrapping is automatically exported to a file. The name of the file is the deployment id, which is a timestamp of current year to second. You can create a custom deployment id using ‘’-n’’ option :
./yorc bootstrap -n a_deploy_name
If you specify an already existing name (an input config file of the same name this already exists), an unique name will be created, of the form ‘’nameN’‘, where N is an integer, generated incrementally.
You can then load a config file using the “-v” option :
./yorc bootstrap -v path_to_a_file_containing_input_values
Please note than if a config is loaded using this option, it will not be exported again.
If you wish to only export the interactive configuration without doing an actual bootstrap, just set the ‘’–config_only’’ flag:
./yorc bootstrap --config_only
it will cause the yorc invocation to terminate straight after the export of interactive config.
Troubleshooting¶
By default, debug logs are disabled. To enable them, you can export the environment
variable YORC_LOG and set it to 1
or DEBUG
before starting the bootstrap:
export YORC_LOG=1
Once the bootstrap deployment has started, the local yorc server logs are available
under <working dir>/yorc.log
, (<working dir> default value being the directory ./work
).
To get the bootstrap deployment ID and current status, run :
./yorc deployments list
To follow deployment logs and see these logs from the beginning, run :
./yorc deployments logs <deployment ID> --from-beginning
When a deployment has failed, in addition to logs failure in the logs, you can also get of summary of the deployment steps statuses to identify quickly which step failed, running :
./yorc deployments info <deployment ID>
If a step failed on a transient error that is now addressed, it is possible to run again manually the failed step, and resume the deployment running the following commands.
First from the previous command ./yorc deployments info <deployment ID>
output,
you can find the task ID that failed.
You can now run this command to get the exact name of the step that failed :
./yorc deployments tasks info --steps <deployment ID> <task ID>
Identify the name of the step that failed.
Let’s say for the example that it is the step TerraformRuntime_create
which failed
on timeout downloading the Terraform distribution.
You can then go to the directory where you will find the ansible playbook corresponding to this step :
cd <working directory>/deployments/<deployment ID>/ansible/<task ID>/TerraformRuntime/standard.create/
And from this directory, run again this step through this command:
ansible-playbook -i hosts run.ansible.yml -v
If this manual execution was successful, you can mark the corresponding step as fixed in the deployment, running :
./yorc deployments tasks fix <deployment ID> <task ID> TerraformRuntime
You can now resume the bootstrap deployment running :
./yorc deployments tasks resume <deployment ID>