Bootstrap a full stack installation

The command yorc bootstrap can be used to bootstrap the full stack, from Alien4Cloud to Yorc and its dependencies, over different types of infrastructures, on a single node or distributed on several nodes.

Prerequisites

Hosts

The local host from where the command yorc bootstrap will be run, as well as remote hosts where the full stack will be deployed, should be Linux x86_64 systems operating with at least 2 CPUs and 4 Go of RAM. The bootstrap was validated on CentOS 7 and Red Hat Enterprise Linux 7.5.

Packages

The bootstrap operation will install a basic Yorc setup on the local host.

It requires the following packages to be installed on the local host:

  • python
  • python-pip
  • zip/unzip
  • openssh-client
  • wget

This basic installation on the local host will attempt to install without sudo privileges python ansible module 2.7.2 if needed as well as python packages MarkupSafe, jinja2, PyYAML, six, cryptography, setuptools. So if this ansible module or python packages are not yet installed on the local host, you could either add them yourself, with sudo privileges, running for example:

sudo pip install ansible==2.7.2

Or you could create a python virtual environment, and let the Yorc bootstrap command install the ansible module within this virtual environment (operation which doesn’t require sudo privileges).

You can run these commands to create a virtual environment, here a virtual environment called yorcenv :

sudo pip install virtualenv
virtualenv yorcenv
source yorcenv/bin/activate

You are now ready to download Yorc binary, running:

wget https://github.com/ystia/yorc/releases/download/v3.1.2/yorc-3.1.2.tgz
tar xzf yorc-3.1.2.tgz
./yorc bootstrap --help

Define firewall rules on your Cloud Provider infrastructure

To access Alien4Cloud UI from the local host, you may need to define firewall rules before attempting to bootstrap the full stack on a cloud provider infrastructure.

For example on Google Cloud, you could define this firewall rule for the port 8088 used by the UI, and associate it to a tag (here a4c) :

$ gcloud compute firewall-rules create a4c-rule \
  --allow tcp:8088 --target-tags a4c

You could then specify this tag a4c in the compute instance to create by the bootstrap deployment, as it is done in Google Configuration file example. This way the created compute instance where Alien4Cloud will be deployed will have its port 8088 open.

Bootstrap process overview

The command yorc bootstrap will configure on the local host a basic setup with Yorc and a Consul data store.

This basic setup will then be used to bootstrap the full stack Alien4Cloud/Yorc and its dependencies over a selected infrastructure.

You can deploy the full stack either on a single node (by default), or distributed on several nodes as described in Run Yorc in HA mode, using yorc bootstrap command line option --deployment_type HA described below.

Configuration values can be provided by the user:

  • in interactive mode,
  • through a configuration file,
  • using yorc bootstrap command line options
  • using environment variables.

You can combine these modes, yorc bootstrap will check in any case if required configuration values are missing, and will ask for missing values.

These configuration values will allow you to specify:

  • optional Alien4Cloud configuration values
  • Yorc configuration values, all optional, except from the path to a ssh private key that will be used by the local orchestrator to connect to the bootstrapped setup
  • Infrastructure configuration with required configuration values depending on the infrastucture, as described at Infrastructures Configuration
  • Configuration of compute Nodes to create on demand,
  • User used to connect to these compute nodes,
  • Configuration of the connection to public network created on demand.

Details of these on-demand resources configuration values are provided in the Alien4Cloud Yorc plugin Documentation at https://yorc-a4c-plugin.readthedocs.io/en/latest/location.html. For example, in the Google Configuration file example, you can see on-demand compute and address configuration values.

Once configuration settings are provided, yorc bootstrap will proceed to the full stack deployment, showing deployment steps progress (by default, but you can see deployment logs instead trough the option --follow logs described below).

Once the deployment is finished, the orchestrator on the local host is still running, so you can perform commands like ./yorc deployments list, ./yorc deployments logs -b, etc… Or perform any deployment troubleshooting if needed.

To undeploy a bootstrapped setup, you can also use the CLI, running ./yorc deployments undeploy <deployment id>.

To clean the local host setup, run:

./yorc bootstrap cleanup

This will only clean the local host environment, it won’t undeploy the bootstrapped setup installed on remote hosts.

Bootstrapping the setup in interactive mode

You can bootstrap the setup in interactive mode running:

./yorc bootstrap [--review]

You will have then to select the type of infrastructure (Google Cloud, AWS, OpenStack, Hosts Pool) on which you want to deploy the full stack, then you will be asked to provide configuration values depending on the selected infrastructure.

The command line option --review allows to review and update all configuration values before proceeding to the deployment, opening the editor specified in the environment variable EDITOR if defined or using vi or vim if available.

Bootstrapping the setup using command line options

The following yorc bootstrap option are available:

  • --alien4cloud_download_url Alien4Cloud download URL (defaults to the Alien4Cloud version compatible with this Yorc, under https://fastconnect.org/maven/content/repositories/opensource/alien4cloud/alien4cloud-dist/)
  • --alien4cloud_password Alien4Cloud password (default, admin)
  • --alien4cloud_port Alien4Cloud port (default 8088)
  • --alien4cloud_user Alien4Cloud user (default, admin)
  • --ansible_extra_package_repository_url URL of package indexes where to find the ansible package, instead of the default Python Package repository
  • --ansible_version Ansible version (default 2.7.2)
  • --consul_download_url Consul download URL (default, Consul version compatible with this Yorc, under https://releases.hashicorp.com/consul/)
  • --consul_port Consul port (default 8500)
  • --credentials_user User Yorc uses to connect to Compute Nodes
  • --deployment_type Define deployment type: single_node or HA (default, single_node)
  • --follow Follow bootstrap deployment steps, logs, or none (default, steps)
  • --infrastructure Define the type of infrastructure where to deploy Yorc: google, openstack, aws, hostspool
  • --jdk_download_url Java Development Kit download URL (default, JDK downloaded from https://edelivery.oracle.com/otn-pub/java/jdk/)
  • --jdk_version Java Development Kit version (default 1.8.0-131-b11)
  • --resources_zip Path to bootstrap resources zip file (default, zip bundled within Yorc)
  • --review Review and update input values before starting the bootstrap
  • --terraform_download_url Terraform download URL (default, Terraform version compatible with this Yorc, under https://releases.hashicorp.com/terraform/)
  • --terraform_plugins_download_urls Terraform plugins download URLs (default, Terraform plugins compatible with this Yorc, under https://releases.hashicorp.com/terraform-provider-xxx/)
  • --values Path to file containing input values
  • --working_directory Working directory where to place deployment files (default, work)
  • --yorc_data_dir Bootstrapped Yorc Home directory (default, /var/yorc)
  • --yorc_download_url Yorc download URL (default, current Yorc release under https://github.com/ystia/yorc/releases/)
  • --yorc_plugin_download_url Yorc plugin download URL (default, current Yorc plugin release under https://github.com/ystia/yorc-a4c-plugin/releases)
  • --yorc_port Yorc HTTP REST API port (default 8800)
  • --yorc_private_key_file Path to ssh private key accessible locally
  • --yorc_workers_number Number of Yorc workers handling bootstrap deployment tasks (default 30)

In addition, similarly to the configuration of infrastructures in yorc server command described at Infrastructures Configuration, you can use options to define infrastructure and on-demand resources configuration values, for example :

  • --infrastructure_openstack_auth_url allows to define the authentication URL of an OpenStack infrastructure.

The option --resources_zip is an advanced usage option allowing you to change the bootstrap deployment description. You need to clone first the Yorc source code repository at https://github.com/ystia/yorc, go into to directory commands, change deployment description files under bootstrap/resources/topology, then zip the content of bootstrap/resources/ so that this zip will be used to perform the bootstrap deployment.

Bootstrapping the setup using environment variables

Similarly to the configuration of yorc server through environment variables described at Yorc Server Configuration, the bootstrap configuration can be provided through environment variables following the same naming rules, for example:

  • YORC_ALIEN4CLOUD_PORT allows to define the Alien4Cloud port
  • YORC_INFRA_OPENSTACK_AUTH_URL allows to define the authentication URL of an OpenStack infrastructure.

Once these environment variables are defined, you can bootstrap the setup running : .. parsed-literal:

./yorc bootstrap [--review]

Bootstrapping the setup using a configuration file

You can bootstrap the setup using a configuration file running:

./yorc bootstrap --values <path to configuration file> [--review]

Similarly to the configuration of yorc server through a configuration file, described at Yorc Server Configuration, the bootstrap configuration can be provided in a configuration file following the same naming rules for configuration variables, for example :

alien4cloud:
  user: admin
  port: 8088
infrastructures:
  openstack:
    auth_url: http://10.1.2.3:5000/v2.0

Sections below provide examples of configuration files for each type of infrastructure.

Example of a Google Cloud deployment configuration file

yorc:
  # Path to private key file on local host
  # used to connect to hosts on the bootstrapped setup
  private_key_file: /home/myuser/.ssh/yorc.pem
infrastructures:
  google:
    # Path on local host to file containing Google service account private keys
    application_credentials: /home/myuser/gcp/myproject-a90a&bf599ef.json
    project: myproject
address:
  region: europe-west1
compute:
  image_project: centos-cloud
  image_family: centos-7
  machine_type: n1-standard-2
  zone: europe-west1-b
  # User and public key to define on created compute instance
  metadata: "ssh-keys=user1:ssh-ed25519 AAAABCd/gV/C+b3h3r5K011evEELMD72S4..."
  tags: a4c
credentials:
  # User on compute instance created on demand
  user: user1

Example of an AWS deployment configuration file

yorc:
  # Path to private key file on local host
  # used to connect to hosts on the bootstrapped setup
  private_key_file: /home/myuser/.ssh/yorc.pem
infrastructures:
  aws:
    region: us-east-2
    access_key: ABCDEFABCDEFABCD12DA
    secret_key: aabcdxYxABC/a1bcdef
address:
  ip_version: 4
compute:
  image_id: ami-18f8df7d
  instance_type: t2.large
  key_name: key-yorc
  security_groups: janus-securityGroup
  delete_volume_on_termination: true
credentials:
  # User on compute instance created on demand
  user: user1

Example of an OpenStack deployment configuration file

yorc:
  # Path to private key file on local host
  # used to connect to hosts on the bootstrapped setup
  private_key_file: /home/myuser/.ssh/yorc.pem
infrastructures:
  openstack:
    auth_url: http://10.1.2.3:5000/v2.0
    default_security_groups:
    - secgroup1
    - secgroup2
    password: mypasswd
    private_network_name: private-test
    region: RegionOne
    tenant_name: mytenant
    user_name: myuser
address:
  floating_network_name: mypublic-net
compute:
  image: "7d9bd308-d9c1-4952-123-95b761672499"
  flavor: 3
  key_pair: yorc
credentials:
  # User on compute instance created on demand
  user: user1

Example of a Hosts Pool deployment configuration file

yorc:
  # Path to private key file on local host
  # used to connect to hosts on the bootstrapped setup
  private_key_file: /home/myuser/.ssh/yorc.pem
compute:
  shareable: "false"
hosts:
- name: host1
  connection:
    user: user1
    host: 10.129.1.10
    port: 22
  labels:
    host.cpu_frequency: 3 GHz
    host.disk_size: 40 GB
    host.mem_size: 4GB
    host.num_cpus: "2"
    os.architecture: x86_64
    os.distribution: centos
    os.type: linux
    os.version: "7.3.1611"
    private_address: "10.0.0.10"
    public_address: "10.129.1.10"
- name: host2
  connection:
    user: user1
    host: 10.129.1.11
    port: 22
  labels:
    environment: dev
    host.cpu_frequency: 3 GHz
    host.disk_size: 40 GB
    host.mem_size: 4GB
    host.num_cpus: "2"
    os.architecture: x86_64
    os.distribution: centos
    os.type: linux
    os.version: "7.3.1611"
    private_address: "10.0.0.11"
    public_address: "10.129.1.11"

Troubleshooting

By default, debug logs are disabled. To enable them, you can export the environment variable YORC_LOG and set it to 1 or DEBUG before starting the bootstrap:

export YORC_LOG=1

Once the bootstrap deployment has started, the local yorc server logs are available under <working dir>/yorc.log, (<working dir> default value being the directory ./work).

To get the bootstrap deployment ID and current status, run :

./yorc deployments list

To follow deployment logs and see these logs from the beginning, run :

./yorc deployments logs <deployment ID> --from-beginning

When a deployment has failed, in addition to logs failure in the logs, you can also get of summary of the deployment steps statuses to identify quickly which step failed, running :

./yorc deployments info <deployment ID>

If a step failed on a transient error that is now addressed, it is possible to run again manually the failed step, and resume the deployment running the following commands.

First from the previous command ./yorc deployments info <deployment ID> output, you can find the task ID that failed.

You can now run this command to get the exact name of the step that failed :

./yorc deployments tasks info --steps <deployment ID> <task ID>

Identify the name of the step that failed.

Let’s say for the example that it is the step TerraformRuntime_create which failed on timeout downloading the Terraform distribution.

You can then go to the directory where you will find the ansible playbook corresponding to this step :

cd <working directory>/deployments/<deployment ID>/ansible/<task ID>/TerraformRuntime/standard.create/

And from this directory, run again this step through this command:

ansible-playbook -i hosts run.ansible.yml -v

If this manual execution was successful, you can mark the corresponding step as fixed in the deployment, running :

./yorc deployments tasks fix <deployment ID> <task ID> TerraformRuntime

You can now resume the bootstrap deployment running :

./yorc deployments tasks resume <deployment ID>