Yorc Server Configuration¶
Yorc has various configuration options that could be specified either by command-line flags, configuration file or environment variables.
If an option is specified several times using flags, environment and config file, command-line flag will have the precedence then the environment variable and finally the value defined in the configuration file.
Globals Command-line options¶
--ansible_use_openssh
: Prefer OpenSSH over Paramiko a Python implementation of SSH (the default) to provision remote hosts. OpenSSH have several optimization like reusing connections that should improve preformance but may lead to issues on older systems.
--ansible_debug
: Prints massive debug information from Ansible especially about connections
--ansible_connection_retries
: Number of retries in case of Ansible SSH connection failure.
--ansible_cache_facts
: If set to true, caches Ansible facts (values fetched on remote hosts about network/hardware/OS/virtualization configuration) so that these facts are not recomputed each time a new operation is a run for a given deployment (false by default: no caching).
--ansible_archive_artifacts
: If set to true, archives operation bash/python scripts locally, copies this archive and unarchives it on remote hosts (requires tar to be installed on remote hosts), to avoid multiple time consuming remote copy operations of individual scripts (false by default: no archive).
--ansible_job_monitoring_time_interval
: Default duration for monitoring time interval for jobs handled by Ansible (defaults to 15s).
--ansible_keep_generated_recipes
: If set to true, generated Ansible recipes on Yorc server are not deleted. (false by default: generated recipes are deleted).
--operation_remote_base_dir
: Specify an alternative working directory for Ansible on provisioned Compute.
--config
or-c
: Specify an alternative configuration file. By default Yorc will look for a file named config.yorc.json in/etc/yorc
directory then if not found in the current directory.
--consul_address
: Specify the address (using the format host:port) of Consul. Consul default is used if not provided.
--consul_token
: Specify the security token to use with Consul. No security token used by default.
--consul_datacenter
: Specify the Consul’s datacenter to use. Consul default (dc1) is used by default.
--consul_key_file
: Specify the Consul client’s key to use when commuicating over TLS.
--consul_cert_file
: Specify the Consul client’s certificate to use when commuicating over TLS.
--consul_ca_cert
: Specify the CA used to sign Consul certificates.
--consul_ca_path
: Specify the path to the CA used to sign Consul certificates
--consul_ssl
: If set to true, enable SSL (false by default).
--consul_ssl_verify
: If set to false, disable Consul certificate checking (true by default is ssl enabled).
--consul_tls_handshake_timeout
: Maximum duration to wait for a TLS handshake with Consul, the default is50s
.
--terraform_plugins_dir
: Specify the directory where to find Terraform pre-installed providers plugins. If not specified, required plugins will be downloaded during deployment. See https://www.terraform.io/guides/running-terraform-in-automation.html#pre-installed-plugins for more information.
--terraform_aws_plugin_version_constraint
: Specify the Terraform AWS plugin version constraint. Default one compatible with our source code is"~> 1.36"
. If you choose another, it’s at your own risk. See https://www.terraform.io/docs/configuration/providers.html#provider-versions for more information.
--terraform_consul_plugin_version_constraint
: Specify the Terraform Consul plugin version constraint. Default one compatible with our source code is"~> 2.1"
. If you choose another, it’s at your own risk. See https://www.terraform.io/docs/configuration/providers.html#provider-versions for more information.
--terraform_google_plugin_version_constraint
: Specify the Terraform Google plugin version constraint. Default one compatible with our source code is"~> 1.18"
. If you choose another, it’s at your own risk. See https://www.terraform.io/docs/configuration/providers.html#provider-versions for more information.
--terraform_openstack_plugin_version_constraint
: Specify the Terraform OpenStack plugin version constraint. Default one compatible with our source code is"~> 1.9"
. If you choose another, it’s at your own risk. See https://www.terraform.io/docs/configuration/providers.html#provider-versions for more information.
--terraform_keep_generated_files
: If set to true, generated Terraform infrastructures files on Yorc server are not deleted. (false by default: generated files are deleted).
--consul_publisher_max_routines
: Maximum number of parallelism used to store key/values in Consul. If you increase the default value you may need to tweak the ulimit max open files. If set to 0 or less the default value (500) will be used.
--graceful_shutdown_timeout
: Timeout to wait for a graceful shutdown of the Yorc server. After this delay the server immediately exits. The default is5m
.
--wf_step_graceful_termination_timeout
: Timeout to wait for a graceful termination of a workflow step during concurrent workflow step failure. After this delay the step is set on error. The default is2m
.
--purged_deployments_eviction_timeout
: When a deployment is purged an event is kept to let a chance to external systems to detect it via the events API, this timeout controls the retention time of such events. The default is30m
.
--http_address
: Restrict the listening interface for the Yorc HTTP REST API. By default Yorc listens on all available interfaces
--http_port
: Port number for the Yorc HTTP REST API. If omitted or set to ‘0’ then the default port number is used, any positive integer will be used as it, and finally any negative value will let use a random port.
--keep_operation_remote_path
: If set to true, do not delete temporary artifacts on provisioned Compute at the end of deployment (false by default for deployment temporary artifacts cleanup).
--key_file
: File path to a PEM-encoded private key. The key is used to enable SSL for the Yorc HTTP REST API. This must be provided along with cert_file. If one of key_file or cert_file is not provided then SSL is disabled.
--cert_file
: File path to a PEM-encoded certificate. The certificate is used to enable SSL for the Yorc HTTP REST API. This must be provided along with key_file. If one of key_file or cert_file is not provided then SSL is disabled.
--ca_file
: If set to true, enable TLS certificate checking. Must be provided with cert_file ; key_file and ca_file. Disabled by default.
--ssl_verify
: If set to true, enable TLS certificate checking for clients of the Yorc’s API. Must be provided with cert_file ; key_file and ca_file. Disabled by default.
--plugins_directory
: The name of the plugins directory of the Yorc server. The default is to use a directory named plugins in the current directory.
--locations_file_path
: File path to locations configuration. This configuration is taken in account for the first time the server starts.
--resources_prefix
: Specify a prefix that will be used for names when creating resources such as Compute instances or volumes. Defaults toyorc-
.
--tasks_dispatcher_long_polling_wait_time
: Wait time (Golang duration format) when long polling for executions tasks to dispatch to workers. If not set the default value of 1m will be used.
--tasks_dispatcher_lock_wait_time
: Wait time (Golang duration format) for acquiring a lock for an execution task. If not set the default value of 50ms will be used.
--tasks_dispatcher_metrics_refresh_time
: Refresh time (Golang duration format) for the tasks dispatcher metrics. If not set the default value of 5m will be used.
--workers_number
: Yorc instances use a pool of workers to handle deployment tasks. This option defines the size of this pool. If not set the default value of 30 will be used.
--working_directory
or-w
: Specify an alternative working directory for Yorc. The default is to use a directory named work in the current directory.
--server_id
: Specify the server ID used to identify the server node in a cluster. The default is the hostname.
--disable_ssh_agent
: Allow disabling ssh-agent use for SSH authentication on provisioned computes. Default is false. If true, compute credentials must provide a path to a private key file instead of key content.
--concurrency_limit_for_upgrades
: Limit of concurrency used in Upgrade processes. If not set the default value of 1000 will be used.
--ssh_connection_timeout
: Timeout to establish SSH connection from Yorc SSH client, especially used for Slurm and HostsPool locations. If not set the default value of 10 s will be used.
--ssh_connection_retry_backoff
: Backoff duration before retrying an ssh connection. This may be superseded by a location attribute if supported. (default 1s)
--ssh_connection_max_retries
: Maximum number of retries (attempts are retries + 1) before giving-up to connect. This may be superseded by a location attribute if supported. (default 3)
Configuration files¶
Configuration files are either JSON or YAML formatted as a single object containing the following configuration options.
By default Yorc will look for a file named config.yorc.json in /etc/yorc
directory then if not found in the current directory.
The –config command line flag allows to specify an alternative configuration file.
Below is an example of configuration file.
{
"resources_prefix": "yorc1-",
"locations_file_path": "path-to-locations-yaml-or-json-config"
}
Below is an example of configuration file with TLS enabled.
{
"resources_prefix": "yorc1-",
"key_file": "/etc/pki/tls/private/yorc.key",
"cert_file": "/etc/pki/tls/certs/yorc.crt",
"locations_file_path": "path-to-locations-yaml-or-json-config"
}
server_graceful_shutdown_timeout
: Equivalent to –graceful_shutdown_timeout command-line flag.
wf_step_graceful_termination_timeout
: Equivalent to –wf_step_graceful_termination_timeout command-line flag.
purged_deployments_eviction_timeout
: Equivalent to –purged_deployments_eviction_timeout command-line flag.
http_address
: Equivalent to –http_address command-line flag.
http_port
: Equivalent to –http_port command-line flag.
key_file
: Equivalent to –key_file command-line flag.
cert_file
: Equivalent to –cert_file command-line flag.
ssl_verify
: Equivalent to –ssl_verify command-line flag.
ca_file
: Equivalent to –ca_file command-line flag.
plugins_directory
: Equivalent to –plugins_directory command-line flag.
resources_prefix
: Equivalent to –resources_prefix command-line flag.
locations_file_path
: Equivalent to –locations_file_path command-line flag.
workers_number
: Equivalent to –workers_number command-line flag.
working_directory
: Equivalent to –working_directory command-line flag.
server_id
: Equivalent to –server_id command-line flag.
disable_ssh_agent
: Equivalent to –disable_ssh_agent command-line flag.
concurrency_limit_for_upgrades
: Equivalent to –concurrency_limit_for_upgrades command-line flag.
ssh_connection_timeout
: Equivalent to –ssh_connection_timeout command-line flag.
ssh_ssh_connection_retry_backoff
: Equivalent to –ssh_ssh_connection_retry_backoff command-line flag.
ssh_connection_max_retries
: Equivalent to –ssh_connection_max_retries command-line flag.
Ansible configuration¶
Below is an example of configuration file with Ansible configuration options.
{
"resources_prefix": "yorc1-",
"locations_file_path": "path-to-locations-yaml-or-json-config",
"ansible": {
"use_openssh": true,
"connection_retries": 3,
"hosted_operations": {
"unsandboxed_operations_allowed": false,
"default_sandbox": {
"image": "jfloff/alpine-python:2.7-slim",
"entrypoint": ["python", "-c"],
"command": ["import time;time.sleep(31536000);"]
}
},
"config": {
"defaults": {
"display_skipped_hosts": "False",
"special_context_filesystems": "nfs,vboxsf,fuse,ramfs,myspecialfs",
"timeout": "60"
}
},
"inventory":{
"target_hosts:vars": ["ansible_python_interpreter=/usr/bin/python3"]
}
}
}
All available configuration options for Ansible are:
use_openssh
: Equivalent to –ansible_use_openssh command-line flag.
debug
: Equivalent to –ansible_debug command-line flag.
connection_retries
: Equivalent to –ansible_connection_retries command-line flag.
cache_facts
: Equivalent to –ansible_cache_facts command-line flag.
archive_artifacts
: Equivalent to –ansible_archive_artifacts command-line flag.
job_monitoring_time_interval
: Equivalent to –ansible_job_monitoring_time_interval command-line flag.
operation_remote_base_dir
: Equivalent to –operation_remote_base_dir command-line flag.
keep_operation_remote_path
: Equivalent to –keep_operation_remote_path command-line flag.
keep_generated_recipes
: Equivalent to –ansible_keep_generated_recipes command-line flag.
hosted_operations
: This is a complex structure that allow to define the behavior of a Yorc server when it executes an hosted operation. For more information about hosted operation please see The hosted operations paragraph in the TOSCA support section. This structure contains the following configuration options:
unsandboxed_operations_allowed
: This option control if operations can be executed directly on the system that hosts Yorc if no default sandbox is defined. This is not permitted by default.
default_sandbox
: This complex structure allows to define the default docker container to use to sandbox orchestrator-hosted operations. Bellow configuration optionsentrypoint
andcommand
should be carefully set to run the container and make it sleep until operations are executed on it. Defaults options will run a python inline script that sleeps for 1 year.
image
: This is the docker image identifier (in the docker format[repository/]name[:tag]
) is option is required.
entrypoint
: This allows to override the default image entrypoint. If bothentrypoint
andcommand
are empty the default value forentrypoint
is["python", "-c"]
.
command
: This allows to run a command within the container. If bothentrypoint
andcommand
are empty the default value forcommand
is["import time;time.sleep(31536000);"]
.
env
: An optional list environment variables to set when creating the container. The format of each variable isvar_name=value
.config
andinventory
are complex structure allowing to configure Ansible behavior, these options are described in more details in next section.
Ansible config option¶
config
is a complex structure allowing to define Ansible configuration settings if
you need a specific Ansible Configuration.
You should first provide the Ansible Configuration section (for example defaults
,
ssh_connection
…).
You should then provide the list of parameters within this section, ie. what
Ansible documentation describes
as the Ini key
within the Ini Section
.
Each parameter value must be provided here as a string : for a boolean parameter,
you would provide the string False
or True
as expected in Ansible Confiugration.
For example, it would give in Yaml:
ansible:
config:
defaults:
display_skipped_hosts: "False"
special_context_filesystems: "nfs,vboxsf,fuse,ramfs,myspecialfs"
timeout: "60"
By default, the Orchestrator will define these Ansible Configuration settings :
host_key_checking: "False"
, to avoid host key checking by the underlying tools Ansible uses to connect to the hosttimeout: "30"
, to set the connection timeout to 30 secondsstdout_callback: "yaml"
, to display ansible output in yaml formatnocows: "1"
, to disable cowsay messages that can cause parsing issues in the Orchestrator
And when ansible fact caching is enabled, the Orchestrator adds these settings :
gathering: "smart"
, to set Ansible fact gathering to smart: each new host that has no facts discovered will be scannedfact_caching: "jsonfile"
, to use a json file-based cache plugin
Warning
Be careful when overriding these settings defined by default by the Orchestrator, as it might lead to unpredictable results.
Ansible inventory option¶
inventory
is a structure allowing to configure Ansible inventory settings if
you need to define variables for hosts or groups.
You should first provide the Ansible Inventory group name. You should then provide the list of parameters to define for this group, which can be any parameter specific to your ansible playbooks, or behavioral inventory parameters describing how Ansible interacts with remote hosts.
For example, for Ansible to use a given python interpreter on target hosts, you must define
the Ansible behavioral inventory parameter ansible_python_interpreter
in the Ansible inventory Yorc configuration, like below in Yaml:
ansible:
inventory:
"target_hosts:vars":
- ansible_python_interpreter=/usr/bin/python3
By default, the Orchestrator will define :
- an inventory group
target_hosts
containing the list of remote hosts, and its associated variable grouptarget_hosts:vars
configuring by default these behavioral parameters:
ansible_ssh_common_args="-o ConnectionAttempts=20"
ansible_python_interpreter="auto_silent"
Warning
Settings defined by the user take precedence over settings defined by the Orchestrator. Be careful when overriding these settings defined by default by the Orchestrator, as it might lead to unpredictable results.
Ansible performance considerations¶
As described in TOSCA Supported Operations implementations, Yorc supports these builtin implementations for operations to execute on remote hosts :
- Bash scripts
- Python scripts
- Ansible Playbooks
It is recommended to implement operations as Ansible Playbooks to get the best execution performance.
When operations are not implemented using Ansible playbooks, see the Performance section on TOSCA Operations to improve the performance of scripts execution on remote hosts.
Consul configuration¶
Below is an example of configuration file with Consul configuration options.
{
"resources_prefix": "yorc1-",
"locations_file_path": "path-to-locations-yaml-or-json-config",
"consul": {
"address": "http://consul-host:8500",
"datacenter": "dc1",
"publisher_max_routines": 500
}
}
All available configuration options for Consul are:
address
: Equivalent to –consul_address command-line flag.
token
: Equivalent to –consul_token command-line flag.
datacenter
: Equivalent to –consul_datacenter command-line flag.
key_file
: Equivalent to –consul_key_file command-line flag.
cert_file
: Equivalent to –consul_cert_file command-line flag.
ca_cert
: Equivalent to –consul_ca_cert command-line flag.
ca_path
: Equivalent to –consul_ca_path command-line flag.
ssl
: Equivalent to –consul_ssl command-line flag.
ssl_verify
: Equivalent to –consul_ssl_verify command-line flag.
tls_handshake_timeout
: Equivalent to –consul_tls_handshake_timeout command-line flag.
publisher_max_routines
: Equivalent to –consul_publisher_max_routines command-line flag.
Terraform configuration¶
Below is an example of configuration file with Terraform configuration options.
{
"resources_prefix": "yorc1-",
"locations_file_path": "path-to-locations-yaml-or-json-config",
"terraform": {
"plugins_dir": "home/yorc/terraform_plugins_directory",
}
}
All available configuration options for Terraform are:
plugins_dir
: Equivalent to –terraform_plugins_dir command-line flag.
aws_plugin_version_constraint
: Equivalent to –terraform_aws_plugin_version_constraint command-line flag.
consul_plugin_version_constraint
: Equivalent to –terraform_consul_plugin_version_constraint command-line flag.
google_plugin_version_constraint
: Equivalent to –terraform_google_plugin_version_constraint command-line flag.
openstack_plugin_version_constraint
: Equivalent to –terraform_openstack_plugin_version_constraint command-line flag.
keep_generated_files
: Equivalent to –terraform_keep_generated_files command-line flag.
Telemetry configuration¶
Telemetry configuration can only be done via the configuration file. By default telemetry data are only stored in memory. See Yorc Telemetry for more information about telemetry.
Below is an example of configuration file with telemetry metrics forwarded to a Statsd
instance and with a Prometheus
HTTP endpoint exposed.
{
"resources_prefix": "yorc1-",
"locations_file_path": "path-to-locations-yaml-or-json-config",
"telemetry": {
"statsd_address": "127.0.0.1:8125",
"expose_prometheus_endpoint": true
}
}
All available configuration options for telemetry are:
service_name
: Metrics keys prefix, defaults toyorc
.
disable_hostname
: Specifies if gauge values should not be prefixed with the local hostname. Defaults tofalse
.
disable_go_runtime_metrics
: Specifies Go runtime metrics (goroutines, memory, …) should not be published. Defaults tofalse
.
statsd_address
: Specify the address (in form <address>:<port>) of a statsd server to forward metrics data to.
statsite_address
: Specify the address (in form <address>:<port>) of a statsite server to forward metrics data to.
expose_prometheus_endpoint
: Specify if an HTTP Prometheus endpoint should be exposed allowing Prometheus to scrape metrics.
Tasks/Workers configuration¶
Below is an example of configuration file with Tasks configuration options.
resources_prefix: "yorc1-"
tasks:
dispatcher:
long_polling_wait_time: "1m"
lock_wait_time: "50ms"
metrics_refresh_time: "5m"
long_polling_wait_time
: Equivalent to –tasks_dispatcher_long_polling_wait_time command-line flag.
lock_wait_time
: Equivalent to –tasks_dispatcher_lock_wait_time command-line flag.
metrics_refresh_time
: Equivalent to –tasks_dispatcher_metrics_refresh_time command-line flag.
Environment variables¶
YORC_ANSIBLE_USE_OPENSSH
: Equivalent to –ansible_use_openssh command-line flag.
YORC_ANSIBLE_DEBUG
: Equivalent to –ansible_debug command-line flag.
YORC_ANSIBLE_CONNECTION_RETRIES
: Equivalent to –ansible_connection_retries command-line flag.
YORC_ANSIBLE_CACHE_FACTS
: Equivalent to –ansible_cache_facts command-line flag.
YORC_ANSIBLE_JOB_MONITORING_TIME_INTERVAL
: Equivalent to –ansible_job_monitoring_time_interval command-line flag.
YORC_ANSIBLE_KEEP_GENERATED_RECIPES
: Equivalent to –ansible_keep_generated_recipes command-line flag.
YORC_OPERATION_REMOTE_BASE_DIR
: Equivalent to –operation_remote_base_dir command-line flag.
YORC_CONSUL_ADDRESS
: Equivalent to –consul_address command-line flag.
YORC_CONSUL_TOKEN
: Equivalent to –consul_token command-line flag.
YORC_CONSUL_DATACENTER
: Equivalent to –consul_datacenter command-line flag.
YORC_CONSUL_KEY_FILE
: Equivalent to –consul_key_file command-line flag.
YORC_CONSUL_CERT_FILE
: Equivalent to –consul_cert_file command-line flag.
YORC_CONSUL_CA_CERT
: Equivalent to –consul_ca_cert command-line flag.
YORC_CONSUL_CA_PATH
: Equivalent to –consul_ca_path command-line flag.
YORC_CONSUL_SSL
: Equivalent to –consul_ssl command-line flag.
YORC_CONSUL_SSL_VERIFY
: Equivalent to –consul_ssl_verify command-line flag.
YORC_CONSUL_TLS_HANDSHAKE_TIMEOUT
: Equivalent to –consul_tls_handshake_timeout command-line flag.
YORC_CONSUL_STORE_TXN_TIMEOUT
: Allows to activate the feature that packs ConsulStore operations into transactions. If set to a valid Go duration, operations are packed into transactions up to 64 ops. This timeout represent the time to wait for new operations before sending an incomplete (less than 64 ops) transaction to Consul.
YORC_CONSUL_PUBLISHER_MAX_ROUTINES
: Equivalent to –consul_publisher_max_routines command-line flag.
YORC_SERVER_GRACEFUL_SHUTDOWN_TIMEOUT
: Equivalent to –graceful_shutdown_timeout command-line flag.
YORC_WF_STEP_GRACEFUL_TERMINATION_TIMEOUT
: Equivalent to –wf_step_graceful_termination_timeout command-line flag.
YORC_PURGED_DEPLOYMENTS_EVICTION_TIMEOUT
: Equivalent to –purged_deployments_eviction_timeout command-line flag.
YORC_HTTP_ADDRESS
: Equivalent to –http_address command-line flag.
YORC_HTTP_PORT
: Equivalent to –http_port command-line flag.
YORC_KEEP_OPERATION_REMOTE_PATH
: Equivalent to –keep_operation_remote_path command-line flag.
YORC_KEY_FILE
: Equivalent to –key_file command-line flag.
YORC_CERT_FILE
: Equivalent to –cert_file command-line flag.
YORC_SSL_VERIFY
: Equivalent to –ssl_verify command-line flag.
YORC_CA_FILE
: Equivalent to –ca_file command-line flag.
YORC_PLUGINS_DIRECTORY
: Equivalent to –plugins_directory command-line flag.
YORC_RESOURCES_PREFIX
: Equivalent to –resources_prefix command-line flag.
YORC_LOCATIONS_FILE_PATH
: Equivalent to –locations_file_path command-line flag.
YORC_TASKS_DISPATCHER_LONG_POLLING_WAIT_TIME
: Equivalent to –tasks_dispatcher_long_polling_wait_time command-line flag.
YORC_TASKS_DISPATCHER_LOCK_WAIT_TIME
: Equivalent to –tasks_dispatcher_lock_wait_time command-line flag.
YORC_TASKS_DISPATCHER_METRICS_REFRESH_TIME
: Equivalent to –tasks_dispatcher_metrics_refresh_time command-line flag.
YORC_WORKERS_NUMBER
: Equivalent to –workers_number command-line flag.
YORC_WORKING_DIRECTORY
: Equivalent to –working_directory command-line flag.
YORC_SERVER_ID
: Equivalent to –server_id command-line flag.
YORC_DISABLE_SSH_AGENT
: Equivalent to –disable_ssh_agent command-line flag.
YORC_CONCURRENCY_LIMIT_FOR_UPGRADES
: Equivalent to –concurrency_limit_for_upgrades command-line flag.
YORC_SSH_CONNECTION_TIMEOUT
: Equivalent to –ssh_connection_timeout command-line flag.
YORC_SSH_CONNECTION_RETRY_BACKOFF
: Equivalent to –ssh_connection_retry_backoff command-line flag.
YORC_SSH_CONNECTION_MAX_RETRIES
: Equivalent to –ssh_connection_max_retries command-line flag.
YORC_LOG
: If set to1
orDEBUG
, enables debug logging for Yorc.
YORC_TERRAFORM_PLUGINS_DIR
: Equivalent to –terraform_plugins_dir command-line flag.
YORC_TERRAFORM_AWS_PLUGIN_VERSION_CONSTRAINT
: Equivalent to –terraform_aws_plugin_version_constraint command-line flag.
YORC_TERRAFORM_CONSUL_PLUGIN_VERSION_CONSTRAINT
: Equivalent to –terraform_consul_plugin_version_constraint command-line flag.
YORC_TERRAFORM_GOOGLE_PLUGIN_VERSION_CONSTRAINT
: Equivalent to –terraform_google_plugin_version_constraint command-line flag.
YORC_TERRAFORM_OPENSTACK_PLUGIN_VERSION_CONSTRAINT
: Equivalent to –terraform_openstack_plugin_version_constraint command-line flag.
YORC_TERRAFORM_KEEP_GENERATED_FILES
: Equivalent to –terraform_keep_generated_files command-line flag.
Locations configuration¶
A location allows Yorc to connect to an infrastructure. A location is identified uniquely by its name
property.
Its type
property Specifies the infrastructure related to this location. Yorc can handle multiple locations of the same infrastructure.
Its properties
property contains a map with all required information for the infrastructure connection.
The –locations_file_path option allows user to define the specific locations configuration file path. This configuration is taken in account for the first time the server starts and allows to populate locations for the Yorc cluster. In near future, a REST API will let users add, remove or update existing locations configured in Yorc.
This file can be written either in JSON or YAML format. Here is a JSON example of locations configuration:
{
"locations": [
{ "name": "myOpenstackLocation1",
"type": "openstack",
"properties": {
"auth_url": "http://openstack:5000/v2.0",
"tenant_name": "Tname",
"tenant_id": "use_tid_or_tname",
"user_name": "{{with (secret \"/secret/yorc/mysecret\").Raw}}{{.Data.value}}{{end}}",
"password": "{{secret \"/secret/yorc/mysecret\" \"data=value\" | print}}",
"region": "RegionOne",
"private_network_name": "private-test",
"public_network_name": "not_supported",
"os_default_security_groups": ["default", "lax"]
}
},
{ "name": "myGoogleLocation1",
"type": "google",
"properties": {
"application_credentials": "creds.json",
"project": "my-project-ref"
}
},
....
Builtin locations types configuration¶
Here we have principal infrastructure configurations retrieved as location properties for a specified type.
OpenStack¶
OpenStack location type is openstack
in lower case.
Property Name | Description | Data Type | Required | Default |
---|---|---|---|---|
auth_url |
Specify the authentication url for OpenStack (should be the Keystone endpoint ie: http://your-openstack:5000/v2.0). | string | yes | |
tenant_id |
Specify the OpenStack tenant id to use. | string | Either this or tenant_name should be provided. |
|
tenant_name |
Specify the OpenStack tenant name to use. | string | Either this or tenant_id should be provided. |
|
user_domain_name |
Specify the domain name where the user is located (Identity v3 only). | string | yes (if use Identity v3) | |
project_id |
Specify the ID of the project to login with (Identity v3 only). | string | Either this or project_name should be provided. |
|
project_name |
Specify the name of the project to login with (Identity v3 only). | string | Either this or project_id should be provided. |
|
user_name |
Specify the OpenStack user name to use. | string | yes | |
password |
Specify the OpenStack password to use. | string | yes | |
region |
Specify the OpenStack region to use | string | no | RegionOne |
private_network_name |
Specify the name of private network to use as primary adminstration network between Yorc and Compute instances. It should be a private network accessible by this instance of Yorc. | string | Required to use the PRIVATE keyword for TOSCA
admin networks |
|
provisioning_over_fip_allowed |
This allows to perform the provisioning of a Compute over the associated floating IP if it exists. This is useful when Yorc is not deployed on the same private network than the provisioned Compute. | boolean | no | false |
default_security_groups |
Default security groups to be used when creating a Compute instance. It should be a comma-separated list of security group names | list of strings | no | |
insecure |
Trust self-signed SSL certificates | boolean | no | false |
cacert_file |
Specify a custom CA certificate when communicating over SSL. You can specify either a path to the file or the contents of the certificate | string | no | |
cert |
Specify client certificate file for SSL client authentication. You can specify either a path to the file or the contents of the certificate | string | no | |
key |
Specify client private key file for SSL client authentication. You can specify either a path to the file or the contents of the key | string | no |
Kubernetes¶
Kubernetes location type is kubernetes
in lower case.
Property Name | Description | Data Type | Required | Default |
---|---|---|---|---|
kubeconfig |
Path or content of Kubernetes cluster configuration file* | string | no | |
application_credentials |
Path or content of file containing credentials** | string | no | |
master_url |
URL of the HTTP API of Kubernetes is exposed. Format: https://<host>:<port> |
string | no | |
ca_file |
Path to a trusted root certificates for server | string | no | |
cert_file |
Path to the TLS client certificate used for authentication | string | no | |
key_file |
Path to the TLS client key used for authentication | string | no | |
insecure |
Server should be accessed without verifying the TLS certificate (testing only) | boolean | no | |
job_monitoring_time_interval |
Default duration for job monitoring time interval | string | no | 5s |
kubeconfig
is the path (accessible to Yorc server) or the content of a Kubernetes cluster configuration file. Whenkubeconfig
is defined, other infrastructure configuration properties (master_url
, keys or certificates) don’t have to be defined here.If neither
kubeconfig
normaster_url
is specified, the Orchestrator will consider it is running within a Kubernetes Cluster and will attempt to authenticate inside this cluster.application_credentials
is the path (accessible to Yorc server) or the content of a file containing Google service account private keys in JSON format. This file can be downloaded from the Google Cloud Console at Google Cloud service account file. It is needed to authenticate against Google Cloud when thekubeconfig
property above refers to a Kubernetes Cluster created on Google Kubernetes Engine, and the orchestrator is running on a host where gcloud is not installed.
Google Cloud Platform¶
Google Cloud Platform location type is google
in lower case.
Property Name | Description | Data Type | Required | Default |
---|---|---|---|---|
project |
ID of the project to apply any resources to | string | yes | |
application_credentials |
Path of file containing credentials* | string | no | Google Application Default Credentials |
credentials |
Content of file containing credentials | string | no | Google Application Default Credentials |
region |
The region to operate under | string | no |
application_credentials
is the path (accessible to Yorc server) of a file containing service account private keys in JSON format.
This file can be downloaded from the Google Cloud Console at Google Cloud service account file.
If no file path is specified in application_credentials
and no file content is specified in credentials
, the orchestrator will fall back to using the Google Application Default Credentials if any.
AWS¶
AWS location type is aws
in lower case.
Property Name | Description | Data Type | Required | Default |
---|---|---|---|---|
access_key |
Specify the AWS access key credential. | string | yes | |
secret_key |
Specify the AWS secret key credential. | string | yes | |
region |
Specify the AWS region to use. | string | yes |
Slurm¶
Slurm location type is slurm
in lower case.
Property Name | Description | Data Type | Required | Default |
---|---|---|---|---|
user_name |
SSH Username to be used to connect to the Slurm Client’s node | string | yes (see below for alternatives) | |
password |
SSH Password to be used to connect to the Slurm Client’s node | string | Either this or private_key should be provided |
|
private_key |
SSH Private key to be used to connect to the Slurm Client’s node | string | Either this or password should be provided |
|
url |
IP address of the Slurm Client’s node | string | yes | |
port |
SSH Port to be used to connect to the Slurm Client’s node | string | yes | |
default_job_name |
Default name for the job allocation. | string | no | |
job_monitoring_time_interval |
Default duration for job monitoring time interval | string | no | 5s |
enforce_accounting |
If true, account properties are mandatory for jobs and computes | boolean | no | false |
keep_job_remote_artifacts |
If true, job artifacts are not deleted at the end of the job. | boolean | no | false |
ssh_connection_timeout |
Allow to supersede –ssh_connection_timeout global server option for this specific location. | Duration | no | false |
ssh_connection_retry_backoff |
Allow to supersede –ssh_connection_retry_backoff global server option for this specific location. | Duration | no | false |
ssh_connection_max_retries |
Allow to supersede –ssh_connection_max_retries global server option for this specific location. | uint64 | no | false |
An alternative way to specify user credentials for SSH connection to the Slurm Client’s node (user_name, password or private_key), is to provide them as application properties. In this case, Yorc gives priority to the application provided properties. Moreover, if all the applications provide their own user credentials, the configuration properties user_name, password and private_key, can be omitted. See Working with jobs for more information.
Storage configuration¶
Different artifacts (topologies, logs, events, tasks…) are stored by Yorc during an application deployment.
Previously, everything was stored in Consul KV.
Starting with version 4.0.0, we choosed to refactor the way Yorc stores data mainly for performance reasons, and also to make it more flexible.
Yorc can now store the different kind of artifacts in different stores
configured in a new section of the configuration file called storage
.
- If defined, the
storage
entry may specify the following properties: - the
stores
property allows to customize storage in a different way than the default one. - the
default_properties
allows to change properties settings for the default fileCache store. - The
reset
property allows to redefine the stores or to change properties for default stores when Yorc re-starts. If no set to true, the existing storage is used.
- the
Property Name | Description | Data Type | Required | Default |
---|---|---|---|---|
reset |
See Storage reset note | boolean | no | False |
default_properties |
Properties for default fileCache store. See File cache properties | map | no | |
stores |
Stores configuration | array | no | See Store types |
So now, users can configure different store types for storing the different kind of artifacts, and using different stores implementations.
- Currently Yorc supports 3 store
types
: Deployment
Log
Event
- Yorc supports 5 store
implementations
: consul
file
cipherFile
fileCache
cipherFileCache
elastic
(experimental)
By default, Log
and Event
store types use consul
implementation, and Deployment
store uses fileCache
.
If these default settings correspond to your needs, the Yorc configuration file does not need to have a storage
entry.
If you want to change properties for the default fileCache
store for Deployment
, you have to set the new values in the default_properties
map.
The cache_num_counters
and cache_max_cost
properties can be used to determine the cache size in function of the expected number of items.
The default values are defined for about 100 deployments if we approximate a cache size of 100 K and 100 items for one single deployment.
See Default cache size for file storage is too large.
Pay attention that the cache size must be defined in function of the Yorc host memory resources and a too large cache size can affect performances.
Here is a JSON example of updating default properties for cache used in fileCache
store for Deployment
:
{
"storage": {
"reset": true,
"default_properties":
{
"cache_num_counters": 1e7,
"cache_max_cost": 1e9
}
}
}
The same sample in YAML
storage:
reset: true
default_properties:
cache_num_counters: 1e7
cache_max_cost: 1e9
A store configuration is defined with:
Property Name | Description | Data Type | Required | Default |
---|---|---|---|---|
name |
unique store ID | string | no | generated |
migrate_data_from_consul |
Log and Event data migration from consul. See note below. | bool | no | false |
implementation |
Store implementation. See Store implementations below. | string | yes | |
types |
Store types handled by this instance. See Store types below. | array | yes | |
properties |
Specific store implementation properties. | map | no |
migrate_data_from_consul
allows to migrate data from consul
to another store implementation.
This is useful when a new store is configured (different from consul…) for logs or events.
Store types¶
Currently 3 different store types are supported by Yorc:
Deployment¶
This store type contains data representing the Tosca topologies types (data, nodes, policies, artifacts, relationships, capabilities) and templates (nodes, polocies, repositories, imports, workflows).
Data in this store is written once when a topology is parsed, then read many times during application lifecycle. fileCache
is the default implementation for this store type.
Log¶
Store that contains the applicative logs, also present in Alien4Cloud logs. consul
is the default implementation for this store type.
If you face some Consul memory usage issues, you can choose file
or cipherFile
as logs may contains private information.
Cache is not useful for this kind of data as we use blocking queries and modification index to retrieve it.
Event¶
Store that contains the applicative events, also present in Alien4Cloud events. consul
is the default implementation for this store type.
Same remarks as for Log as it’s same kind of data and usage.
Store implementations¶
Currently Yorc provide 5 implementations (in fact 2 real ones with combinations around file) described below but you’re welcome to contribute and bring your own implementation, you just need to implement the Store interface See Storage interface.
consul¶
This is the Consul KV store used by Yorc for main internal storage stuff. For example, the configuration of the stores is kept in the Consul KV. As Consul is already configurable here: Consul configuration, no other configuration is provided in this section.
file¶
This is a file store without cache system. It can be used for logs and events as this data is retrieved via blocking queries and modification index which can be used with a cache system.
Here are specific properties for this implementation:
Property Name | Description | Data Type | Required | Default |
---|---|---|---|---|
root_dir |
Root directory used for file storage | string | no | work/store |
blocking_query_default_timeout |
default timeout for blocking queries | string | no | 5m (5 minutes) |
concurrency_limit |
Limit for concurrent operations | integer | no | 1000 |
cipherFile¶
This is a file store with file data encryption (AES-256 bits key) which requires a 32-bits length passphrase.
Here are specific properties for this implementation in addition to file
properties:
Property Name | Description | Data Type | Required | Default |
---|---|---|---|---|
passphrase |
Passphrase used to generate the encryption key Required to be 32-bits length | string | yes |
Passphrase
can be set with Secret function
and retrieved from Vault as explained in the Vault integration chapter.
Here is a JSON example of stores configuration with a cipherFile store implementation for logs.
{
"storage": {
"reset": false,
"stores": [
{
"name": "myCipherFileStore",
"implementation": "cipherFile",
"migrate_data_from_consul": true,
"types": ["Log"],
"properties": {
"root_dir": "/mypath/to/store",
"passphrase": "myverystrongpasswordo32bitlength"
}
}
]}
}
The same sample in YAML
storage:
reset: false
stores:
- name: myCipherFileStore
implementation: cipherFile
migrate_data_from_consul: true
types:
- Log
properties:
root_dir: "/mypath/to/store"
passphrase: "myverystrongpasswordo32bitlength"
fileCache¶
This is a file store with a cache system.
Here are specific properties for this implementation:
Property Name | Description | Data Type | Required | Default |
---|---|---|---|---|
cache_num_counters |
number of keys to track frequency of | int64 | no | 1e5 (100 000) |
cache_max_cost |
maximum cost of cache | int64 | no | 1e7 (10 M) |
cache_buffer_items |
number of keys per Get buffer | int64 | no | 64 |
For more information on cache properties, you can refer to Ristretto README
cipherFileCache¶
This is a file store with a cache system and file data encryption (AES-256 bits key) which requires a 32-bits length passphrase.
Stores configuration is saved once when Yorc server starts. If you want to re-initialize storage, you have to set the reset
property to True and restart Yorc.
Warning
Pay attention that if any data is still existing after reset, Yorc will ignore it.
If no storage configuration is set, default stores implementations are used as defined previously to handle all store types (Deployment
, Log
and Event
).
If any storage configuration is set with partial stores types, the missing store types will be added with default implementations.
elastic¶
This store ables you to store Log
s and Event
s in elasticsearch.
Warning
This storage is only suitable to store logs and events.
Per Yorc cluster : 1 index for logs, 1 index for events.
Property Name | Description | Data Type | Required | Default |
---|---|---|---|---|
es_urls |
the ES cluster urls | []string | yes | |
ca_cert_path |
path to the PEM encoded CA’s certificate file when TLS is activated for ES | string | no | |
cert_path |
path to a PEM encoded certificate file when TLS is activated for ES | string | no | |
key_path |
path to a PEM encoded private key file when TLS is activated for ES | string | no | |
index_prefix |
indexes used by yorc can be prefixed | string | no | yorc_ |
es_query_period |
when querying logs and event, we wait this timeout before each request when it returns nothing (until something is returned or the waitTimeout is reached) | duration | no | 4s |
es_refresh_wait_timeout |
used to wait for more than refresh_interval (1s) (until something is returned or the waitTimeout is is reached) | duration | no | 2s |
es_force_refresh |
when querying ES, force refresh index before when waiting for refresh. | bool | no | false |
max_bulk_size |
the maximum size (in kB) of bulk request sent when while migrating data | int64 | no | 4000 |
max_bulk_count |
maximum size (in term of number of documents) when of bulk request sent while migrating data | int64 | no | 1000 |
cluster_id |
used to distinguish logs & events in the indexes if different yorc cluster are writing in the same elastic cluster. If not set, the consul.datacenter will be used. | string | no | |
trace_requests |
to print ES requests (for debug only) | bool | no | false |
trace_events |
to trace events & logs when sent (for debug only) | bool | no | false |
initial_shards |
number of shards used to initialize indices | int64 | no | |
initial_replicas |
number of replicas used to initialize indices | int64 | no |
Vault configuration¶
Due to the pluggable nature of vaults support in Yorc their configuration differ from other configurable options. A vault configuration option could be specified by either its configuration placeholder in the configuration file, a command line flag or an environment variable.
The general principle is for a configurable option option_1
it should be specified in the configuration file as following:
{
"vault": {
"type": "vault_implementation",
"option_1": "value"
}
}
Similarly a command line flag with the name --vault_option_1
and an environment variable with the name YORC_VAULT_OPTION_1
will be
automatically supported and recognized. The default order of precedence apply here.
type
is the only mandatory option for all vaults configurations, it allows to select the vault implementation by specifying it’s ID. If the
type
option is not present either in the config file, as a command line flag or as an environment variable, Vault configuration will be ignored.
The integration with a Vault is totally optional and this configuration part may be leave empty.
Builtin Vaults configuration¶
HashiCorp’s Vault¶
This is the only builtin supported Vault implementation.
Implementation ID to use with the vault type configuration parameter is hashicorp
.
Bellow are recognized configuration options for Vault:
Option Name | Description | Data Type | Required | Default |
---|---|---|---|---|
address |
Address is the address of the Vault server. This should be a complete URL such as “https://vault.example.com”. | string | yes | |
max_retries |
MaxRetries controls the maximum number of times to retry when a 5xx error occurs. Set to 0 or less to disable retrying. | integer | no | 0 |
timeout |
Timeout is for setting custom timeout parameter in the HttpClient. | string | no | |
ca_cert |
CACert is the path to a PEM-encoded CA cert file to use to verify the Vault server SSL certificate. | string | no | |
ca_path |
CAPath is the path to a directory of PEM-encoded CA cert files to verify the Vault server SSL certificate. | string | no | |
client_cert |
ClientCert is the path to the certificate for Vault communication. | string | no | |
client_key |
ClientKey is the path to the private key for Vault communication | string | no | |
tls_server_name |
TLSServerName, if set, is used to set the SNI host when connecting via TLS. | string | no | |
tls_skip_verify |
Disables SSL verification | boolean | no | false |
token |
Specifies the access token to use to connect to vault. This is highly discouraged to this option in the configuration file as the token is a sensitive data and should not be written on disk. Prefer the associated environment variable | string | no |
Yorc Client CLI Configuration¶
This section is dedicated to the CLI part of yorc that covers everything except the server configuration detailed above. It focus on configuration options commons to all the commands. Sub commands may have additional options please use the cli help command to see them.
Just like for its server part Yorc Client CLI has various configuration options that could be specified either by command-line flags, configuration file or environment variables.
If an option is specified several times using flags, environment and config file, command-line flag will have the precedence then the environment variable and finally the value defined in the configuration file.
Command-line options¶
--ca_file
: This provides a file path to a PEM-encoded certificate authority. This implies the use of HTTPS to connect to the Yorc REST API.
--ca_path
: Path to a directory of PEM-encoded certificates authorities. This implies the use of HTTPS to connect to the Yorc REST API.
--cert_file
: File path to a PEM-encoded client certificate used to authenticate to the Yorc API. This must be provided along with key-file. If one of key-file or cert-file is not provided then SSL authentication is disabled. If both cert-file and key-file are provided this implies the use of HTTPS to connect to the Yorc REST API.
-c
or--config
: config file (default is /etc/yorc/yorc-client.[json|yaml])
--key_file
: File path to a PEM-encoded client private key used to authenticate to the Yorc API. This must be provided along with cert-file. If one of key-file or cert-file is not provided then SSL authentication is disabled. If both cert-file and key-file are provided this implies the use of HTTPS to connect to the Yorc REST API.
--skip_tls_verify
: Controls whether a client verifies the server’s certificate chain and host name. If set to true, TLS accepts any certificate presented by the server and any host name in that certificate. In this mode, TLS is susceptible to man-in-the-middle attacks. This should be used only for testing. This implies the use of HTTPS to connect to the Yorc REST API.
-s
or--ssl_enabled
: Use HTTPS to connect to the Yorc REST API. This is automatically implied if one of--ca_file
,--ca_path
,--cert_file
,--key_file
or--skip_tls_verify
is provided.
--yorc_api
: specify the host and port used to join the Yorc’ REST API (default “localhost:8800”)
Configuration files¶
Configuration files are either JSON or YAML formatted as a single object containing the following configuration options.
By default Yorc will look for a file named yorc-client.json or yorc-client.yaml in /etc/yorc
directory then if not found in the current directory.
The –config command line flag allows to specify an alternative configuration file.
ca_file
: Equivalent to –ca_file command-line flag.
ca_path
: Equivalent to –ca_path command-line flag.
cert_file
: Equivalent to –cert_file command-line flag.
key_file
: Equivalent to –key_file command-line flag.
skip_tls_verify
: Equivalent to –skip_tls_verify command-line flag.
ssl_enabled
: Equivalent to –ssl_enabled command-line flag.
yorc_api
: Equivalent to –yorc_api command-line flag.
Environment variables¶
YORC_CA_FILE
: Equivalent to –ca_file command-line flag.
YORC_CA_PATH
: Equivalent to –ca_path command-line flag.
YORC_CERT_FILE
: Equivalent to –cert_file command-line flag.
YORC_KEY_FILE
: Equivalent to –key_file command-line flag.
YORC_SKIP_TLS_VERIFY
: Equivalent to –skip_tls_verify command-line flag.
YORC_SSL_ENABLED
: Equivalent to –ssl_enabled command-line flag.
YORC_API
: Equivalent to –yorc_api command-line flag.