Yorc Telemetry

Yorc collects various runtime metrics. These metrics are aggregated on a ten second interval and are retained for one minute.

To view this data, you must send a signal to the Yorc process: on Unix, this is USR1 while on Windows it is BREAK. Once Yorc receives the signal, it will dump the current telemetry information to stderr.

Telemetry information can be streamed to both statsite as well as statsd or pull from Prometheus based on providing the appropriate configuration options. See Telemetry configuration for more information.

Below is sample output (lot of metrics omitted for brevity) of a telemetry dump:

[2017-07-19 16:31:00 +0200 CEST][G] 'yorc.yorc-server-0.runtime.alloc_bytes': 73723728.000
[2017-07-19 16:31:00 +0200 CEST][G] 'yorc.yorc-server-0.workers.free': 2.000
[2017-07-19 16:31:00 +0200 CEST][C] 'yorc.http.200.GET.metrics': Count: 2 Sum: 2.000 LastUpdated: 2017-07-19 16:31:06.253380804 +0200 CEST
[2017-07-19 16:31:00 +0200 CEST][S] 'yorc.tasks.maxBlockTimeMs': Count: 10 Sum: 0.000 LastUpdated: 2017-07-19 16:31:09.805073861 +0200 CEST
[2017-07-19 16:31:00 +0200 CEST][S] 'yorc.http.GET.metrics': Count: 2 Min: 27.765 Mean: 29.474 Max: 31.183 Stddev: 2.417 Sum: 58.948 LastUpdated: 2017-07-19 16:31:06.253392224 +0200 CEST
[2017-07-19 16:31:10 +0200 CEST][S] 'yorc.tasks.maxBlockTimeMs': Count: 10 Sum: 0.000 LastUpdated: 2017-07-19 16:31:19.986227315 +0200 CEST
[2017-07-19 16:31:20 +0200 CEST][C] 'yorc.http.200.GET.metrics': Count: 2 Sum: 2.000 LastUpdated: 2017-07-19 16:31:26.257243322 +0200 CEST
[2017-07-19 16:31:20 +0200 CEST][S] 'yorc.tasks.maxBlockTimeMs': Count: 9 Sum: 0.000 LastUpdated: 2017-07-19 16:31:29.138694946 +0200 CEST
[2017-07-19 16:31:20 +0200 CEST][S] 'yorc.http.GET.metrics': Count: 2 Min: 32.371 Mean: 41.727 Max: 51.083 Stddev: 13.232 Sum: 83.454 LastUpdated: 2017-07-19 16:31:26.257253638 +0200 CEST

Key metrics

Metric Types

Type Description Quantiles
Gauge Gauge types report an absolute number at the end of the aggregation interval. false
Counter Counts are incremented and flushed at the end of the aggregation interval and then are reset to zero. true
Timer Timers measure the time to complete a task and will include quantiles, means, standard deviation, etc per interval. true

Go Runtime metrics

Metric Name Description Unit Metric Type
yorc.runtime.num_goroutines This tracks the number of running goroutines and is a general load pressure indicator. This may burst from time to time but should return to a steady state value. number of goroutines gauge
yorc.runtime.alloc_bytes This measures the number of bytes allocated by the Yorc process. This may burst from time to time but should return to a steady state value. bytes gauge
yorc.runtime.heap_objects This measures the number of objects allocated on the heap and is a general memory pressure indicator. This may burst from time to time but should return to a steady state value. number of objects gauge
yorc.runtime.sys Sys is the total bytes of memory obtained from the OS.Sys measures the virtual address space reserved by the Go runtime for the heap, stacks, and other internal data structures. It’s likely that not all of the virtual address space is backed by physical memory at any given moment, though in general it all was at some point. bytes gauge
yorc.runtime.malloc_count Mallocs is the cumulative count of heap objects allocated. The number of live objects is Mallocs - Frees. number of Mallocs gauge
yorc.runtime.free_count Frees is the cumulative count of heap objects freed. number of frees gauge
yorc.runtime.total_gc_pause_ns PauseTotalNs is the cumulative nanoseconds in GC stop-the-world pauses since the program started. During a stop-the-world pause, all goroutines are paused and only the garbage collector can run. nanoseconds gauge
yorc.runtime.total_gc_runs Gc runs is the number of completed GC cycles. number of cycles gauge
yorc.runtime.gc_pause_ns Latest GC run stop-the-world pause duration. nanoseconds timer

Yorc REST API metrics

Metric Name Description Unit Metric Type
yorc.http.<Method>.<Path> This measures the duration of an API call. <Method> is the HTTP verb and <Path> the Path part of the URL where slashes are replaced by dashes. milliseconds timer
yorc.http.<Status>.<Method>.<Path> This counts the number of API calls by HTTP status codes (ie: 200, 404, 500, …) , HTTP verb and URL path as described above. number of requests counter

Yorc Workers & Tasks metrics

Metric Name Description Unit Metric Type
yorc.workers.free This tracks the number of free Yorc workers. number of free workers gauge
tasks.maxBlockTimeMs This measures the highest duration since creation for all waiting tasks. milliseconds timer
tasks.nbWaiting This tracks the number of tasks waiting for being processed. number of tasks gauge
tasks.wait This measures the finally waited time for a task being processed. milliseconds timer
task.<DepID>.<Type>.<FinalStatus> This counts by deployment and task type the final status of a task. number of tasks counter
task.<DepID>.<Type> This measures the task processing duration. milliseconds timer

Yorc Executors metrics

There are two types of executors in Yorc “delegates executors” and “operations executors”. Delegates executors handle the deployment of Yorc natively supported TOSCA nodes (like an Openstack compute for instance) while Operations executors handle implementations of an lifecycle operations provided as part of the TOSCA node definition (like a shell script or an ansible playbook).

In the below table <ExecType> is the executor type, <DepID> the deployment ID, <NodeType> the fully qualified TOSCA node type where dots where replaced by dashes and <OpName> the TOSCA operation name where dots where replaced by dashes.

Metric Name Description Unit Metric Type
yorc.executor.<ExecType>.<DepID>.<NodeType>.<OpName> This measures the duration of an execution. milliseconds timer
yorc.executor.<ExecType>.<DepID>.<NodeType>.<OpName>.failures This counts the number of failed executions. number of failures counter
yorc.executor.<ExecType>.<DepID>.<NodeType>.<OpName>.successes This counts the number of successful executions. number of successes counter

Yorc SSH connection pool

Metric Name Description Unit Metric Type
yorc.ssh-connections-pool.<connection_id>.sessions.open-failed This tracks the number of failures when opening an SSH session (multiplexed on top of an existing connection). number of failures counter
yorc.ssh-connections-pool.<connection_id>.sessions.creations This measures the number of sessions created for a given connection. number of sessions counter
yorc.ssh-connections-pool.<connection_id>.sessions.closes This measures the number of sessions closed for a given connection. number of sessions counter
yorc.ssh-connections-pool.<connection_id>.sessions.open This tracks the number of currently open sessions per connection number of sessions gauge
yorc.ssh-connections-pool.creations.<connection_id> This measures the number of created connections. number of connection counter
yorc.ssh-connections-pool.closes.<connection_id> This measures the number of closed connections. number of connection counter