The OpenTelemetry injector is a shared library (written in Zig) that is intended to be
used via the environment variable LD_PRELOAD
, the
/etc/ld.so.preload
file, or similar mechanisms to inject
environment variables into processes at startup.
It serves two main purposes:
- Inject an OpenTelemetry Auto Instrumentation agent into the process to capture and report distributed traces and metrics to the OpenTelemetry Collector for supported runtimes.
- Set resource attributes automatically, (for example Kubernetes related resource attributes and service related resource attributes in environments where this is applicable).
The injector can be used to enable automatic zero-touch instrumentation of processes. For this to work, the injector binary needs to be bundled together with the OpenTelemetry auto-instrumentation agents for the target runtimes.
Official RPM and DEB packages that contain the injector as well as the auto-instrumentation agents are available, and
can be downloaded from the releases page
(once the first relase has been published).
The OpenTelemetry injector Debian/RPM packages install the OpenTelemetry auto-instrumentation agents, the
libotelinject.so
shared object library, and a default configuration file to automatically instrument applications and
services to capture and report distributed traces and metrics to the
OpenTelemetry Collector
The opentelemetry-injector
deb/rpm package installs and supports configuration of the following Auto
Instrumentation agents:
The following methods are supported to manually activate and configure Auto Instrumentation after installation of the
opentelemetry-injector
deb/rpm package (requires root
privileges):
Note: To prevent conflicts and duplicate traces/metrics, only one method should be activated on the target system.
-
Add the path of the provided
/usr/lib/opentelemetry/libotelinject.so
shared object library to the/etc/ld.so.preload
file to activate Auto Instrumentation for all supported processes on the system. For example:echo /usr/lib/opentelemetry/libotelinject.so >> /etc/ld.so.preload
Alternatively, set the environment variable
LD_PRELOAD=/usr/lib/opentelemetry/libotelinject.so
for a specific process to activate auto-instrumentation for tha process. For example:LD_PRELOAD=/usr/lib/opentelemetry/libotelinject.so node myapp.js
-
The default configuration file
/etc/opentelemetry/otelinject.conf
includes the required settings, i.e. the paths to the respective auto-instrumentation agents per runtime:dotnet_auto_instrumentation_agent_path_prefix=/usr/lib/opentelemetry/dotnet jvm_auto_instrumentation_agent_path=/usr/lib/opentelemetry/javaagent.jar nodejs_auto_instrumentation_agent_path=/usr/lib/opentelemetry/otel-js/node_modules/@opentelemetry-js/otel/instrument
There is usually no need to modify this file, unless you want to provide your own instrumentation files.
However, the configuration file
/etc/opentelemetry/otelinject.conf
can also be used to selectively disable auto-instrumentation for a specific runtime, by setting the respective path to an empty string. For example, the following file would leave JVM and Node.js auto-instrumentation active, while disabling .NET auto-instrumentation:dotnet_auto_instrumentation_agent_path_prefix= jvm_auto_instrumentation_agent_path=/usr/lib/opentelemetry/javaagent.jar nodejs_auto_instrumentation_agent_path=/usr/lib/opentelemetry/otel-js/node_modules/@opentelemetry-js/otel/instrument
The paths set in
/etc/opentelemetry/otelinject.conf
can be overridden with environment variables. (This should usually not be necessary.)DOTNET_AUTO_INSTRUMENTATION_AGENT_PATH_PREFIX
: the path to the directory containing the .NET Auto Instrumentation agent filesJVM_AUTO_INSTRUMENTATION_AGENT_PATH
: the path to the Java auto-instrumentation agent JAR fileNODEJS_AUTO_INSTRUMENTATION_AGENT_PATH
: the path to the Node.js auto-instrumentation agent registration file
These aforementioned environment variables can also be used to selectively disable auto-instrumentation for a specific runtime, by setting the respective variable to an empty string, that is, set:
DOTNET_AUTO_INSTRUMENTATION_AGENT_PATH_PREFIX=""
to disable .NET auto-instrumentationJVM_AUTO_INSTRUMENTATION_AGENT_PATH=""
to disable JVM auto-instrumentationNODEJS_AUTO_INSTRUMENTATION_AGENT_PATH=""
to disable Node.js auto-instrumentation
-
Reboot the system or restart the applications/services for any changes to take effect. The
libotelinject.so
shared object library will then be preloaded for all subsequent processes and inject the environment variables from the/etc/opentelemetry/otelinject
configuration files for Java and Node.js processes.
When providing your own instrumentation files (for example via environment variables like DOTNET_AUTO_INSTRUMENTATION_AGENT_PATH_PREFIX
) the following directory structure is expected:
JVM_AUTO_INSTRUMENTATION_AGENT_PATH
: This path must point to the Java auto-instrumentation agent JAR fileopentelemetry-javaagent.jar
.NODEJS_AUTO_INSTRUMENTATION_AGENT_PATH
: The path to an installation of the npm module@opentelemetry/auto-instrumentations-node
.DOTNET_AUTO_INSTRUMENTATION_AGENT_PATH_PREFIX
: this path must be a directory that contains the following subdirectories and files:- For
x86_64
systems usingglibc
:glibc/linux-x64/OpenTelemetry.AutoInstrumentation.Native.so
glibc/AdditionalDeps
glibc/store
glibc/net/OpenTelemetry.AutoInstrumentation.StartupHook.dll
- For
x86_64
systems usingmusl
:musl/linux-musl-x64/OpenTelemetry.AutoInstrumentation.Native.so
musl/AdditionalDeps
musl/store
musl/net/OpenTelemetry.AutoInstrumentation.StartupHook.dll
- For
arm64
systems usingglibc
:glibc/linux-arm64/OpenTelemetry.AutoInstrumentation.Native.so
glibc/AdditionalDeps
glibc/store
glibc/net/OpenTelemetry.AutoInstrumentation.StartupHook.dll
- For
arm
systems usingmusl
:musl/linux-musl-arm64/OpenTelemetry.AutoInstrumentation.Native.so
musl/AdditionalDeps
musl/store
musl/net/OpenTelemetry.AutoInstrumentation.StartupHook.dll
- For
Note that the defaults provided by the RPM and Debian packages take care of all of that, and it is not necessary to
edit /etc/opentelemetry/otelinject.conf
or set any of the above environment variables.
Check the following for details about the auto-instrumtation agents and further configuration options:
Note: The following steps utilize a sample
systemd
drop-in file to activate/configure the provided agents for allsystemd
services via default environment variables.Systemd
supports many options, methods, and paths for configuring environment variables at the system level or for individual services, and are not limited to the steps below. Before making any changes, it is recommended to consult the documentation specific to your Linux distribution or service, and check the existing configurations of the system and individual services for potential conflicts or to override an environment variable for a particular service. For general details aboutsystemd
, see thesystemd
man page.
- Copy the provided sample
systemd
drop-in file/usr/lib/opentelemetry/examples/systemd/00-otelinject-instrumentation.conf
to the host'ssystemd
drop-in configuration directory to activate Auto Instrumentation for all supported applications running assystemd
services. For example:This file includes the required environment variables to activate the respective agents with the default options:mkdir -p /usr/lib/systemd/system.conf.d/ && cp /usr/lib/opentelemetry/examples/systemd/00-otelinject-instrumentation.conf /usr/lib/systemd/system.conf.d/
- Java:
DefaultEnvironment="JAVA_TOOL_OPTIONS=-javaagent:/usr/lib/opentelemetry/otel-javaagent.jar"
- Node.js:
DefaultEnvironment="NODE_OPTIONS=-r /usr/lib/opentelemetry/otel-js/node_modules/@opentelemetry/auto-instrumentations-node/register"
- .NET
DefaultEnvironment="CORECLR_ENABLE_PROFILING=1" DefaultEnvironment="CORECLR_PROFILER={918728DD-259F-4A6A-AC2B-B85E1B658318}" DefaultEnvironment="CORECLR_PROFILER_PATH=/usr/lib/opentelemetry/otel-dotnet/linux-x64/OpenTelemetry.AutoInstrumentation.Native.so" DefaultEnvironment="DOTNET_ADDITIONAL_DEPS=/usr/lib/opentelemetry/otel-dotnet/AdditionalDeps" DefaultEnvironment="DOTNET_SHARED_STORE=/usr/lib/opentelemetry/otel-dotnet/store" DefaultEnvironment="DOTNET_STARTUP_HOOKS=/usr/lib/opentelemetry/otel-dotnet/net/OpenTelemetry.AutoInstrumentation.StartupHook.dll" DefaultEnvironment="OTEL_DOTNET_AUTO_HOME=/usr/lib/opentelemetry/otel-dotnet"
- Java:
- To configure the activated agents, add/update
DefaultEnvironment
within the target file from the previous step for the desired environment variables. For example:Check the following for all supported environment variables and default values:cat <<EOH >> /usr/lib/systemd/system.conf.d/00-otelinject-instrumentation.conf DefaultEnvironment="OTEL_EXPORTER_OTLP_ENDPOINT=http://127.0.0.1:4317" DefaultEnvironment="OTEL_RESOURCE_ATTRIBUTES=deployment.environment=my_deployment_environment" DefaultEnvironment="OTEL_SERVICE_NAME=my_service_name" EOH
- Reboot the system, or run
systemctl daemon-reload
and then restart the applicablesystemd
services for any changes to take effect.
See CONTRIBUTING.md.
- Antoine Toulme, Splunk
- Jacob Aronoff, Omlet
- Michele Mancioppi, Dash0
- Bastian Krol, Dash0
For more information about the maintainer role, see the community repository.