This README is a quick-start guide that shows how to deploy a minimal tracing setup and view your charm’s traces in action.

On your machine, make sure you have the following prerequisites met:

• Terraform >= v1.5
• Juju >= 3.6.0
• Charmcraft >= 3
• A bootsrapped K8s Juju controller

This is the juju K8s model where the testing tracing setup will be deployed

juju add-model cos

In this directory, use terraform to deploy the module:

terraform -chdir=terraform init
terraform -chdir=terraform apply -var="model=cos" -auto-approve

Wait for a couple of minutes (~6m) until all deployed charms are in active/idle.

Step 1: add a charm-tracing relation to your charmcraft.yaml:

requires:
    charm-tracing:
        interface: tracing
        limit: 1
        optional: true

Step 2: change your ops dependency to ops[tracing] in pyproject.toml or requirements.txt and update the lock file, if any.

Step 3: instantiate the Tracing object in your charm's __init__:

    def __init__(self, framework: ops.Framework):
        super().__init__(framework)
        self.tracing = ops.tracing.Tracing(self, "charm-tracing")
        ...

Note that you don't need an extra import, ops.tracing is a namespace, not a module.

Step 4: run unit tests to check against typos or silly errors.

Step 1: add a cos-agent relation to your charmcraft.yaml:

provides: 
  cos-agent:
    interface: cos_agent
    limit: 1
    optional: true

Step 2: change your ops dependency to ops[tracing] in pyproject.toml or requirements.txt and update the lock file, if any.

Step 3: fetch the cos_agent charm library

charmcraft fetch-lib charms.grafana_agent.v0.cos_agent

Step 4: instantiate the COSAgentProvider object in your charm's __init__:

...
import ops
from charms.grafana_agent.v0.cos_agent import COSAgentProvider, charm_tracing_config
...
    def __init__(self, framework: ops.Framework):
        super().__init__(framework)
        self._cos_agent = COSAgentProvider(
            self,
            tracing_protocols=["otlp_http"],
        )
        ...
        self._reconcile_charm_tracing()

    def _reconcile_charm_tracing(self):
        endpoint, _ = charm_tracing_config(self._cos_agent, None)
        if not endpoint:
            return
        ops.tracing.set_destination(
            url=endpoint + "/v1/traces",
            ca=None,
        )

Note that this instrumentation is just a basic setup to send charm traces to Tempo over plain HTTP. For more production-ready code, you’ll likely need to handle CA certs (since Tempo would probably run behind TLS) and have a more mature Observability instrumentation in place (logging, metrics scraping, dashboards, alert rules, etc.).

After instrumenting your charm, pack and deploy it:

cd <charm-path>
charmcraft pack
juju deploy <charm-path> $(yq eval '.resources | to_entries | map("--resource \(.key)=\(.value.upstream-source)") | .[]' charmcraft.yaml)

Then, if it's a K8s charm, integrate it directly with Tempo

juju integrate <your-charm-app-name>:charm-tracing tempo

If it's a machine charm, it's recommended to use the cos_agent interface.

1. In the COS K8s model:

juju offer tempo:tracing

2. In your machine model, deploy an otel collector

juju deploy opentelemetry-collector otelcol --channel 2/edge

3. Integrate your charm with otel collector over cos-agent

juju integrate <your-charm-app-name> otelcol:cos-agent

4. CMR with Tempo:

juju consume <k8s-controller>:admin/cos.tempo
juju integrate otelcol:send-traces tempo

It's ok if otelcol is in blocked/idle. Charm traces should still go through.

To open Grafana's UI, In your COS K8s model, you can:

juju run grafana/0 get-admin-password

That should provide you with grafana's ingressed url that you can open from your browser + an initial admin password that you can use to login.

username: admin
password: <whatever was outputted from the juju action>

Then,

1. Open the Toggle Menu(☰) in the top-left corner
2. Select Explore
3. In the datasource dropdown, choose your Tempo datasource
4. Use the Filters section to search for your charm’s Service Name
5. Click Run query to view traces from your charm

Now, all what's left is to inspect your spans and look for potentials bottlenecks! 🔍