containerd/docs/tracing.md
Swagat Bora 3b87d46ce2 Add tracing spans in CRI image service and pull.go
Signed-off-by: Swagat Bora <sbora@amazon.com>

Add spans around image unpack operations
Use image.ref to denote image name and image.id for the image config digest
Add top-level spand and record errors in the CRI instrumentation service
2022-11-03 17:03:43 +00:00

2.5 KiB

Tracing

containerd supports OpenTelemetry tracing since v1.6.0. Tracing currently targets only gRPC calls.

Sending traces from containerd deamon

By configuring io.containerd.tracing.processor.v1.otlp plugin. containerd daemon can send traces to the specified OpenTelemetry endpoint.

version = 2

[plugins."io.containerd.tracing.processor.v1.otlp"]
    endpoint = "http://localhost:4318"

The following options are supported.

  • endpoint: The address of a server that receives OpenTelemetry Protocol.
  • protocol: OpenTelemetry supports multiple protocols. The default value is "http/protobuf". "grpc" is also supported.
  • insecure: Disable transport security when the protocol is "grpc". The default is false. "http/protobuf" always uses the schema provided by the endpoint and the value of this setting being ignored.

The sampling ratio and the service name on the traces could be configured by io.containerd.internal.v1.tracing plugin.

version = 2

[plugins."io.containerd.internal.v1.tracing"]
    sampling_ratio = 1.0
    service_name = "containerd"

Sending traces from containerd client

By configuring its underlying gRPC client, containerd's Go client can send traces to an OpenTelemetry endpoint.

Note that the Go client's methods and gRPC calls are not 1:1. Single method call would issue multiple gRPC calls.

func clientWithTrace() error {
	exp, err := otlptracehttp.New(ctx,
		otlptracehttp.WithEndpoint("localhost:4318"),
		otlptracehttp.WithInsecure(),
	)
	if err != nil {
		return err
	}

	res, err := resource.New(ctx, resource.WithAttributes(
		semconv.ServiceNameKey.String("CLIENT NAME"),
	))
	if err != nil {
		return err
	}

	provider := trace.NewTracerProvider(
		trace.WithSampler(trace.AlwaysSample()),
		trace.WithSpanProcessor(trace.NewSimpleSpanProcessor(exp)),
		trace.WithResource(res),
	)
	otel.SetTracerProvider(provider)
	otel.SetTextMapPropagator(propagation.TraceContext{})

    ...

    dialOpts := []grpc.DialOption{
        grpc.WithTransportCredentials(insecure.NewCredentials()),
        grpc.WithStreamInterceptor(otelgrpc.StreamClientInterceptor()),
        grpc.WithUnaryInterceptor(otelgrpc.UnaryClientInterceptor()),
    }
    client, ctx, cancel, err := commands.NewClient(context, containerd.WithDialOpts(dialOpts))
    if err != nil {
        return err
    }
    defer cancel()

    ctx, span := tracing.StartSpan(ctx, "OPERATION NAME")
    defer tracing.StopSpan(span)
    ...
}