Oracle Cloud Infrastructure Documentation

opc-request-id values seen by the client, API Gateway logs, and backend do not match as expected

Find out how to troubleshoot opc-request-id correlation and propagation behavior when calling APIs, having successfully created API gateways and API deployments with the API Gateway service.

Use opc-request-id values to correlate a client request with API Gateway logs and back-end logs. The value that the client sends might differ from the value that the client receives in the response and the value that the back-end service receives from API Gateway.

API Gateway parses the incoming value, keeps selected parts of the value, and generates a new span ID for the request. Use the parsing behavior in this topic to choose the correct value for log correlation.

Issue Symptoms

You might see one or more of the following symptoms:

  • The opc-request-id response header returned to the client does not match the value that the client sent.

  • The back-end service logs an opc-request-id value that differs from the value returned to the client.

  • Application Performance Monitoring (APM) or back-end logs show unexpected collisions, missing values, or truncation when a client-supplied opc-request-id contains a slash (/).

  • You cannot find the same request across the client response, API Gateway access logs, API Gateway execution logs, and back-end logs by using a single header value.

Possible Causes

The values can differ for the following reasons:

  • API Gateway does not treat opc-request-id as an opaque pass-through value.

  • API Gateway splits the incoming value on the slash (/) character and uses only the customer ID and trace ID parts.

  • API Gateway generates a new span ID for each request.

  • The response header returned to the client includes the generated span ID, but the header sent to the back-end service does not include the span ID.

  • Any client-supplied third segment, such as a span ID, is ignored for propagation.

Understand opc-request-id Parsing

API Gateway splits an incoming opc-request-id value on slash (/) characters, with a maximum of three parts. It uses the first two parsed parts and ignores any third part.

API Gateway interprets the parsed parts as follows:

  • First part: Customer ID.

  • Second part: Trace ID.

  • Third part: Ignored value.

API Gateway then generates a span ID for the request and builds the following values:

  • Response header returned to the client: customerId/traceId/spanId.

  • Request header sent to the back-end service: customerId/traceId.

Use the full response header value when you search API Gateway logs. Use the two-part downstream value when you search back-end logs.

Review Common Input Patterns

Use these examples to understand how API Gateway changes common opc-request-id values.

No opc-request-id Header

If the client does not send an opc-request-id header, API Gateway generates a trace ID and a span ID. The customer ID is empty.

The resulting values have the following format:

  • Client response header: /<generated-trace-id>/<generated-span-id>

  • Back-end request header: /<generated-trace-id>

One Segment: abcdef

If the client sends the following header, API Gateway treats the value as the customer ID:

opc-request-id: abcdef

The interpreted values are as follows:

  • Customer ID: abcdef.

  • Trace ID: Generated by API Gateway.

  • Span ID: Generated by API Gateway.

The resulting values have the following format:

  • Client response header: abcdef/<generated-trace-id>/<generated-span-id>

  • Back-end request header: abcdef/<generated-trace-id>

Leading Slash: /abcdef

If the client sends the following header, API Gateway treats the value after the slash as the trace ID:

opc-request-id: /abcdef

The interpreted values are as follows:

  • Customer ID: Empty.

  • Trace ID: abcdef.

  • Span ID: Generated by API Gateway.

The resulting values have the following format:

  • Client response header: /abcdef/<generated-span-id>

  • Back-end request header: /abcdef

A leading slash changes the meaning of the value because the customer ID part is empty.

Two Segments: customer/trace

If the client sends the following header, API Gateway uses both supplied parts:

opc-request-id: customer/trace

The interpreted values are as follows:

  • Customer ID: customer

  • Trace ID: trace

  • Span ID: Generated by API Gateway

The resulting values have the following format:

  • Client response header: customer/trace/<generated-span-id>

  • Back-end request header: customer/trace

Three Segments: customer/trace/span

If the client sends the following header, API Gateway ignores the third segment and generates a new span ID:

opc-request-id: customer/trace/span

The interpreted values are as follows:

  • Customer ID: customer.

  • Trace ID: trace.

  • Third segment: Ignored.

  • Span ID: Generated by API Gateway.

The resulting values have the following format:

  • Client response header: customer/trace/<generated-span-id>

  • Back-end request header: customer/trace.

The client-supplied span value is not preserved.

More Than Three Segments: customer/trace/span/extra

If the client sends the following header, API Gateway still uses only the first two segments:

opc-request-id: customer/trace/span/extra

The interpreted values are as follows:

  • Customer ID: customer

  • Trace ID: trace

All content after the second slash is ignored for propagation.

The resulting values have the following format:

  • Client response header: customer/trace/<generated-span-id>

  • Back-end request header: customer/trace

Compare Request ID Values

Send test requests and compare the opc-request-id value in the following locations:

  • The header value that the client sends.

  • The response header value that API Gateway returns to the client.

  • The request header value that the back-end service receives from API Gateway.

For example, send a request with a known opc-request-id value:

curl -i https://<gateway-hostname>/<deployment-path-prefix>/<api-route-path> \
-H "opc-request-id: <your-request-id>"

Test at least the following values:

  • No opc-request-id header.

  • opc-request-id: abcdef.

  • opc-request-id: /abcdef.

  • opc-request-id: customer/trace.

  • opc-request-id: customer/trace/span.

Review Log Entries

Use the full opc-request-id value returned in the client response header when you search API Gateway access logs and execution logs.

Check the following log entries:

  • The API Gateway access log entry for the request.

  • The API Gateway execution log entry for the request.

  • The back-end log entry for the forwarded request.

Do not assume that the back-end service logs the same full value that API Gateway returns to the client. The back-end service receives the downstream form, customerId/traceId, not the client-visible form, customerId/traceId/spanId.

Fix Correlation Problems

Use the following practices to avoid correlation problems:

  • Do not start a client-supplied opc-request-id value with a slash (/) unless you want an empty customer ID and an explicit trace ID.

  • Do not rely on a client-supplied third segment being preserved.

  • If your back-end service or APM tool expects a single opaque correlation string, use a separate application header for that correlation value.

  • Use the full client-visible response header value for API Gateway log correlation.

  • Use the two-part downstream value when you check back-end logs.

Verify Request ID Correlation

After you update the request format or correlation strategy, verify the result:

  • Send the same request again.

  • Confirm that the opc-request-id response header has the expected structure.

  • Confirm that you can find the same request in access logs and execution logs.

  • Confirm that the back-end service receives the expected two-part opc-request-id value.

  • Confirm that your APM tool or back-end log parser does not require the response header value and the downstream request value to be identical.

For More Information

For more information, see: