Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions solutions/observability/apm/opentelemetry/attributes.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,12 @@

In OpenTelemetry, an attribute is a key-value pair. Attributes are similar to [labels](/solutions/observability/apm/metadata.md#apm-data-model-labels) in that they add metadata to transactions, spans, and other entities.

:::{note}
This page describes how attributes are handled when sending OTLP data to {{apm-server-or-mis}} (the `.apm` endpoint). On this path, attributes are mapped to ECS fields, and unmapped attributes are stored under `labels.*`.

If you use the [{{motlp}}](opentelemetry://reference/motlp.md) or [{{agent}} in OTel mode](elastic-agent://reference/edot-collector/modes.md) as your ingest path, your data is stored in OTel-native format. Attributes are preserved under `resource.attributes.*` and `attributes.*`. For a comparison of both formats and guidance on choosing your ingest path, refer to [How your ingest path determines the data format](opentelemetry://reference/compatibility/data-streams.md#ingest-path-data-format).

Check failure on line 21 in solutions/observability/apm/opentelemetry/attributes.md

View workflow job for this annotation

GitHub Actions / build / build

'reference/compatibility/data-streams.md' has no anchor named: '#ingest-path-data-format'.
:::

Resource attributes are a type of attribute that contains information about the entities that produce telemetry. Resource attributes map to Elastic Common Schema (ECS) fields like `service.*`, `cloud.*`, `process.*`, and so on. These fields describe the service and its environment.

For example, Elastic APM maps the OpenTelemetry `deployment.environment` field to the ECS `service.environment` field on ingestion.
Expand Down
9 changes: 5 additions & 4 deletions solutions/observability/apm/opentelemetry/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@
Other Integrations which are beats-based include dashboards based on ECS data and are not compatible with OpenTelemetry semantic conventions.
:::

Elastic offers several [{{edot}}](opentelemetry://reference/index.md) distributions. Each is a customized version of an OpenTelemetry language SDK and the OpenTelemetry Collector, ready to send data to the [Managed OTLP endpoint](opentelemetry://reference/motlp.md), APM Server, or directly to {{es}}.
Elastic offers several [{{edot}}](opentelemetry://reference/index.md) distributions. Each is a customized version of an OpenTelemetry language SDK and the OpenTelemetry Collector, ready to send data to the [{{motlp}}](opentelemetry://reference/motlp.md), {{apm-server-or-mis}}, or directly to {{es}}. **The destination you choose affects the format your data is stored in.** For details, refer to [How your ingest path determines the data format](opentelemetry://reference/compatibility/data-streams.md#ingest-path-data-format).

Check failure on line 31 in solutions/observability/apm/opentelemetry/index.md

View workflow job for this annotation

GitHub Actions / build / build

'reference/compatibility/data-streams.md' has no anchor named: '#ingest-path-data-format'.

:::{include} /solutions/_snippets/edot-reference-arch.md
:::
Expand All @@ -45,13 +45,14 @@

### Collect, process, and export data

* **OTel Collector in {{agent}}** {applies_to}`stack: ga 9.2+`: The OTel Collector runs embedded inside {{agent}}, sharing a single `elastic-agent.yml` configuration file. No separate Collector installation is needed. Refer to [{{agent}} as an OpenTelemetry Collector](/reference/fleet/elastic-agent-as-otel-collector.md) for more information.
* **Standalone {{agent}}**: Run {{agent}} independently as its own process. Refer to [{{agent}}](elastic-agent://reference/edot-collector/index.md) for more information.
* **OTel Collector in {{agent}}** {applies_to}`stack: ga 9.2+`: The OTel Collector runs embedded inside {{agent}}, sharing a single `elastic-agent.yml` configuration file. No separate Collector installation is needed. Data is stored in **OTel-native format** by default. Refer to [{{agent}} as an OpenTelemetry Collector](/reference/fleet/elastic-agent-as-otel-collector.md) for more information.
* **Standalone {{agent}}**: Run {{agent}} independently as its own process. Data is stored in **OTel-native format** by default. Refer to [{{agent}}](elastic-agent://reference/edot-collector/index.md) for more information.
* **Upstream `otelcol-contrib` Collector**: Use the community-built Collector to forward data to {{agent}} or directly to {{apm-server-or-mis}} using OTLP. Useful for a vendor-neutral pipeline or fanning out to multiple observability backends, but it's community-supported only. Refer to [Contrib OpenTelemetry Collectors and language SDKs](/solutions/observability/apm/opentelemetry/upstream-opentelemetry-collectors-language-sdks.md) for more information.

### Send data directly

* **Managed OTLP endpoint** ({{serverless-short}} and {{ech}}): Send OpenTelemetry data directly to the [Managed OTLP endpoint](opentelemetry://reference/motlp.md) without managing your own Collector.
* **{{motlp}}** ({{serverless-short}} and {{ech}}): Send OpenTelemetry data directly to the [{{motlp}}](opentelemetry://reference/motlp.md) without managing your own Collector. Data is stored in **OTel-native format**.
* **{{apm-server-or-mis}} OTLP intake** (legacy): Send OTLP data directly to {{apm-server-or-mis}} using its built-in OTLP endpoint. Data is **translated to ECS format** before storage. This path is not recommended for new users, and EDOT SDKs are not supported here. For details on the format difference, refer to [How your ingest path determines the data format](opentelemetry://reference/compatibility/data-streams.md#ingest-path-data-format).

Check failure on line 55 in solutions/observability/apm/opentelemetry/index.md

View workflow job for this annotation

GitHub Actions / build / build

'reference/compatibility/data-streams.md' has no anchor named: '#ingest-path-data-format'.

## Why use the Elastic Distributions of OpenTelemetry?

Expand Down
13 changes: 13 additions & 0 deletions solutions/observability/apm/opentelemetry/limitations.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,19 @@

# OpenTelemetry limitations in Elastic APM [apm-open-telemetry-known-limitations]

## Data format and compatibility with the `.apm` OTLP intake [apm-open-telemetry-data-format]

When sending OTLP data to {{apm-server-or-mis}} using its built-in OTLP endpoint (the `.apm` path), your data is **translated to ECS format** before storage. This is different from the {{motlp}} and {{agent}} in OTel mode, which store data in OTel-native format.

Consequences of the ECS translation:

* **Custom attributes**: Unmapped span and transaction attributes are stored under `labels.*` (strings) or `numeric_labels.*` (numbers), with dots replaced by underscores. For example, `order.id` becomes `labels.order_id`. On OTel-native paths, they are stored under `attributes.order.id`, with dots preserved.
* **Dashboards and content packs**: OpenTelemetry content packs, which provide OTel-compatible dashboards, are not applicable to ECS-translated data. Use prebuilt ECS-based integrations and dashboards, or the classic {{product.apm}} UI.
* **Ingest pipelines**: ECS-translated data passes through {{es}} ingest pipelines in the usual way. OTel-native data bypasses ingest pipelines.
* **EDOT SDK support**: EDOT SDKs are not supported with the {{apm-server-or-mis}} OTLP intake. Use the {{motlp}} or {{agent}} in OTel mode instead.

For a detailed comparison and guidance on choosing your ingest path, refer to [How your ingest path determines the data format](opentelemetry://reference/compatibility/data-streams.md#ingest-path-data-format).

Check failure on line 28 in solutions/observability/apm/opentelemetry/limitations.md

View workflow job for this annotation

GitHub Actions / build / build

'reference/compatibility/data-streams.md' has no anchor named: '#ingest-path-data-format'.

## OpenTelemetry traces [apm-open-telemetry-traces-limitations]

* Traces of applications using `messaging` semantics might be wrongly displayed as `transactions` in the Applications UI, while they should be considered `spans` (see issue [#7001](https://github.com/elastic/apm-server/issues/7001)).
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
The {{motlp}} stores OpenTelemetry data **without schema translation**, preserving OpenTelemetry semantic conventions and resource attributes in OTel-native format. Custom attributes are stored under `attributes.*` with dots preserved. It supports ingesting OTLP logs, metrics, and traces.

The {{apm-server-or-mis}} OTLP intake (the `.apm` endpoint) is a legacy path that **translates OTLP data to ECS format** before storage. Custom attributes land in `labels.*` with dots replaced by underscores. This path is not recommended for new users, and EDOT SDKs are not supported here.

For a full comparison of ingest paths and their data format implications, refer to [How your ingest path determines the data format](opentelemetry://reference/compatibility/data-streams.md#ingest-path-data-format).

Check failure on line 5 in solutions/observability/get-started/opentelemetry/_snippets/motlp-vs-apm-endpoint.md

View workflow job for this annotation

GitHub Actions / build / build

'reference/compatibility/data-streams.md' has no anchor named: '#ingest-path-data-format'.

Check failure on line 5 in solutions/observability/get-started/opentelemetry/_snippets/motlp-vs-apm-endpoint.md

View workflow job for this annotation

GitHub Actions / build / build

'reference/compatibility/data-streams.md' has no anchor named: '#ingest-path-data-format'.

Check failure on line 5 in solutions/observability/get-started/opentelemetry/_snippets/motlp-vs-apm-endpoint.md

View workflow job for this annotation

GitHub Actions / build / build

'reference/compatibility/data-streams.md' has no anchor named: '#ingest-path-data-format'.

Check failure on line 5 in solutions/observability/get-started/opentelemetry/_snippets/motlp-vs-apm-endpoint.md

View workflow job for this annotation

GitHub Actions / build / build

'reference/compatibility/data-streams.md' has no anchor named: '#ingest-path-data-format'.

Check failure on line 5 in solutions/observability/get-started/opentelemetry/_snippets/motlp-vs-apm-endpoint.md

View workflow job for this annotation

GitHub Actions / build / build

'reference/compatibility/data-streams.md' has no anchor named: '#ingest-path-data-format'.

Check failure on line 5 in solutions/observability/get-started/opentelemetry/_snippets/motlp-vs-apm-endpoint.md

View workflow job for this annotation

GitHub Actions / build / build

'reference/compatibility/data-streams.md' has no anchor named: '#ingest-path-data-format'.
Original file line number Diff line number Diff line change
Expand Up @@ -45,9 +45,10 @@
- [Docker on serverless](docker.md)
- [Hosts or VMs on serverless](hosts_vms.md)

## Differences with Elastic APM Endpoint
## Differences from the Elastic APM endpoint

The {{motlp}} ensures that OpenTelemetry data is stored without any schema translation, preserving both OpenTelemetry semantic conventions and resource attributes. It supports ingesting OTLP logs, metrics, and traces in a unified manner, ensuring consistent treatment across all telemetry data.
:::{include} /solutions/observability/get-started/opentelemetry/_snippets/motlp-vs-apm-endpoint.md

Check warning on line 50 in solutions/observability/get-started/opentelemetry/quickstart/serverless/index.md

View workflow job for this annotation

GitHub Actions / build / vale

Elastic.Latinisms: Latin terms and abbreviations are a common source of confusion. Use 'versus' instead of 'vs'.
:::

## Provide feedback

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -242,9 +242,10 @@

:::::::

## Differences from the Elastic APM Endpoint
## Differences from the Elastic APM endpoint

The Elastic Cloud Managed OTLP Endpoint ensures that OpenTelemetry data is stored without any schema translation, preserving both OpenTelemetry semantic conventions and resource attributes. It supports ingesting OTLP logs, metrics, and traces in a unified manner, ensuring consistent treatment across all telemetry data.
:::{include} /solutions/observability/get-started/opentelemetry/_snippets/motlp-vs-apm-endpoint.md

Check warning on line 247 in solutions/observability/get-started/quickstart-elastic-cloud-otel-endpoint.md

View workflow job for this annotation

GitHub Actions / build / vale

Elastic.Latinisms: Latin terms and abbreviations are a common source of confusion. Use 'versus' instead of 'vs'.
:::

## Troubleshooting

Expand Down
Loading