Skip to content

Adding EcsNamespaceProcessor #125699

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 45 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
30b0ed3
Adding EcsNamespacingProcessor
eyalkoren Mar 26, 2025
b96cc0c
Adding module-info
eyalkoren Mar 27, 2025
469df61
Merge remote-tracking branch 'upstream/main' into ECS-namespacing-pro…
eyalkoren Mar 27, 2025
2bd819f
Exposing and testing the processor
eyalkoren Mar 27, 2025
1c2a670
Add test and some algorithm fixes
eyalkoren Mar 27, 2025
904c19c
Making scope non-mandatory
eyalkoren Mar 27, 2025
bd75b06
Minimize dependencies
eyalkoren Mar 27, 2025
50f3c4d
Extending REST tests
eyalkoren Mar 27, 2025
cb0dcee
Merge remote-tracking branch 'upstream/main' into ECS-namespacing-pro…
eyalkoren Mar 27, 2025
f68cf93
Update docs/changelog/125699.yaml
eyalkoren Mar 27, 2025
9dbe94b
Merge remote-tracking branch 'upstream/main' into ECS-namespacing-pro…
eyalkoren Mar 30, 2025
79bf683
instanceOf with pattern matching
eyalkoren Mar 30, 2025
dbc4d4a
instanceOf with pattern matching
eyalkoren Mar 30, 2025
d160fe6
revert constants usage
eyalkoren Mar 30, 2025
dfe33fc
Merge remote-tracking branch 'eyalkoren/ECS-namespacing-processor' in…
eyalkoren Mar 30, 2025
2b77e3c
Complete review change proposals
eyalkoren Mar 30, 2025
4773cf6
fix typo in test name
eyalkoren Mar 30, 2025
054a76f
[CI] Auto commit changes from spotless
elasticsearchmachine Mar 30, 2025
4afacd0
Applying review suggestions
eyalkoren Mar 30, 2025
d67fa2c
Merge remote-tracking branch 'eyalkoren/ECS-namespacing-processor' in…
eyalkoren Mar 30, 2025
bef3643
Silence a warning from Intellij
joegallo Apr 3, 2025
49d439f
Rename this variable
joegallo Apr 3, 2025
e15984b
Fix some typos and reflow a comment
joegallo Apr 3, 2025
e09fa14
Use ofEntries for increased clarity
joegallo Apr 3, 2025
b71fa3f
Save a rehash and some traversals
joegallo Apr 3, 2025
e8338f8
Merge branch 'main' into ECS-namespacing-processor
joegallo Apr 3, 2025
98880ca
This can be static
joegallo Apr 9, 2025
d37587d
Rely on mutability for these tests
joegallo Apr 9, 2025
21f2f9e
Rename some variables
joegallo Apr 9, 2025
b0037c3
Drop the top-level warnings suppression
joegallo Apr 9, 2025
96ff183
Merge branch 'main' into ECS-namespacing-processor
joegallo Apr 9, 2025
65f7ea2
Rewrite this test
joegallo Apr 9, 2025
3639c77
Merge branch 'main' into ECS-namespacing-processor
joegallo Apr 25, 2025
f9421ac
Drop yaml-rest-compat-test
joegallo Apr 25, 2025
b2dd61d
Refactor ECS Namespacing to Normalize to OTel
eyalkoren Apr 28, 2025
c66663b
Apply review comments
eyalkoren Apr 28, 2025
7d536a7
Adding accurate resource attributes handling
eyalkoren May 5, 2025
9493a73
Merge remote-tracking branch 'upstream/main' into ECS-namespacing-pro…
eyalkoren May 5, 2025
bb538a6
Suppress warning for forbidden usage of System.out in tests
eyalkoren May 5, 2025
e7c1f9d
Eliminating some more forbidden APIs
eyalkoren May 5, 2025
605bdc9
[CI] Auto commit changes from spotless
elasticsearchmachine May 5, 2025
e65c0ef
Reverting shameful refactoring errors
eyalkoren May 6, 2025
916eaca
Merge remote-tracking branch 'eyalkoren/ECS-namespacing-processor' in…
eyalkoren May 6, 2025
07e78a2
Merge remote-tracking branch 'upstream/main' into ECS-namespacing-pro…
eyalkoren May 6, 2025
5c34f94
Disabling ResourceAttributesTests
eyalkoren May 6, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions docs/changelog/125699.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
pr: 125699
summary: Adding `NormalizeToOTelProcessor`
area: Ingest Node
type: feature
issues: []
21 changes: 21 additions & 0 deletions modules/ingest-otel/build.gradle
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the "Elastic License
* 2.0", the "GNU Affero General Public License v3.0 only", and the "Server Side
* Public License v 1"; you may not use this file except in compliance with, at
* your election, the "Elastic License 2.0", the "GNU Affero General Public
* License v3.0 only", or the "Server Side Public License, v 1".
*/

apply plugin: 'elasticsearch.internal-yaml-rest-test'

esplugin {
description = 'Ingest processor that normalizes ECS documents to OpenTelemetry-compatible namespaces'
classname ='org.elasticsearch.ingest.otel.NormalizeToOTelPlugin'
}

restResources {
restApi {
include '_common', 'indices', 'index', 'cluster', 'nodes', 'get', 'ingest'
}
}
13 changes: 13 additions & 0 deletions modules/ingest-otel/src/main/java/module-info.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the "Elastic License
* 2.0", the "GNU Affero General Public License v3.0 only", and the "Server Side
* Public License v 1"; you may not use this file except in compliance with, at
* your election, the "Elastic License 2.0", the "GNU Affero General Public
* License v3.0 only", or the "Server Side Public License, v 1".
*/

module org.elasticsearch.ingest.otel {
requires org.elasticsearch.base;
requires org.elasticsearch.server;
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the "Elastic License
* 2.0", the "GNU Affero General Public License v3.0 only", and the "Server Side
* Public License v 1"; you may not use this file except in compliance with, at
* your election, the "Elastic License 2.0", the "GNU Affero General Public
* License v3.0 only", or the "Server Side Public License, v 1".
*/

package org.elasticsearch.ingest.otel;

import java.util.Set;

final class EcsOTelResourceAttributes {

/**
* The set of ECS (Elastic Common Schema) field names that are mapped to OpenTelemetry resource attributes,
* as defined by the OpenTelemetry Semantic Conventions.
* The list is produced by the {@code ResourceAttributesTests#testAttributesSetUpToDate} test.
*
* @see <a href="https://github.com/open-telemetry/semantic-conventions">OpenTelemetry Semantic Conventions</a>
*/
static final Set<String> LATEST = Set.of(
"agent.type",
"agent.build.original",
"agent.name",
"agent.id",
"agent.ephemeral_id",
"agent.version",
"container.image.tag",
"device.model.identifier",
"container.image.hash.all",
"service.node.name",
"process.pid",
"device.id",
"host.mac",
"host.type",
"container.id",
"cloud.availability_zone",
"host.ip",
"container.name",
"container.image.name",
"device.model.name",
"host.name",
"host.id",
"process.executable",
"user_agent.original",
"service.environment",
"cloud.region",
"service.name",
"faas.name",
"device.manufacturer",
"process.args",
"host.architecture",
"cloud.provider",
"container.runtime",
"service.version",
"cloud.service.name",
"cloud.account.id",
"process.command_line",
"faas.version"
);

private EcsOTelResourceAttributes() {}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the "Elastic License
* 2.0", the "GNU Affero General Public License v3.0 only", and the "Server Side
* Public License v 1"; you may not use this file except in compliance with, at
* your election, the "Elastic License 2.0", the "GNU Affero General Public
* License v3.0 only", or the "Server Side Public License, v 1".
*/

package org.elasticsearch.ingest.otel;

import org.elasticsearch.ingest.Processor;
import org.elasticsearch.plugins.IngestPlugin;
import org.elasticsearch.plugins.Plugin;

import java.util.Map;

public class NormalizeToOTelPlugin extends Plugin implements IngestPlugin {

@Override
public Map<String, Processor.Factory> getProcessors(Processor.Parameters parameters) {
return Map.of(NormalizeToOTelProcessor.TYPE, new NormalizeToOTelProcessor.Factory());
}
}
Loading
Loading