Discussion on MarkdownHeaderLevelInferrer component

[davidsbatista]

This is the discussion board for MarkdownHeaderLevelInferrer

We introduced the MarkdownHeaderLevelInferrer component, which infers and rewrites header levels in Markdown text to normalize hierarchy.

First header → Always becomes level 1 (#)
Subsequent headers → Level increases if no content between headers, stays same if content exists
Maximum level → Capped at 6 (######)

from haystack import Document
from haystack_experimental.components.preprocessors import MarkdownHeaderLevelInferrer

# Create a document with uniform header levels
text = "## Title\nSome content\n## Section\nMore content\n## Subsection\nFinal content"
doc = Document(content=text)

# Initialize the inferrer and process the document
inferrer = MarkdownHeaderLevelInferrer()
result = inferrer.run([doc])

# The headers are now normalized with proper hierarchy
print(result["documents"][0].content)
> # Title\nSome content\n## Section\nMore content\n### Subsection\nFinal content