Enable doc value skippers on primary and secondary sort fields in favor of indexed data structures

[martijnvg]

Currently there is a feature flag that controls whether doc value skippers are enabled on host.name and @timestamp fields in case of logsdb index mode and _tsid and @timestamp fields in case of times series index mode.

Initial benchmark results showed that sometimes filtering on @timestamp field became significantly slower (upto 3 times). This is because the bkd tree (points) was swapped for doc values skipper and the default query logic doesn't always perform well if timestamp field is secondary index sort field. The feature flag has been temporarily disabled.

This issue is about figuring out how to improve filtering by timestamp when doc value skippers are enabled. The performance of filtering by timestamp will likely not be the same as when the bkd tree is enabled. There maybe ways to mitigate some of the performance drop.

Note that replacing the bkd tree with doc value skippers is a trade off. By not storing the bkd tree, we reduce storage and indecing footprint in favor for slower timestamp filtering.

Tasks:

• Investigate special query logic for filtering by @timestamp for logsdb and time_series index modes. (when @timestamp field is always the secondary index sort). (Re-enable use_doc_values_skipper and add specialized lucene query for @timestamp field filtering. #127260)
• Enable feature flag by default.
• Remove feature flag.

[elasticsearchmachine]

Pinging @elastic/es-storage-engine (Team:StorageEngine)