Skip to content

[RFC] Remove/hook unused metrics #2901 #5182

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 17 commits into
base: main
Choose a base branch
from

Conversation

mdhaduk
Copy link

@mdhaduk mdhaduk commented May 5, 2025

Opening this as a Draft PR

Changes

Removed unused metrics; used this script to generally identify which metrics might not be used, then manually removed them

To identify unused metrics, I used the following convention

For SharedIncMetric:

  • Not incremented/added to
  • Not stored
  • Not fetched

For LatencyAggregateMetrics:

  • Not calling record_latency_metrics

Metrics Removed:

  • sync_response_fails
  • sync_vmm_send_timeout_count
  • deprecated_cmd_line_api_calls
  • log_fails
  • device_events
  • rx_partial_writes
  • tx_partial_reads

Reason

Resolving issue #2901 - Remove/hook unused metrics

License Acceptance

By submitting this pull request, I confirm that my contribution is made under
the terms of the Apache 2.0 license. For more information on following Developer
Certificate of Origin and signing off your commits, please check
CONTRIBUTING.md.

PR Checklist

  • I have read and understand CONTRIBUTING.md.
  • I have run tools/devtool checkstyle to verify that the PR passes the
    automated style checks.
  • I have described what is done in these changes, why they are needed, and
    how they are solving the problem in a clear and encompassing way.
  • I have updated any relevant documentation (both in code and in the docs)
    in the PR.
  • I have mentioned all user-facing changes in CHANGELOG.md.
  • If a specific issue led to this PR, this PR closes the issue.
  • When making API changes, I have followed the
    Runbook for Firecracker API changes.
  • I have tested all new and changed functionalities in unit tests and/or
    integration tests.
  • I have linked an issue to every new TODO.

  • This functionality cannot be added in rust-vmm.

block throughput metrics on m8g.metal instances for test scenarios using
the async fio engine and more than 1 vcpu are volatile, so exclude them
from A/B-testing.

Suggested-by: Riccardo Mancini <[email protected]>
Signed-off-by: Patrick Roy <[email protected]>
Signed-off-by: LDagnachew <[email protected]>
@mdhaduk mdhaduk force-pushed the rmv_unused_metrics branch from 9990ff7 to 2cc23e7 Compare May 5, 2025 07:39
@roypat
Copy link
Contributor

roypat commented May 6, 2025

This is great, thanks for tackling this! I think we can probably even include your script in this PR (somewhere under tools) and then run it as part of our regular test suite, so we don't again reintroduce unneeded metrics later on :)

Btw, I think you can catch a few more unused ones with a tiny tweak to your script: Add metrics.rs files to the check that excludes specific use-sites. For example, the rx_partial_writes metric in net/metrics.rs is unused, but there's a .add call due to metrics aggregation inside the metrics files itself that makes your script believe it is in use.

LDagnachew and others added 13 commits May 6, 2025 20:23
script finds metric files, extracts fields, and reports
validity of each metrics

Signed-off-by: Milan Dhaduk <[email protected]>

Signed-off-by: LDagnachew <[email protected]>
- metric is not used

Signed-off-by: Milan Dhaduk <[email protected]>

Signed-off-by: LDagnachew <[email protected]>
- metric is not used

Signed-off-by: Milan Dhaduk <[email protected]>

Signed-off-by: LDagnachew <[email protected]>
- metric is not used

Signed-off-by: Milan Dhaduk <[email protected]>

Signed-off-by: LDagnachew <[email protected]>
- metric is not used

Signed-off-by: Milan Dhaduk <[email protected]>

Signed-off-by: LDagnachew <[email protected]>
- metric is not used

Signed-off-by: Milan Dhaduk <[email protected]>

Signed-off-by: LDagnachew <[email protected]>
Bumps the firecracker group with 9 updates:

| Package | From | To |
| --- | --- | --- |
| [aws-lc-fips-sys](https://github.com/aws/aws-lc-rs) | `0.13.5` | `0.13.6` |
| [cc](https://github.com/rust-lang/cc-rs) | `1.2.20` | `1.2.21` |
| [hashbrown](https://github.com/rust-lang/hashbrown) | `0.15.2` | `0.15.3` |
| [jiff](https://github.com/BurntSushi/jiff) | `0.2.10` | `0.2.12` |
| [jiff-static](https://github.com/BurntSushi/jiff) | `0.2.10` | `0.2.12` |
| [toml](https://github.com/toml-rs/toml) | `0.8.21` | `0.8.22` |
| [toml_edit](https://github.com/toml-rs/toml) | `0.22.25` | `0.22.26` |
| [toml_write](https://github.com/toml-rs/toml) | `0.1.0` | `0.1.1` |
| [winnow](https://github.com/winnow-rs/winnow) | `0.7.7` | `0.7.9` |


Updates `aws-lc-fips-sys` from 0.13.5 to 0.13.6
- [Release notes](https://github.com/aws/aws-lc-rs/releases)
- [Commits](aws/aws-lc-rs@aws-lc-fips-sys/v0.13.5...aws-lc-fips-sys/v0.13.6)

Updates `cc` from 1.2.20 to 1.2.21
- [Release notes](https://github.com/rust-lang/cc-rs/releases)
- [Changelog](https://github.com/rust-lang/cc-rs/blob/main/CHANGELOG.md)
- [Commits](rust-lang/cc-rs@cc-v1.2.20...cc-v1.2.21)

Updates `hashbrown` from 0.15.2 to 0.15.3
- [Release notes](https://github.com/rust-lang/hashbrown/releases)
- [Changelog](https://github.com/rust-lang/hashbrown/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/hashbrown/commits/v0.15.3)

Updates `jiff` from 0.2.10 to 0.2.12
- [Release notes](https://github.com/BurntSushi/jiff/releases)
- [Changelog](https://github.com/BurntSushi/jiff/blob/master/CHANGELOG.md)
- [Commits](BurntSushi/jiff@jiff-static-0.2.10...jiff-static-0.2.12)

Updates `jiff-static` from 0.2.10 to 0.2.12
- [Release notes](https://github.com/BurntSushi/jiff/releases)
- [Changelog](https://github.com/BurntSushi/jiff/blob/master/CHANGELOG.md)
- [Commits](BurntSushi/jiff@jiff-static-0.2.10...jiff-static-0.2.12)

Updates `toml` from 0.8.21 to 0.8.22
- [Commits](toml-rs/toml@toml-v0.8.21...toml-v0.8.22)

Updates `toml_edit` from 0.22.25 to 0.22.26
- [Commits](toml-rs/toml@v0.22.25...v0.22.26)

Updates `toml_write` from 0.1.0 to 0.1.1
- [Commits](toml-rs/toml@toml_write-v0.1.0...toml_write-v0.1.1)

Updates `winnow` from 0.7.7 to 0.7.9
- [Changelog](https://github.com/winnow-rs/winnow/blob/main/CHANGELOG.md)
- [Commits](winnow-rs/winnow@v0.7.7...v0.7.9)

---
updated-dependencies:
- dependency-name: aws-lc-fips-sys
  dependency-version: 0.13.6
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: firecracker
- dependency-name: cc
  dependency-version: 1.2.21
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: firecracker
- dependency-name: hashbrown
  dependency-version: 0.15.3
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: firecracker
- dependency-name: jiff
  dependency-version: 0.2.12
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: firecracker
- dependency-name: jiff-static
  dependency-version: 0.2.12
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: firecracker
- dependency-name: toml
  dependency-version: 0.8.22
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: firecracker
- dependency-name: toml_edit
  dependency-version: 0.22.26
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: firecracker
- dependency-name: toml_write
  dependency-version: 0.1.1
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: firecracker
- dependency-name: winnow
  dependency-version: 0.7.9
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: firecracker
...

Signed-off-by: dependabot[bot] <[email protected]>
When starting firecracker without the API server and configuring it
through a config files, specifying a custom cpu template requires
writing this cpu template into a separate json file, and then passing
the path to this separate json file in the config json file. This is a
bit silly, since everything is just json. So additionally allow just
directly specifying the custom cpu template inside the config file.

Signed-off-by: Patrick Roy <[email protected]>
Creates a new [Unreleased] section and move changes to be included in
v1.12.0 to [1.12.0] section.

Signed-off-by: Takahiro Itazuri <[email protected]>
Adds v1.12 to the release status table and marks v1.10 as unsupported.

Signed-off-by: Takahiro Itazuri <[email protected]>
Intel Sapphire Rapids and ARM Graviton4 are not supported.

Signed-off-by: Takahiro Itazuri <[email protected]>
Runs `cargo update`.

Signed-off-by: Takahiro Itazuri <[email protected]>
We're going to release v1.12.0 tomorrow. Bumps the version to 1.13.0-dev
in main branch.

Signed-off-by: Takahiro Itazuri <[email protected]>
@mdhaduk mdhaduk force-pushed the rmv_unused_metrics branch from 78eac72 to d8bfb5a Compare May 6, 2025 20:28
LDagnachew added 2 commits May 6, 2025 20:42
- metric is not used

Signed-off-by: Milan Dhaduk <[email protected]>
Signed-off-by: LDagnachew <[email protected]>
- metric is not used

Signed-off-by: Milan Dhaduk <[email protected]>
Signed-off-by: LDagnachew <[email protected]>
@mdhaduk mdhaduk marked this pull request as ready for review May 6, 2025 21:06
@mdhaduk
Copy link
Author

mdhaduk commented May 6, 2025

Great! Thank you for all the feedback, and I'm super stoked to potentially have my scripted integrated within the test suite!!

I went ahead and made the change to the script as you recommended which you can see above or here. Additionally, the following metrics may or may not be removed:

  • metrics_count (SharedIncMetric)
  • metrics_fails (SharedIncMetric)
  • missed_metrics_count (SharedIncMetric)

I opted to ask you since removing them would require reworking src/metrics.rs and requests/metrics.rs; if you deem that is okay I will go ahead and make those changes as well!

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants