feat(ai): AI cost calculation #4840
Merged
+114
−103
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
vgrozdanic
commented
Jun 24, 2025
Comment on lines
+2206
to
+2210
| "ai_prompt_tokens_used": { | ||
| "value": 1000 | ||
| }, | ||
| "ai_completion_tokens_used": { | ||
| "value": 2000 |
There was a problem hiding this comment.
these are legacy fields that are still being sent, until we do a cut-off and fully transition to a new agent monitoring we need to support them
vgrozdanic
commented
Jun 24, 2025
Comment on lines
-26
to
-36
| // If we only have total tokens and no breakdown, use input cost for all tokens | ||
| // (assuming it's more common to have input cost defined in V1 configs) | ||
| if prompt_tokens_used.is_none() && completion_tokens_used.is_none() { | ||
| if let Some(total_tokens) = total_tokens_used { | ||
| if cost_per_token.input_per_token > 0.0 { | ||
| result += cost_per_token.input_per_token * total_tokens; | ||
| } else if cost_per_token.output_per_token > 0.0 { | ||
| result += cost_per_token.output_per_token * total_tokens; | ||
| } | ||
| } | ||
| } |
There was a problem hiding this comment.
It doesn't make sense to base our calculation on total_tokens used anymore, so this can now be removed
vgrozdanic
commented
Jun 24, 2025
| .and_then(|val| val.as_str()) | ||
| // xxx (vgrozdanic): temporal fallback to legacy field, until we fix | ||
| // sentry conventions and standardize what SDKs send | ||
| .or_else(|| data.ai_model_id.value().and_then(|val| val.as_str())) |
There was a problem hiding this comment.
Currently we still want to use ai_model_id as a fallback, but we first need to clean up our sentry-conventions (we have made a mistake there), and we need to clean up the data that is being sent from our SDKs (currently both of these fields are sent, and one has priority)
loewenheim
reviewed
Jun 24, 2025
vgrozdanic
force-pushed
the
vg/ai-cost-calculation
branch
2 times, most recently
from
bbbfd61 to
5c10171
Compare
vgrozdanic
force-pushed
the
vg/ai-cost-calculation
branch
from
5c10171 to
3d7289a
Compare
loewenheim
approved these changes
Jun 24, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes problems with old way of calculating costs where we only included input and output tokens into cost calculation, and refactors the whole
calculate_ai_model_costto be more robust, so that there is no need to change multiple functions when we want to include new field into cost calculation.Closes TET-648: Refactor cost calculation