Skip to content

Proper tracking of conditional "partial" lines #1962

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
kingbuzzman opened this issue May 5, 2025 · 0 comments
Open

Proper tracking of conditional "partial" lines #1962

kingbuzzman opened this issue May 5, 2025 · 0 comments
Labels
enhancement New feature or request

Comments

@kingbuzzman
Copy link

Is your feature request related to a problem? Please describe.

Currently, when using code coverage with complex boolean expressions in conditional statements (if, elif), the reporting lacks sufficient granularity.

For example, consider a condition like:

if a or b or c or d:
    # code block A
elif e or f:
    # code block B
else:
    # code block C

Standard branch coverage often marks the entire line if a or b or c or d: as "covered" if the condition evaluates to True, regardless of which specific sub-expression (a, b, c, or d) caused it to be true due to short-circuiting. This means we might execute code block A because b was True, but never actually test the cases where a, c, or d would have individually caused the condition to be True.

This makes it difficult to ensure that all logical paths within a complex boolean expression have been exercised by the test suite, particularly in large or legacy codebases.

Describe the solution you'd like

I would like the coverage tool to provide more detailed insight into the evaluation of complex boolean expressions within conditionals. Specifically, it would be beneficial to know which individual sub-expressions within a compound condition were evaluated and which ones were decisive in determining the outcome (True or False) of the condition.

Ideally, the coverage report could indicate if a boolean expression was only "partially" covered, meaning not all potential evaluation paths or short-circuiting scenarios within that specific condition were tested.

Possible ways to report this could include:

  1. Annotated Source View: Highlighting or annotating parts of the boolean expression that were evaluated or determined the outcome during a test run.

    # Example annotation showing 'b' was the decisive factor:
    if a # F, evaluated # or b # T, decisive # or c # Not evaluated # or d # Not evaluated #:
        ...

    Or perhaps marking the line as "partial" if not all sub-expressions were evaluated across all test runs hitting that line.

  2. Detailed Report/Table: A separate report or section detailing complex conditions and which specific components within them were triggered by which test cases.

Let's consider the expression if a or (b and (c or d)):.

Test Case Section 1 (a) Evaluation Section 2 (c or d) Evaluation (if reached) Section 3 (b and (c or d)) Evaluation Overall Condition Outcome
test_a_is_true True Short-Circuited Short-Circuited True
test_a_false_b_true_c_true False True (because c is True) True (because b & Sec 2 are True) True
test_a_false_b_true_d_true False True (because d is True) True (because b & Sec 2 are True) True
test_a_false_b_true_c_d_false False False False (because Sec 2 is False) False
test_a_false_b_false False Short-Circuited False (because b is False) False

Please note that I'm only advocating for this to be shown on or since the current solution works amazing for and.

Describe alternatives you've considered

The primary alternative is to refactor the complex boolean expressions into simpler conditions, potentially breaking them down into multiple lines, intermediate variables, or helper functions.

However, this presents a catch-22 situation: refactoring risky legacy code requires high confidence in the existing test suite to ensure no behavior is changed unintentionally. Without detailed condition coverage information, it is difficult to know if the current tests are sufficient to cover all the nuances of the existing complex logic before attempting to refactor it. This feature would provide the necessary insight to confidently identify testing gaps and improve coverage prior to or alongside refactoring efforts.


@kingbuzzman kingbuzzman added the enhancement New feature or request label May 5, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant