Skip to content

feat: add new checks and floating api #24

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
May 11, 2025

Conversation

flitzpiepe93
Copy link
Contributor

🚀 Add Fluent API and New Row-Level String Checks

This PR introduces major improvements to the sparkdq framework, including a fluent-style CheckSet API and a suite of reusable, declarative string validation checks.


✨ What's New

✅ Fluent API for CheckSet

Checks can now be added using a chainable (floating) style:

check_set = (
    CheckSet()
    .add_check(...)
    .add_check(...)
)

🧪 New Row-Level Checks

  • ColumnLessThanCheck
    Ensures that values in one column are strictly less than values in another (supports timestamps, dates, numerics)

  • StringMinLengthCheck
    Validates that strings meet a minimum length (inclusive=True|False)

  • StringMaxLengthCheck
    Ensures that string values do not exceed a maximum length

  • StringLengthBetweenCheck
    Checks if string length lies within a specified [min, max] range

  • RegexMatchCheck
    Flags values that do not match a given regex pattern
    Supports ignore_case and optional failure on nulls


🔧 Other Improvements

  • __init__.py now includes a __version__ for programmatic version access
  • Simplified logic in StringMinLengthCheck for readability and consistency

📚 Docs & Tests

  • Full unit test coverage for all new checks
  • Sphinx-ready documentation snippets (Python & YAML)
  • Includes descriptions, config options, and use case examples

✅ Checklist

  • All new checks registered and documented
  • Fully tested with null handling and edge cases
  • Complies with Spark-native logic and framework architecture

@flitzpiepe93 flitzpiepe93 self-assigned this May 11, 2025
@flitzpiepe93 flitzpiepe93 added enhancement New feature or request chore Maintenance tasks like updates to docs, configs, workflows, or dependencies — no impact on core func labels May 11, 2025
@flitzpiepe93 flitzpiepe93 changed the title Feature/add new checks and floating api feat: add new checks and floating api May 11, 2025
Copy link

codecov bot commented May 11, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

📢 Thoughts on this report? Let us know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
chore Maintenance tasks like updates to docs, configs, workflows, or dependencies — no impact on core func enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant