Skip to content

EE Refactor #5844

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 18 commits into from
Jun 2, 2025
Merged

EE Refactor #5844

merged 18 commits into from
Jun 2, 2025

Conversation

diegoimbert
Copy link
Contributor

@diegoimbert diegoimbert commented May 30, 2025

  • Refactor _ee files into _oss so that we can gitignore all _ee files and avoid git annoyances with tracked symlinks

Important

Refactor codebase to separate enterprise and open-source features by renaming files, updating imports, and modifying feature flags.

  • Refactor EE to OSS:
    • Rename *_ee.rs files to *_oss.rs in backend/src, windmill-api/src, windmill-audit/src, windmill-autoscaling/src, windmill-common/src, windmill-git-sync/src, windmill-indexer/src, windmill-queue/src, and windmill-worker/src.
    • Update imports to use *_oss instead of *_ee in main.rs, monitor.rs, ai.rs, apps.rs, auth.rs, capture.rs, configs.rs, db.rs, flows.rs, folders.rs, groups.rs, http_triggers.rs, jobs.rs, lib.rs, mqtt_triggers.rs, postgres_triggers/handler.rs, raw_apps.rs, resources.rs, schedule.rs, scripts.rs, settings.rs, variables.rs, websocket_triggers.rs, workspaces.rs, workspaces_export.rs, workspaces_extra.rs, worker.rs, worker_flow.rs.
    • Add #[cfg(feature = "private")] and #[cfg(not(feature = "private"))] to conditionally compile EE and OSS code.
  • Scripts and Features:
    • Add all_features_oss.sh script to list all non-private features.
    • Update backend-check.yml and update_sqlx.sh to use all_features_oss.sh.
  • Miscellaneous:
    • Update .gitignore to ignore *ee.rs files.
    • Update Cargo.toml files to include private feature.

This description was created by Ellipsis for 7648f01. You can customize this summary. It will automatically update as commits are pushed.

Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important

Looks good to me! 👍

Reviewed everything up to 4a68ec8 in 2 minutes and 9 seconds. Click for details.
  • Reviewed 3855 lines of code in 105 files
  • Skipped 0 files when reviewing.
  • Skipped posting 22 draft comments. View those below.
  • Modify your settings and rules to customize what types of comments Ellipsis leaves. And don't forget to react with 👍 or 👎 to teach Ellipsis.
1. backend/windmill-worker/src/worker_flow.rs:74
  • Draft comment:
    The function update_flow_status_after_job_completion_internal is extremely long and deeply nested. Consider refactoring it into smaller, testable functions.
  • Reason this comment was not posted:
    Comment was not on a location in the diff, so it can't be submitted as a review comment.
2. backend/windmill-worker/src/worker_flow.rs:1732
  • Draft comment:
    In push_next_flow_job, SQL queries are executed inside a loop for each payload. Consider batching queries or reducing per-iteration overhead.
  • Reason this comment was not posted:
    Comment was not on a location in the diff, so it can't be submitted as a review comment.
3. backend/windmill-worker/src/worker_flow.rs:90
  • Draft comment:
    Variable names like 'rec', 'nrec', and 'ns' are too terse; using descriptive names would greatly improve code readability.
  • Reason this comment was not posted:
    Comment was not on a location in the diff, so it can't be submitted as a review comment.
4. backend/windmill-worker/src/worker_flow.rs:4000
  • Draft comment:
    There are several unwrap/expect calls (e.g. in get_script_info_for_hash and flow_to_payload). This risks panics on unexpected data. Consider propagating errors instead.
  • Reason this comment was not posted:
    Comment was not on a location in the diff, so it can't be submitted as a review comment.
5. backend/windmill-worker/src/worker_flow.rs:3170
  • Draft comment:
    The recursive function insert_iter_arg may lead to deep recursion with conflicting keys. Consider adding a recursion limit or iterative alternative.
  • Reason this comment was not posted:
    Comment was not on a location in the diff, so it can't be submitted as a review comment.
6. backend/windmill-worker/src/worker_flow.rs:60
  • Draft comment:
    Several complex types like Arc<Box> are used repeatedly. Consider defining type aliases to improve code clarity.
  • Reason this comment was not posted:
    Comment was not on a location in the diff, so it can't be submitted as a review comment.
7. backend/windmill-worker/src/worker_flow.rs:240
  • Draft comment:
    Multiple inline SQL query strings are intermingled with business logic. Extracting them into helper functions or constants could both simplify maintenance and improve readability.
  • Reason this comment was not posted:
    Comment was not on a location in the diff, so it can't be submitted as a review comment.
8. backend/windmill-worker/src/worker_flow.rs:320
  • Draft comment:
    The branching logic in compute_next_flow_transform is very dense. Adding inline comments or refactoring the match arms into separate functions would help.
  • Reason this comment was not posted:
    Comment was not on a location in the diff, so it can't be submitted as a review comment.
9. backend/windmill-worker/src/worker_flow.rs:1485
  • Draft comment:
    Repeated calls to get_transform_context and eval_timeout within loops may impact performance. Consider caching or optimizing these evaluations if possible.
  • Reason this comment was not posted:
    Comment was not on a location in the diff, so it can't be submitted as a review comment.
10. backend/windmill-worker/src/worker_flow.rs:1
  • Draft comment:
    Overall, this module is highly complex with deep nesting and many intertwined responsibilities. Consider adding additional inline documentation and splitting the module into smaller submodules for better maintainability.
  • Reason this comment was not posted:
    Comment was not on a location in the diff, so it can't be submitted as a review comment.
11. backend/src/monitor.rs:48
  • Draft comment:
    Typo detected: 'BUNFIG_INSTALL_SCOPES_SETTING' looks like it might be a mistake. Should this be 'BUNDLE_INSTALL_SCOPES_SETTING' or something similar?
  • Reason this comment was not posted:
    Comment was not on a location in the diff, so it can't be submitted as a review comment.
12. backend/windmill-api/Cargo.toml:17
  • Draft comment:
    I noticed a potential typographical error in the dependency specification for the enterprise_saml feature: it uses "dep:samael". Was "dep:saml" intended instead? Please double-check.
  • Reason this comment was not posted:
    Comment was not on a location in the diff, so it can't be submitted as a review comment.
13. backend/windmill-api/src/agent_workers_oss.rs:28
  • Draft comment:
    Typo alert: The function name 'workspaced_service' might be a typo. Did you mean 'workspace_service'?
  • Reason this comment was not posted:
    Comment was on unchanged code.
14. backend/windmill-api/src/git_sync_oss.rs:9
  • Draft comment:
    Typo found: The function name "workspaced_service" might be intended as "workspace_service". Please verify.
  • Reason this comment was not posted:
    Decided after close inspection that this draft comment was likely wrong and/or not actionable: usefulness confidence = 10% vs. threshold = 50% 1. This is a new file, so we can't compare to previous naming. 2. 'Workspaced' could be intentional - it's an adjective form meaning "having a workspace". 3. Without seeing where this function is used or what it's meant to represent, we can't be sure if this is actually a typo. 4. The comment is speculative and asks for verification, which violates our rules. The function name could indeed be a typo, and fixing it early could prevent confusion. The alternative name does sound more conventional. However, we don't have enough context to know if this is actually a typo, and the comment asks for verification rather than making a clear case for the change. Delete the comment. It's speculative, asks for verification, and we don't have strong evidence that the current name is wrong.
15. backend/windmill-api/src/indexer_oss.rs:9
  • Draft comment:
    Typographical error: Consider renaming workspaced_service to something like workspace_service if the extra 'd' is unintentional.
  • Reason this comment was not posted:
    Decided after close inspection that this draft comment was likely wrong and/or not actionable: usefulness confidence = 10% vs. threshold = 50% The term 'workspaced' could be intentional as it might mean 'related to a workspace' or 'within a workspace context'. Without more context about the codebase's naming conventions or the specific meaning of this term, suggesting a rename could be presumptuous. The code is being newly added, so the name was likely chosen deliberately. I might be missing established naming patterns in the codebase where 'workspace_service' is the standard form. The 'd' could genuinely be a typo that should be fixed for consistency. However, this is a new file being added, and without clear evidence that this is actually a typo (such as seeing other similar function names in the codebase), we should assume the author chose this name intentionally. Delete this comment as it's speculative and assumes the naming is unintentional without strong evidence. The function name could be deliberately chosen.
16. backend/windmill-api/src/job_helpers_oss.rs:45
  • Draft comment:
    Line 45: There's an extra attribute #[allow(dead_code)] in the LoadImagePreviewQuery struct which has no associated field. Please verify whether this line was left over inadvertently or if a field is missing.
  • Reason this comment was not posted:
    Comment was not on a location in the diff, so it can't be submitted as a review comment.
17. backend/windmill-api/src/job_helpers_oss.rs:61
  • Draft comment:
    Typo: The function name workspaced_service may be a mistake. Perhaps it was intended to be workspace_service?
  • Reason this comment was not posted:
    Comment was on unchanged code.
18. backend/windmill-api/src/kafka_triggers_oss.rs:17
  • Draft comment:
    Typo suggestion: The function name workspaced_service might be intended to be workspace_service. Please confirm if workspaced is correct or if it should be updated to workspace.
  • Reason this comment was not posted:
    Comment was on unchanged code.
19. backend/windmill-api/src/oauth2_oss.rs:52
  • Draft comment:
    Typo candidate: The function name "workspaced_service" might be a typographical error. Did you mean "workspace_service"?
  • Reason this comment was not posted:
    Comment was on unchanged code.
20. backend/windmill-api/src/oidc_oss.rs:22
  • Draft comment:
    Typo suggestion: The function name "workspaced_service" might be a typo. If intended to refer to a workspace-based service, consider renaming it to "workspace_service" for clarity.
  • Reason this comment was not posted:
    Comment was on unchanged code.
21. backend/windmill-api/src/sqs_triggers_oss.rs:15
  • Draft comment:
    The function name 'workspaced_service' might be a typographical error. Perhaps it was intended to be 'workspace_service'? Please check if this is the desired naming.
  • Reason this comment was not posted:
    Comment was on unchanged code.
22. backend/windmill-worker/src/job_logger_oss.rs:34
  • Draft comment:
    Typographical suggestion: The log message on this line reads "Implementation to store excess on disk in not OSS", which is awkward. Consider rephrasing it (e.g., "Implementation to store excess on disk is not available in OSS") for clarity.
  • Reason this comment was not posted:
    Comment was on unchanged code.

Workflow ID: wflow_pukyPcTE06a22lE7

You can customize Ellipsis by changing your verbosity settings, reacting with 👍 or 👎, replying to comments, or adding code review rules.

Copy link

cloudflare-workers-and-pages bot commented May 30, 2025

Deploying windmill with  Cloudflare Pages  Cloudflare Pages

Latest commit: 0296436
Status: ✅  Deploy successful!
Preview URL: https://7a534b7a.windmill.pages.dev
Branch Preview URL: https://di-ee-refactor-3.windmill.pages.dev

View logs

@diegoimbert diegoimbert marked this pull request as draft May 30, 2025 19:43
@diegoimbert
Copy link
Contributor Author

Actually I will spend more time monday fixing the ee repo because the CI has been failing there for a while

@diegoimbert diegoimbert force-pushed the di/ee-refactor-3 branch 2 times, most recently from 1b7b62d to f79d105 Compare June 2, 2025 15:06
@diegoimbert diegoimbert marked this pull request as ready for review June 2, 2025 18:08
Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important

Looks good to me! 👍

Reviewed everything up to 7648f01 in 2 minutes and 24 seconds. Click for details.
  • Reviewed 3944 lines of code in 109 files
  • Skipped 0 files when reviewing.
  • Skipped posting 12 draft comments. View those below.
  • Modify your settings and rules to customize what types of comments Ellipsis leaves. And don't forget to react with 👍 or 👎 to teach Ellipsis.
1. backend/windmill-worker/src/worker_flow.rs:90
  • Draft comment:
    The function update_flow_status_after_job_completion (starting around here) is highly complex with deep nesting of async calls and multiple SQL updates. Consider refactoring parts into smaller, self‑contained helper functions and add inline documentation to improve readability and maintainability.
  • Reason this comment was not posted:
    Comment was not on a location in the diff, so it can't be submitted as a review comment.
2. backend/windmill-worker/src/worker_flow.rs:1518
  • Draft comment:
    In compute_bool_from_expr (lines ~1508–1524), the code strictly compares the evaluated expression against the strings "true" and "false". This may be brittle if the evaluator returns boolean values or differently formatted strings. Consider adopting a more robust conversion that accepts actual booleans.
  • Reason this comment was not posted:
    Comment was not on a location in the diff, so it can't be submitted as a review comment.
3. backend/windmill-worker/src/worker_flow.rs:2960
  • Draft comment:
    The SQL queries using JSONB_SET to update the flow status (e.g. around lines 2960–2995) are very complex. Adding more inline comments explaining the intended structure of the JSON object and the logic behind those updates would greatly improve code clarity and ease future maintenance.
  • Reason this comment was not posted:
    Comment was not on a location in the diff, so it can't be submitted as a review comment.
4. backend/windmill-worker/src/worker_flow.rs:3189
  • Draft comment:
    In functions like payload_from_modules (lines ~3189–3200) and raw_script_to_payload (lines ~3971–3997), unwrap_or_else is used to obtain values (e.g. cache ttl, module values). If the assumptions fail these calls may panic. Consider handling these cases gracefully and propagating errors instead of panicking.
  • Reason this comment was not posted:
    Comment was not on a location in the diff, so it can't be submitted as a review comment.
5. backend/src/monitor.rs:47
  • Draft comment:
    There appears to be a potential typographical error in the constant name "BUNFIG_INSTALL_SCOPES_SETTING". Please confirm if this spelling is intentional or if it should be corrected (e.g., to "BUNDLE_INSTALL_SCOPES_SETTING").
  • Reason this comment was not posted:
    Comment was not on a location in the diff, so it can't be submitted as a review comment.
6. backend/windmill-api/src/agent_workers_oss.rs:28
  • Draft comment:
    The function name 'workspaced_service' might be a typographical error. Did you mean 'workspace_service'?
  • Reason this comment was not posted:
    Comment was on unchanged code.
7. backend/windmill-api/src/git_sync_oss.rs:9
  • Draft comment:
    Typo: The function name 'workspaced_service' may be intended to be 'workspace_service'. Please confirm and adjust if it was an unintended naming.
  • Reason this comment was not posted:
    Decided after close inspection that this draft comment was likely wrong and/or not actionable: usefulness confidence = 10% vs. threshold = 50% The function name doesn't appear to be a typo - 'workspaced' could be intentionally used as an adjective meaning "related to a workspace". Without more context from the codebase, we can't know if this naming convention is used elsewhere. The comment is asking for confirmation rather than pointing out a clear issue. This violates our rule about not asking authors to confirm their intentions. I could be wrong about the intentionality of the naming convention. There might be a consistent pattern in the codebase where 'workspace_service' is the standard name. Even if there is a naming convention, the comment violates our rules by asking for confirmation rather than stating a clear issue. We should not keep speculative comments or those asking for verification. Delete the comment as it violates our rules by asking for confirmation of intention rather than pointing out a clear issue.
8. backend/windmill-api/src/indexer_oss.rs:9
  • Draft comment:
    Typo detected: The function name workspaced_service appears to be misspelled. Did you mean workspace_service?
  • Reason this comment was not posted:
    Decided after close inspection that this draft comment was likely wrong and/or not actionable: usefulness confidence = 10% vs. threshold = 50% The function name could be intentionally using 'workspaced' as an adjective. Without seeing the rest of the codebase or understanding the naming conventions, we can't be certain this is a typo. The comment is speculative and asks for verification rather than pointing out a clear issue. According to our rules, we should not make speculative comments or ask for verification. The function name does look unusual, and 'workspace_service' would be a more common naming pattern. Maybe this really is a typo that should be fixed? Even if it might be a typo, we don't have strong evidence that it is incorrect. Our rules state we need strong evidence to keep a comment, and we should not make speculative suggestions. Delete the comment because it's speculative and asks for verification without strong evidence of being incorrect.
9. backend/windmill-api/src/kafka_triggers_oss.rs:17
  • Draft comment:
    The function name "workspaced_service" might be a typographical error. Did you mean "workspace_service"?
  • Reason this comment was not posted:
    Comment was on unchanged code.
10. backend/windmill-api/src/oauth2_oss.rs:51
  • Draft comment:
    The function name workspaced_service (line 51) appears to be a potential typographical error. Did you mean workspace_service?
  • Reason this comment was not posted:
    Comment was on unchanged code.
11. backend/windmill-api/src/oidc_oss.rs:22
  • Draft comment:
    Typographical suggestion: The function name "workspaced_service" might be a typo. Did you mean "workspace_service"?
  • Reason this comment was not posted:
    Comment was on unchanged code.
12. backend/windmill-worker/src/job_logger_oss.rs:20
  • Draft comment:
    Typo: In the log message, the phrase "in not OSS" appears ungrammatical. Consider revising it to something like "is not available in OSS".
  • Reason this comment was not posted:
    Comment was on unchanged code.

Workflow ID: wflow_R4UKJArWbtbOsNVi

You can customize Ellipsis by changing your verbosity settings, reacting with 👍 or 👎, replying to comments, or adding code review rules.

@rubenfiszel rubenfiszel merged commit 0e31623 into main Jun 2, 2025
15 checks passed
@rubenfiszel rubenfiszel deleted the di/ee-refactor-3 branch June 2, 2025 20:12
@github-actions github-actions bot locked and limited conversation to collaborators Jun 2, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants