feat(workflow_engine): Setup the Task for process_workflow_updates #93553

saponifi3d · 2025-06-13T19:59:44Z

Description

Create a celery task namespace for the workflow engine tasks
Create a task for processing an activity in the workflow_engine -- this will allow us to create a celery task to async execute process workflows when the issue platform creates an activity

In the code I mention there being a follow-up PR: #93580 is the initial PR to support the Activity / Group models.

codecov · 2025-06-13T20:17:00Z

Codecov Report

Attention: Patch coverage is 98.11321% with 1 line in your changes missing coverage. Please review.

✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
src/sentry/workflow_engine/tasks.py	95.45%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #93553      +/-   ##
==========================================
+ Coverage   88.00%   88.04%   +0.03%     
==========================================
  Files       10322    10340      +18     
  Lines      595043   596685    +1642     
  Branches    23141    23141              
==========================================
+ Hits       523691   525355    +1664     
+ Misses      70859    70837      -22     
  Partials      493      493

## Description Starting implementation for: https://www.notion.so/sentry/Workflow-Status-Changes-1fa8b10e4b5d80a48acddb95d160da1f?source=copy_link#1fa8b10e4b5d80e6bb1aef39cab2c6dc This will add a registry of handlers when an activity is created, we'll use this in the [workflow engine to kick-off a task](#93553) to execute `process_workflows`. This also adds a way to pass the detector_id through the StatusMessageData all the way to a hook that we can invoke and read directly from the StatusMessageData later. We are also passed the `activity` so `workflow_engine` can trigger `.send_notification` (if evaluated to do so).

- Create a task for handling activity updates

… if it's working

saponifi3d · 2025-06-13T22:48:53Z

src/sentry/workflow_engine/tasks.py

+    max_retries=5,
+    soft_time_limit=50,
+    time_limit=60,
+    silo_mode=SiloMode.REGION,
+    taskworker_config=config.TaskworkerConfig(
+        namespace=namespaces.workflow_engine_tasks,
+        processing_deadline_duration=60,
+        retry=retry.Retry(
+            times=3,
+            delay=5,


Anyone have any thoughts on what these values for retry / limits should be? just using what we have been.

There isn't any right/wrong answer here. However, if you are going to define Retry you should also define on which exceptions that retrys should be performed. Without that retries will only be automatically performed for TimeoutError or an explicit RetryError.

👍 thanks for the additional info -- seems like a number we can fudge around with in the future then, and we can use the same configuration as we are with issue alert processing (that's where i pulled these initial numbers from)

when would we not want to go with acks_late=True in cases where we'd rather double process than drop one?

src/sentry/taskworker/namespaces.py

markstory · 2025-06-17T15:45:00Z

src/sentry/workflow_engine/tasks.py

+    max_retries=5,
+    soft_time_limit=50,
+    time_limit=60,
+    silo_mode=SiloMode.REGION,
+    taskworker_config=config.TaskworkerConfig(
+        namespace=namespaces.workflow_engine_tasks,
+        processing_deadline_duration=60,
+        retry=retry.Retry(
+            times=3,
+            delay=5,


There isn't any right/wrong answer here. However, if you are going to define Retry you should also define on which exceptions that retrys should be performed. Without that retries will only be automatically performed for TimeoutError or an explicit RetryError.

markstory · 2025-06-17T15:45:59Z

src/sentry/workflow_engine/tasks.py

+
+    The task will get the Activity from the database, create a WorkflowEventData object,
+    and then process the data in `process_workflows`.
+    """


Any ideas on what kind of throughput tasks in the workflow_tasks namespace will have?

Hmm - not sure about throughput, i think that it will vary pretty heavily depending on configurations / products. for example, a metric issue with anomaly detection will be more computing time than evaluating if an event is new.

In the end, the throughput would be similar to issue alerts / metric alerts / crons -- all three of those product verticals will be executing here, but they're all fairly different rates.

Ok. Knowing that it will be similar to alerts is helpful.

We can add this to the alerts taskworker pools when we get US deployed. In other regions throughput will be small enough that it isn't a concern.

- Add queue to server config - improved task name / queue - added acks_late=True

kcons · 2025-06-18T19:50:29Z

src/sentry/workflow_engine/tasks.py

+    """
+    Process a workflow task identified by the given Activity ID and Detector ID.
+
+    This task will retry up to 3 times with a delay of 5 seconds between attempts.


I think we can leave this to the annotation to specify and remove this.

## Description Starting implementation for: https://www.notion.so/sentry/Workflow-Status-Changes-1fa8b10e4b5d80a48acddb95d160da1f?source=copy_link#1fa8b10e4b5d80e6bb1aef39cab2c6dc This will add a registry of handlers when an activity is created, we'll use this in the [workflow engine to kick-off a task](#93553) to execute `process_workflows`. This also adds a way to pass the detector_id through the StatusMessageData all the way to a hook that we can invoke and read directly from the StatusMessageData later. We are also passed the `activity` so `workflow_engine` can trigger `.send_notification` (if evaluated to do so).

…93553) ## Description - Create a celery task namespace for the workflow engine tasks - Create a task for processing an activity in the workflow_engine -- this will allow us to create a celery task to async execute process workflows when the issue platform creates an activity In the code I mention there being a follow-up PR: #93580 is the initial PR to support the `Activity` / `Group` models.

github-actions bot added the Scope: Backend Automatically applied to PRs that change backend components label Jun 13, 2025

saponifi3d mentioned this pull request Jun 13, 2025

feat(workflow_engine): Add registry hook for Activity creation #93522

Merged

saponifi3d force-pushed the jcallender/aci/workflow-status-update-handler branch from 5001848 to 8b6fffc Compare June 13, 2025 21:52

vercel bot deployed to Preview June 13, 2025 21:54 View deployment

vercel bot deployed to Preview June 13, 2025 22:44 View deployment

Base automatically changed from jcalelnder/aci/activity-registry to master June 13, 2025 22:45

saponifi3d added 5 commits June 13, 2025 15:46

- Create a task namespace for workflow engine

8b6f227

- Create a task for handling activity updates

Start setting up the task for the activity handler

4a2faaa

add the detector_id from the other stuff to the handler so we can see…

e7011de

… if it's working

update the task after changing the base branch

5495d63

add some tests and make sure things are working

62fc6a1

saponifi3d force-pushed the jcallender/aci/workflow-status-update-handler branch from 8814284 to 62fc6a1 Compare June 13, 2025 22:46

saponifi3d marked this pull request as ready for review June 13, 2025 22:47

saponifi3d requested review from a team as code owners June 13, 2025 22:47

vercel bot deployed to Preview June 13, 2025 22:48 View deployment

saponifi3d commented Jun 13, 2025

View reviewed changes

src/sentry/taskworker/namespaces.py Show resolved Hide resolved

markstory reviewed Jun 17, 2025

View reviewed changes

markstory mentioned this pull request Jun 18, 2025

Provision brokers, workers and scheduler in US getsentry/taskbroker#381

Open

saponifi3d added 2 commits June 18, 2025 10:23

cleanup code a little and fix first broken test

8e94ad9

Address the remaining PR comments and fix remaining failing test

a90fcdb

- Add queue to server config - improved task name / queue - added acks_late=True

saponifi3d requested a review from a team as a code owner June 18, 2025 17:48

vercel bot deployed to Preview June 18, 2025 17:49 View deployment

saponifi3d requested a review from kcons June 18, 2025 19:00

kcons approved these changes Jun 18, 2025

View reviewed changes

that time i put def instead of class.

1e0f9c4

vercel bot deployed to Preview June 18, 2025 20:53 View deployment

don't let ai drive alone

4a13c46

saponifi3d enabled auto-merge (squash) June 18, 2025 20:55

vercel bot deployed to Preview June 18, 2025 20:56 View deployment

saponifi3d merged commit fb69b01 into master Jun 18, 2025
64 checks passed

saponifi3d deleted the jcallender/aci/workflow-status-update-handler branch June 18, 2025 21:13

saponifi3d restored the jcallender/aci/workflow-status-update-handler branch June 18, 2025 22:51

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

feat(workflow_engine): Setup the Task for process_workflow_updates #93553

feat(workflow_engine): Setup the Task for process_workflow_updates #93553

Uh oh!

saponifi3d commented Jun 13, 2025 •

edited

Loading

Uh oh!

codecov bot commented Jun 13, 2025 •

edited

Loading

Uh oh!

saponifi3d Jun 13, 2025

Uh oh!

markstory Jun 17, 2025

Uh oh!

saponifi3d Jun 17, 2025

Uh oh!

kcons Jun 17, 2025

Uh oh!

Uh oh!

Uh oh!

markstory Jun 17, 2025

Uh oh!

markstory Jun 17, 2025

Uh oh!

saponifi3d Jun 17, 2025

Uh oh!

markstory Jun 18, 2025

Uh oh!

markstory Jun 18, 2025

Uh oh!

kcons Jun 18, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

feat(workflow_engine): Setup the Task for process_workflow_updates #93553

feat(workflow_engine): Setup the Task for process_workflow_updates #93553

Uh oh!

Conversation

saponifi3d commented Jun 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Uh oh!

codecov bot commented Jun 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

saponifi3d commented Jun 13, 2025 •

edited

Loading

codecov bot commented Jun 13, 2025 •

edited

Loading