Skip to content

Convert to new split pipeline approach for CI/CD #278

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 33 commits into from
Jun 26, 2020
Merged

Convert to new split pipeline approach for CI/CD #278

merged 33 commits into from
Jun 26, 2020

Conversation

j-so
Copy link
Contributor

@j-so j-so commented May 21, 2020 •
edited
Loading

Introduces a new set of pipelines that split model/train/eval and deployment.

  1. Model training/eval/registration - performs Model CI, trains, evaluates, and registers the model. If the model is registered, then it produces a pipeline artifact containing the model JSON.
  2. Model deployment - takes the pipeline artifact (model JSON) from the triggering pipeline and deploys that model.

NOTE: There is a limitation with the model deployment pipeline. It relies solely on the output of the model training pipeline. Right now it cannot run off the model name/version env vars.

Also addresses #274

@j-so j-so marked this pull request as ready for review May 27, 2020 00:40
@j-so j-so requested review from daniel-hanson and shivp950 June 4, 2020 20:45
@j-so j-so linked an issue Jun 9, 2020 that may be closed by this pull request
Q: Run pipeline on non-model changes #274
Closed
daniel-hanson
daniel-hanson approved these changes Jun 10, 2020
View reviewed changes
j-so added 4 commits June 10, 2020 12:19
add more docs and add to bootstrap
1128194
linting
7a69c0e
fix model package to show logs
bae4b61
Squashed commit of the following:
dbe8b33 
commit 01af8da
Author: j-so <[email protected]>
Date:   Fri Jun 12 14:15:35 2020 -0700

    fixed failure handling

commit 1e6f906
Author: j-so <[email protected]>
Date:   Fri Jun 12 14:03:09 2020 -0700

    test failed conda dep

commit a8030d7
Author: j-so <[email protected]>
Date:   Fri Jun 12 13:55:00 2020 -0700

    test package fail

commit c7845aa
Author: j-so <[email protected]>
Date:   Fri Jun 12 13:46:27 2020 -0700

    fail on deploy error'
dtzar
dtzar suggested changes Jun 15, 2020
View reviewed changes
Copy link
Contributor

@dtzar dtzar left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the CI steps are identical, I believe we should keep thing consistent by having a CI and then a CD pipeline which is triggered when the CI pipeline is complete. Sooo... This means for this PR:

  1. No need for a separate "getting started doc"
  2. Remove duplicate CI pipeline, fix the existing one to move the CD steps into a "Task based deployment" pipeline.
  3. Fix the main getting started guide to ask people if they want the AzDO task deployment or Az CLI deployment, setting whatever you feel is the best default one triggered by the CI pipeline. If you feel the engineering team doesn't want people to have the task deployment, then go ahead and remove this choice.

j-so added 2 commits June 15, 2020 18:32
Squashed commit of the following:
9bc9f8b 
commit 5af6eeb
Author: j-so <[email protected]>
Date:   Mon Jun 15 18:31:15 2020 -0700

    fix bootstrap

commit f61e103
Merge: 2796b40 08bb6f4
Author: j-so <[email protected]>
Date:   Mon Jun 15 18:30:21 2020 -0700

    Merge branch 'master' into jenns/splitpipeline_docfix

commit 2796b40
Author: j-so <[email protected]>
Date:   Mon Jun 15 18:30:00 2020 -0700

    remove old pipeline

commit 08bb6f4
Author: David Tesar <[email protected]>
Date:   Mon Jun 15 14:09:12 2020 -0700

    Simplify docs flow (#297)

commit cd762ec
Author: jotaylo <[email protected]>
Date:   Mon Jun 15 12:28:23 2020 -0700

    Move instruction to install AML extension to Azure Devops setup instructions (#298)
Merge branch 'master' into jenns/splitpipeline
82ab9ec
j-so added 6 commits June 18, 2020 14:38
Merge branch 'master' into jenns/splitpipeline
f58e0df
Merge branch 'master' into jenns/splitpipeline
bb61306
fix batch scoring
c6167eb
use model version for batch scoring
4bfa69b
linting
c9fc6ed
Squashed commit of the following:
a95183b 
commit 493ed3e
Author: j-so <[email protected]>
Date:   Mon Jun 22 16:42:07 2020 -0700

    mark as output

commit 1ca7a59
Author: j-so <[email protected]>
Date:   Mon Jun 22 16:12:10 2020 -0700

    fix import

commit 743e301
Author: j-so <[email protected]>
Date:   Mon Jun 22 15:59:43 2020 -0700

    more fixes

commit 44abcac
Author: j-so <[email protected]>
Date:   Mon Jun 22 15:50:49 2020 -0700

    fix batch scoring
@j-so j-so requested a review from jitghosh June 23, 2020 03:42
@j-so j-so changed the title Adding new split pipeline approach for CI/CD Convert to new split pipeline approach for CI/CD Jun 24, 2020
@j-so j-so merged commit b21a46e into master Jun 26, 2020
@j-so j-so linked an issue Jun 26, 2020 that may be closed by this pull request
create_scoring_image.py should use scoring/inference_config.yml #279
Open
@j-so j-so removed a link to an issue Jun 26, 2020
create_scoring_image.py should use scoring/inference_config.yml #279
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dtzar dtzar dtzar approved these changes

@daniel-hanson daniel-hanson daniel-hanson approved these changes

@shivp950 shivp950 Awaiting requested review from shivp950

@jitghosh jitghosh Awaiting requested review from jitghosh

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Q: Run pipeline on non-model changes
3 participants
@j-so @dtzar @daniel-hanson