Run selected checkoutScripts in build-only mode

[jkloetzke]

By default no checkout scripts are run when building with --build-only.
Some use cases practically require the checkoutScript to be always run,
through. A typical example are code generators that generate sources
from some high level description. These generators must be run every
time when the user has changed the input. A recipe or class can
explicitly opt in to run their checkoutScript also in build-only mode to
cover such a use case. This is done by either setting checkoutUpdateIf
to True or by a boolean expression that is evaluated to True. Otherwise
the checkoutScript is ignored even if some other class enables its
script. The checkoutUpdateIf property thus only applies to the
corresponding checkoutScript in the same recipe/class.

[codecov]

Codecov Report

Base: 86.24% // Head: 86.39% // Increases project coverage by +0.15% 🎉

Coverage data is based on head (5efc959) compared to base (4676adf).
Patch coverage: 96.38% of modified lines in pull request are covered.

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #493      +/-   ##
==========================================
+ Coverage   86.24%   86.39%   +0.15%     
==========================================
  Files          46       46              
  Lines       14202    14219      +17     
==========================================
+ Hits        12248    12285      +37     
+ Misses       1954     1934      -20     

Impacted Files	Coverage Δ
pym/bob/invoker.py	86.68% <83.33%> (+3.26%)	⬆️
pym/bob/intermediate.py	94.60% <92.85%> (-0.07%)	⬇️
pym/bob/languages.py	89.83% <95.45%> (+0.17%)	⬆️
pym/bob/builder.py	92.75% <100.00%> (+0.37%)	⬆️
pym/bob/cmds/invoke.py	88.09% <100.00%> (ø)
pym/bob/input.py	91.88% <100.00%> (+0.09%)	⬆️
pym/bob/stringparser.py	90.99% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

[jkloetzke]

@mahaase @rhubert It would be good to have your input on this. It should be rather flexible and cover the known use cases without introducing a new kind of checkoutScript...

[rhubert]

I did some simple tests and it looks good to me. Just to get it right - atm we have something like:

checkoutSCM:
 scm: url
 url: foo
 extract: false

checkoutScript: |
  extract foo

buildScript: |
  run_gen $1/foo
  build..

Now with the checkoutUpdateIf this could be changed to:

checkoutSCM:
 scm: url
 url: foo
 extract: false

checkoutUpdateIf: True
checkoutScript: |
  if  [ foo_has_changed ]; then
     extract foo
  fi
  run_gen .

buildScript: |
  build ..

Is there any possibility to avoid the need of a custom foo_has_changed? Maybe a BOB_BUILD_ONLY_MODE env var?

[jkloetzke]

Is there any possibility to avoid the need of a custom foo_has_changed? Maybe a BOB_BUILD_ONLY_MODE env var?

Actually I have already thought about something along these lines. The thing is, it is usually not needed to know. The checkout script can be re-run at any time. The user could add -f when building, a tool that used by the checkout script could change, you could manually call checkout.sh... In all these cases the checkoutScript should only do the extract foo if foo has actually changed. That's how the automatic extraction is implemented in Bob. So looking at a "build mode" environment variable would not be sufficient. Or do I overlook something?

[rhubert]

Hmm...I think you're right. If extract foo needs to be rerun did not only depend on -b. What if extract is a checkoutTool,... 🤔
Looks like special cases need special treatment....