Skip to content

ADD: neurodebian freeze interface #240

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Nov 26, 2018
Merged

ADD: neurodebian freeze interface #240

merged 6 commits into from
Nov 26, 2018

Conversation

kaczmarj
Copy link
Collaborator

Closes #238

This pull request proposes adding an interface for neurodebian freeze (nd_freeze on the command line). This will pin the apt sources in a neurodebian image to a certain date (and optionally time).

Usage:

neurodocker generate docker -b neurodebian:stretch -p apt --ndfreeze date=20180312

Note that the call to nd_freeze will happen as early as possible in the Docker or Singularity specification.

@codecov-io
Copy link

codecov-io commented Nov 13, 2018

Codecov Report

Merging #240 into master will decrease coverage by 0.19%.
The diff coverage is 75%.

Impacted file tree graph

@@            Coverage Diff            @@
##           master     #240     +/-   ##
=========================================
- Coverage   80.95%   80.75%   -0.2%     
=========================================
  Files          33       33             
  Lines        1512     1533     +21     
  Branches      156      160      +4     
=========================================
+ Hits         1224     1238     +14     
- Misses        213      217      +4     
- Partials       75       78      +3
Impacted Files Coverage Δ
neurodocker/generators/common.py 83.63% <ø> (ø) ⬆️
neurodocker/tests/test_neurodocker.py 94.44% <100%> (+0.32%) ⬆️
neurodocker/parser.py 61.36% <100%> (ø) ⬆️
neurodocker/utils.py 72.44% <54.54%> (-2.84%) ⬇️
neurodocker/generators/singularity.py 76.8% <71.42%> (+0.18%) ⬆️
neurodocker/neurodocker.py 81.15% <75%> (ø) ⬆️
neurodocker/interfaces/interfaces.py 76.5% <75%> (-0.23%) ⬇️
neurodocker/generators/docker.py 88.46% <83.33%> (+0.07%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f7055a1...9a5b3fd. Read the comment docs.

@kaczmarj
Copy link
Collaborator Author

@yarikoptic - do you have any comments on this pr?

Copy link
Member

@yarikoptic yarikoptic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

left minor comments and suggestions around the points which have freeze mentioned. Otherwise - hard to review since confounded with all the formatting changes

--ndfreeze date=20180312
```

The call to `nd_freeze` will occur close to the beginning of the Docker or Singularity specification, regardless of the position of `--ndfreeze` on the command line.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

might be worth adding a note that it is in effect only for Debian and NeuroDebian (not Ubuntu) APT repositories, since they are the only ones providing snapshots.

dependencies:
apt: neurodebian-freeze
instructions: |
{{ ndfreeze.install_dependencies() }}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if it could/should be optimized -- nd_freeze will come available pre-installed in NeuroDebian docker base images (very soon hopefully), so in principle no dedicated line to install it would be needed.
Although probably there should be no harm if a dedicated apt-get install of it would be called first... and neurodebian/neurodebian#43 probably would be of no (negative) effect since neurodocker would clean up anyways.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can hold off on merging this pr until the neurodebian images are rebuilt, and i'll remove the apt-get install.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think they are now:

$> docker pull neurodebian:stretch
stretch: Pulling from library/neurodebian
bc9ab73e5b14: Already exists 
b323f9a9fdf6: Already exists 
9294e84bac26: Already exists 
474b78678a62: Already exists 
e955ce81ae0d: Pull complete 
Digest: sha256:c3eb8abe2fde32b4333b4fe9efdd698afabf4c49ba07784372710b813c802c8e
Status: Downloaded newer image for neurodebian:stretch
$> docker run -it --rm neurodebian:stretch nd_freeze
Invalid command parameters
USAGE: nd_freeze [--keep-original-apt-sources] date

but that means

  • build would fail if ppl have older version of image
  • build would not work if base image a stock debian image (which soon will get neurodebian-freeze)
  • also might be worth apt-get ing it just to get nd_freeze updated happen we fix things up
    So may be let's leave indeed as is for now ;)?

" method, and install_path",
"ndfreeze":
"Use the NeuroDebian command `nd_freeze` to freeze the apt"
" sources to a certain date.",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

again, might be worth adding that of effect only for debian and neurodebian, not ubuntu


_name = "ndfreeze"

def __init__(self, date, *args, **kwargs):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note that later on we would also add options, at least one, to e.g. force downgrade of already installed packages to the ones available from the snapshots (the last TODO in neurodebian/neurodebian#39 ;))

@yarikoptic
Copy link
Member

so IMHO could be merged to claim the victory ;) the rest could be tuned/fixed up later

FYI: nd_freeze now, and by default, would try to downgrade any pre-installed packages to the versions available from snapshots to guarantee that even the core installation is reproducible. I guess in some cases some config files (which might be not downgraded) could stick out but hopefully there is nothing like that in base images.

@kaczmarj kaczmarj merged commit 18c0299 into master Nov 26, 2018
@kaczmarj kaczmarj deleted the add/nd_freeze branch November 26, 2018 14:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants