Open
Description
Problem Description
After running a web.Batch()
of simulations, it's time to download the results.
This is typically not a problem, but particularly for large batches I've run into:
- My computer decides to restart, killing the download task (it's windows...)
- There's some kind of connection issue and the download fails
I can easily use batch.load()
or Batch.download()
to get new files, but at the moment this re-downloads every file in the batch.
Desired Solution
I would love if there were an option to only download the "outstanding" files -- I'm already providing the download directory, so just check the set difference between the batch filenames and the filenames in the target directory and download the difference.
An even more advanced version of this would check that the downloaded files are complete, and finish downloading those which got cut off.