File downloads should be atomic

[jewettaijfc]

File downloads should be atomic.

That means if the file exists at the target location with the correct name, we can assume the file transferred successfully.

Suggested implementation

• Download the file to a temporary location (in the same folder) with a different name. After download is successful, rename the file to the target name (eg. using os.rename()).

Describe alternatives you've considered

• More documentation to inform users how to check if their file download was successful.

Additional context

• Solving this problem will make it easier to solve problems like issue Downloading batch files should check to see if any data is already downloaded #2600 . All we have to do is check to see if the file exists at the target location.
• Currently, all Tidy3D downloads use a single function So we only have to update this one function. (That function is a wrapper for S3.Client.download_file() from amazon's boto3 module. We just have to add a few lines of logic to that function to rename the file afterwards.)

Priority

Moderate. Issue 2600 is a higher priority. Making file downloads atomic will make that issue easier to solve. But if for some reason, solving this issue is harder than we thought to implement, then we can try and implement some kind of work-around or warning.