Multistage Docker / Ubuntu 22.04 WIP

[dokterbob]

Thus far sloppy, very much work in progress. Just figured sharing is caring. :)

[dokterbob]

Just tested this locally with Brighton Beach Road at Defaults and it's working!

It's branched off from my prior Docker cleanup branch. If this is too fast or needs more validation you can safely merge the other first.

This does require a more 'modern' Docker with BuildKit as it uses cache binds mounts and might require DOCKER_BUILDKIT=1 when building on Docker Engine <23. Shouldn't change run time deps though and GH Actions should build fine.

[pierotofy]

Nice, but be careful, some subtle issues might be introduced in upgrading Ubuntu, especially because it means upgrading gdal, proj & company. So extensive testing is needed.

It looks like the test suite already caught something.

[kikislater]

Great ! My question is why Ubuntu nowadays ?
They are doing some non standard stuffs like this weird and annoying netplan causing ton of issues and recently the implement of sudo in rust whereas they are the only one doing this. IMO it is better to move to debian : far away more stable even if ubuntu is based on debian !

[dokterbob]

Great ! My question is why Ubuntu nowadays ? They are doing some non standard stuffs like this weird and annoying netplan causing ton of issues and recently the implement of sudo in rust whereas they are the only one doing this. IMO it is better to move to debian : far away more stable even if ubuntu is based on debian !

As @pierotofy said, even changing deps. in slight ways can (and does) cause unexpected issues. So I try to change as little as I can at a time.

Perhaps a later PR can test how/whether stuff runs in Alpine, which is generally more suited for Docker containers.

For now, let me have a look at that CI failure. :)

[Saijin-Naib]

If you do investigate Alpine, I have been dailying it as a desktop OS for five years now and have been learning abuild/packaging with the eventual goal of trying to get everything in WebODM running on Alpine.

To what you said before about security, being musl instead of glibc seems to reduce applicable CVEs significantly, and further, default policy of compiling with Stack Smash Protection and Position Independent Executables seems to basically take care of the rest. -Os is the optimization level, so very small and usually exact same runtime perf as -O2.

In my usage, it has no noticeable performance penalty compared to more common desktop distros, though I believe musl malloc is slower in benchmarks 🤷

All that to say, I am over my head trying to port our whole stack, but this is a longterm goal of mine and I'm available to help in any way I can, and know a good number of folks in the Alpine community so maybe we can get some help/advice as well.

[dokterbob]

If you do investigate Alpine, I have been dailying it as a desktop OS for five years now and have been learning abuild/packaging with the eventual goal of trying to get everything in WebODM running on Alpine.

Wow, Alpine as desktop. Didn't know that was a thing. :)

To what you said before about security, being musl instead of glibc seems to reduce applicable CVEs significantly, and further, default policy of compiling with Stack Smash Protection and Position Independent Executables seems to basically take care of the rest. -Os is the optimization level, so very small and usually exact same runtime perf as -O2.

In my usage, it has no noticeable performance penalty compared to more common desktop distros, though I believe musl malloc is slower in benchmarks 🤷

Yes all of that is correct but some stuff has difficulty running/compiling AFAIK.

All that to say, I am over my head trying to port our whole stack, but this is a longterm goal of mine and I'm available to help in any way I can, and know a good number of folks in the Alpine community so maybe we can get some help/advice as well.

Consider the current effort a preamble to that.

[dokterbob]

Running this locally, I get two most interesting fails:

======================================================================
FAIL: test_profile (app.tests.test_api_admin.TestApi)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/webodm/app/tests/test_api_admin.py", line 314, in test_profile
    self.assertIsNone(user.profile.get_quota_deadline())
AssertionError: 1747405717.1662018 is not None

======================================================================
FAIL: test_quota (app.tests.test_quota.TestQuota)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/webodm/app/tests/test_quota.py", line 41, in test_quota
    self.assertFalse("disk quota is being exceeded" in body)
AssertionError: True is not false

----------------------------------------------------------------------
Ran 56 tests in 155.392s

Perhaps due to bf3bc03 as 9afaf99 passed just fine.

[dokterbob]

Ready for manual testing by others, I'd say!

[pierotofy]

Running this locally, I get two most interesting fails:

Mm, does it pass locally if you modify test_quota.py and test_api_admin.py to have:

from django.core.cache import cache

class ....
    def setUp(self):
        cache.clear() # <-- add this

?

This is looking pretty good; I will run some tests myself over the next day or two. 🙏

[dokterbob]

Running this locally, I get two most interesting fails:

Mm, does it pass locally if you modify test_quota.py and test_api_admin.py to have:

from django.core.cache import cache

class ....
    def setUp(self):
        cache.clear() # <-- add this

?

I will have to do this when back home, gimme a couple of days.