Description
Currently, rules_pkg uses a custom intermediary manifest format and a custom python tool to create tar archives. However there is a growing appetite for using rules_pkg with bsdtar for various reasons, eg performance, support for compression formats etc. See: bazel-contrib/tar.bzl#9
In order to achieve this, we need to make some changes here rules_pkg which will abstract the manifest/tar builder so that i can be swapped out by another party. Ideally rules_pkg would provide one of these toolchains that maintains the status quo, and bazel-lib can provide one compatibility to allow users to use both interchangeably.
I believe that there are two sensible choices.
Option 1:
Standardize the currently in use intermediary format and toolchainize the tar tool, so that it can be swapped out by the users.
Option 2:
Entails the same requirements as the first one, but instead of a custom format, we settle on BSD standard mtree format. https://man.freebsd.org/cgi/man.cgi?mtree(8)
This format has been use in bazel-lib#tar for a long time, that is easy to produce, mutate, process even in starlark context. https://github.com/bazel-contrib/bazel-lib/blob/bdc6ade0ba1ebe88d822bcdf4d4aaa2ce7e2cd37/lib/private/tar.bzl#L440-L447