Skip to content

Added root path prefix in listings template #135

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 5 commits into from
Closed

Added root path prefix in listings template #135

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
6990e59
Added root path prefix in listings template
zc-devs May 13, 2023
e78a969
Added env var in other Docker files standalone_ubuntu_oss_install.sh
zc-devs May 31, 2023
c369849
Added documentation
zc-devs May 31, 2023
87e51fa
Fixed cache config and documentation
zc-devs May 31, 2023
325b1b5
Fixed installation script
zc-devs May 31, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Dockerfile.buildkit.plus
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ ENV PROXY_CACHE_VALID_OK "1h"
ENV PROXY_CACHE_VALID_NOTFOUND "1m"
ENV PROXY_CACHE_VALID_FORBIDDEN "30s"
ENV CORS_ENABLED 0
ENV DIRECTORY_LISTING_PATH_PREFIX ""

COPY plus/usr /usr

Expand Down
1 change: 1 addition & 0 deletions Dockerfile.oss
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ ENV PROXY_CACHE_VALID_OK "1h"
ENV PROXY_CACHE_VALID_NOTFOUND "1m"
ENV PROXY_CACHE_VALID_FORBIDDEN "30s"
ENV CORS_ENABLED 0
ENV DIRECTORY_LISTING_PATH_PREFIX ""

# We modify the nginx base image by:
# 1. Adding configuration files needed for proxying private S3 buckets
Expand Down
1 change: 1 addition & 0 deletions Dockerfile.plus
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ ENV PROXY_CACHE_VALID_OK "1h"
ENV PROXY_CACHE_VALID_NOTFOUND "1m"
ENV PROXY_CACHE_VALID_FORBIDDEN "30s"
ENV CORS_ENABLED 0
ENV DIRECTORY_LISTING_PATH_PREFIX ""

COPY plus/etc/ssl /etc/ssl
COPY plus/usr /usr
Expand Down
33 changes: 33 additions & 0 deletions common/docker-entrypoint.d/23-envsubst-on-include-template.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
#!/bin/sh

set -e

ME=$(basename $0)

entrypoint_log() {
if [ -z "${NGINX_ENTRYPOINT_QUIET_LOGS:-}" ]; then
echo "$@"
fi
}

auto_envsubst() {
local include_dir="/etc/nginx/include"
local suffix="${NGINX_ENVSUBST_TEMPLATE_SUFFIX:-.template}"
local filter="${NGINX_ENVSUBST_FILTER:-}"

local template defined_envs relative_path output_path subdir
defined_envs=$(printf '${%s} ' $(awk "END { for (name in ENVIRON) { print ( name ~ /${filter}/ ) ? name : \"\" } }" < /dev/null ))
if [ ! -w "$include_dir" ]; then
entrypoint_log "$ME: ERROR: $include_dir is not writable"
return 0
fi
find "$include_dir" -follow -type f -name "*$suffix" -print | while read -r template; do
output_path="${template%$suffix}"
entrypoint_log "$ME: Running envsubst on $template to $output_path"
envsubst "$defined_envs" < "$template" > "$output_path"
done
}

auto_envsubst

exit 0
7 changes: 5 additions & 2 deletions common/etc/nginx/include/listing.xsl → ...on/etc/nginx/include/listing.xsl.template
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
<xsl:output method="html" encoding="utf-8" indent="yes"/>
<xsl:strip-space elements="*" />

<xsl:variable name="rootPath" select="'$DIRECTORY_LISTING_PATH_PREFIX'" />

<xsl:template match="/">
<xsl:choose>
<xsl:when test="//*[local-name()='Contents'] or //*[local-name()='CommonPrefixes']">
Expand Down Expand Up @@ -36,7 +38,7 @@
</title>
</head>
<body>
<h1>Index of /<xsl:value-of select="$globalPrefix"/></h1>
<h1>Index of /<xsl:value-of select="concat($rootPath, $globalPrefix)"/></h1>
<hr/>
<table id="list">
<thead>
Expand Down Expand Up @@ -124,7 +126,8 @@
encoded so that they form a valid link that NGINX can parse -->
<xsl:template name="encode-uri">
<xsl:param name="uri"/>
<xsl:for-each select="str:split($uri, '/')">
<xsl:variable name="prefixed_uri" select="concat($rootPath, $uri)" />
<xsl:for-each select="str:split($prefixed_uri, '/')">
<xsl:variable name="encoded" select="str:encode-uri(., 'true', 'UTF-8')" />
<xsl:variable name="more-encoded" select="
str:replace(
Expand Down
1 change: 1 addition & 0 deletions common/etc/nginx/nginx.conf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ env S3_STYLE;
env ALLOW_DIRECTORY_LIST;
env PROVIDE_INDEX_PAGE;
env APPEND_SLASH_FOR_POSSIBLE_DIRECTORY;
env DIRECTORY_LISTING_PATH_PREFIX;
env PROXY_CACHE_MAX_SIZE;
env PROXY_CACHE_INACTIVE;
env PROXY_CACHE_VALID_OK;
Expand Down
16 changes: 11 additions & 5 deletions docs/getting_started.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,12 +28,13 @@ running as a Container or as a Systemd service.
| `S3_STYLE` | Yes | `virtual`, `path`, `default` | `default` | The S3 host/path method. <li>`virtual` is the method that that uses DNS-style bucket+hostname:port. This is the `default` value. <li>`path` is a method that appends the bucket name as the first directory in the URI's path. This method is used by many S3 compatible services. <br/><br/>See this [AWS blog article](https://aws.amazon.com/blogs/aws/amazon-s3-path-deprecation-plan-the-rest-of-the-story/) for further information. |
| `DEBUG` | No | `true`, `false` | `false` | Flag enabling AWS signatures debug output |
| `APPEND_SLASH_FOR_POSSIBLE_DIRECTORY` | No | `true`, `false` | `false` | Flag enabling the return a 302 with a `/` appended to the path. This is independent of the behavior selected in `ALLOW_DIRECTORY_LIST` or `PROVIDE_INDEX_PAGE`. |
| `DIRECTORY_LISTING_PATH_PREFIX` | No | | | In `ALLOW_DIRECTORY_LIST=true` mode [adds defined prefix to links](#configuring-directory-listing) |
| `DNS_RESOLVERS` | No | | | DNS resolvers (separated by single spaces) to configure NGINX with |
| `PROXY_CACHE_MAX_SIZE` | No | | | Limits cache size |
| `PROXY_CACHE_INACTIVE` | No | | | Cached data that are not accessed during the time specified by the parameter get removed from the cache regardless of their freshness |
| `PROXY_CACHE_VALID_OK` | No | | | Sets caching time for response code 200 and 302 |
| `PROXY_CACHE_VALID_NOTFOUND` | No | | | Sets caching time for response code 404 |
| `PROXY_CACHE_VALID_FORBIDDEN` | No | | | Sets caching time for response code 403 |
| `PROXY_CACHE_MAX_SIZE` | No | | `10g` | Limits cache size |
| `PROXY_CACHE_INACTIVE` | No | | `60m` | Cached data that are not accessed during the time specified by the parameter get removed from the cache regardless of their freshness |
| `PROXY_CACHE_VALID_OK` | No | | `1h` | Sets caching time for response code 200 and 302 |
| `PROXY_CACHE_VALID_NOTFOUND` | No | | `1m` | Sets caching time for response code 404 |
| `PROXY_CACHE_VALID_FORBIDDEN` | No | | `30s` | Sets caching time for response code 403 |
| `PROVIDE_INDEX_PAGE` | No | `true`, `false` | `false` | Flag which returns the index page if there is one when requesting a directory. Cannot be enabled with `ALLOW_DIRECTORY_LIST`. |
| `JS_TRUSTED_CERT_PATH` | No | | | Enables the `js_fetch_trusted_certificate` directive when retrieving AWS credentials and sets the path (on the container) to the specified path |
| `HEADER_PREFIXES_TO_STRIP` | No | | | A list of HTTP header prefixes that exclude headers client responses. List should be specified in lower-case and a semicolon (;) should be used to as a deliminator between values. For example: `x-goog-;x-something-` |
Expand Down Expand Up @@ -78,6 +79,11 @@ result in log messages like:
Another limitation is that when using v2 signatures with HEAD requests, the
gateway will not return 200 for valid folders.

If you want list bucket root at some prefix, you can place some reverse proxy before nginx-s3-gateway
to strip that prefix and `DIRECTORY_LISTING_PATH_PREFIX` to add that prefix in listing page's header and links.
Example below uses `DIRECTORY_LISTING_PATH_PREFIX='main/'` and HAProxy's `http-request set-path %[path,regsub(^/main,/)]`:
![](./img/nginx-s3-gateway-directory-listing-path-prefix.png)

### Static Site Hosting

When `PROVIDE_INDEX_PAGE` environment variable is set to 1, the gateway will
Expand Down
Binary file added docs/img/nginx-s3-gateway-directory-listing-path-prefix.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions settings.example
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ AWS_SIGS_VERSION=4
ALLOW_DIRECTORY_LIST=false
PROVIDE_INDEX_PAGE=false
APPEND_SLASH_FOR_POSSIBLE_DIRECTORY=false
DIRECTORY_LISTING_PATH_PREFIX=""
PROXY_CACHE_MAX_SIZE=10g
PROXY_CACHE_INACTIVE=60m
PROXY_CACHE_VALID_OK=1h
Expand Down
Loading