Skip to content

rustdoc-json: Output is is zstd compressed twice. #2833

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
aDotInTheVoid opened this issue May 26, 2025 · 1 comment · Fixed by #2834
Closed

rustdoc-json: Output is is zstd compressed twice. #2833

aDotInTheVoid opened this issue May 26, 2025 · 1 comment · Fixed by #2834

Comments

@aDotInTheVoid
Copy link
Member

The new rustdoc json enpoints (https://docs.rs/about/rustdoc-json) return files that need to be decompressed twice before getting the json:

alona@Ashtabula:~/tmp/docsrszstd$ curl --location  https://docs.rs/crate/clap/latest/json -o ./docs-rs-output
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 68106  100 68106    0     0   241k      0 --:--:-- --:--:-- --:--:--  241k
alona@Ashtabula:~/tmp/docsrszstd$ file ./docs-rs-output
./docs-rs-output: Zstandard compressed data (v0.8+), Dictionary ID: None
alona@Ashtabula:~/tmp/docsrszstd$ unzstd ./docs-rs-output -o decompressed-once
./docs-rs-output    : 68097 bytes
alona@Ashtabula:~/tmp/docsrszstd$ file ./decompressed-once
./decompressed-once: Zstandard compressed data (v0.8+), Dictionary ID: None
alona@Ashtabula:~/tmp/docsrszstd$ unzstd ./decompressed-once -o decompressed-twice
./decompressed-once : 422580 bytes
alona@Ashtabula:~/tmp/docsrszstd$ file ./decompressed-twice
./decompressed-twice: JSON text data
alona@Ashtabula:~/tmp/docsrszstd$ du -sh * | sort -h
68K     decompressed-once
68K     docs-rs-output
416K    decompressed-twice

This doesn't save any space, but makes using this more cumbersome and slower.

The files served from docs.rs should be the json compressed once, not twice.

@syphar
Copy link
Member

syphar commented May 26, 2025

oh damn, very valid bug report, I also know why.

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants