Skip to content

zlib: add zstd support#52100

Closed
hybrist wants to merge 2 commits intonodejs:mainfrom
hybrist:zstd
Closed

zlib: add zstd support#52100
hybrist wants to merge 2 commits intonodejs:mainfrom
hybrist:zstd

Conversation

@hybrist
Copy link
Contributor

@hybrist hybrist commented Mar 15, 2024

Adds ZstdCompress and ZsdDecompress to the zlib module which can be used to compress/decompress with the Zstandard ("zstd") algorithm.

Notable omissions:

  • Providing dictionaries isn't implemented.
  • The docs in zlib.md don't call out any params beyond the basic compression level.

The code follows similar patterns to the PR that added Brotli support. Just that instead of brotli, it adds the equivalent zstd APIs. Just like Brotli, this required separate compression/decompression context objects.

Zstd itself has been around and stable for multiple years but this PR is early in terms of web support: It only just starts shipping by default in Chrome 123. On the other hand, by shipping in Chrome it will soon be supported quite widely on the web. Firefox also signaled support (https://bugzilla.mozilla.org/show_bug.cgi?id=1301878#c65). Zstd is now enabled on the web and supported by both Chrome and Firefox: https://caniuse.com/zstd

Official support in node.js would allow passing additional WPTs around fetch (nodejs/undici#2847).

@nodejs-github-bot
Copy link
Collaborator

Review requested:

  • @nodejs/security-wg

@nodejs-github-bot nodejs-github-bot added dependencies Pull requests that update a dependency file. needs-ci PRs that need a full CI run. labels Mar 15, 2024
@hybrist hybrist force-pushed the zstd branch 4 times, most recently from ec19b6e to 28914c7 Compare March 17, 2024 00:07
@hybrist
Copy link
Contributor Author

hybrist commented Mar 17, 2024

Alright, got up to making the basics (seemingly) work:

$ out/Debug/node -p 'zlib.zstdDecompressSync(zlib.zstdCompressSync("Hello World")).toString()'
Hello World

@hybrist hybrist force-pushed the zstd branch 6 times, most recently from 383ff46 to 47c7ab8 Compare March 17, 2024 20:59
@hybrist hybrist marked this pull request as ready for review March 17, 2024 21:00
hybrist added a commit to hybrist/node that referenced this pull request Mar 17, 2024
hybrist added a commit to hybrist/node that referenced this pull request Mar 17, 2024
hybrist added a commit to hybrist/node that referenced this pull request Mar 17, 2024
hybrist added a commit to hybrist/node that referenced this pull request Mar 17, 2024
hybrist added a commit to hybrist/node that referenced this pull request Mar 17, 2024
hybrist added a commit to hybrist/node that referenced this pull request Mar 17, 2024
hybrist added a commit to hybrist/node that referenced this pull request Mar 17, 2024
hybrist added a commit to hybrist/node that referenced this pull request Mar 17, 2024
@bricss
Copy link
Contributor

bricss commented Feb 2, 2025

That's will be great to have this in a main branch 🌿after all 🫠

@KhafraDev
Copy link
Member

@nodejs/tsc There aren't any outstanding objections, but this PR does appear to be at an impasse. @jkrems did not receive an answer to their question #52100 (comment). I have held off for months from doing this, hoping someone could review and collaborate since I couldn't.

@KhafraDev KhafraDev added the tsc-agenda Issues and PRs to discuss during the meetings of the TSC. label Feb 7, 2025
@marco-ippolito
Copy link
Member

marco-ippolito commented Feb 7, 2025

Nobody reviewed and approved the PR, there is nothing officially blocking it. If someone wants to volunteer to review it and approve it without objections, then its gonna land.
I don't think there is anything the TSC can do about it, or to be discussed TSC side

@KhafraDev
Copy link
Member

Maybe not, but I don't think I'm in a position to give guidance or give definitive instructions on the approach. I have pinged the zlib team, I pinged the security team, I have asked people to review it in private, so I didn't know what else to do lol.

@anonrig
Copy link
Member

anonrig commented Feb 7, 2025

I'm interested in landing this PR as well. If we can have one last push (and rebase this PR), I'm more than happy to review and land this PR.

@hybrist
Copy link
Contributor Author

hybrist commented Feb 7, 2025

@anonrig Thanks! I'll work on a rebase.

hybrist added a commit to hybrist/node that referenced this pull request Feb 7, 2025
@anonrig anonrig added request-ci Add this label to start a Jenkins CI on a PR. commit-queue-rebase Add this label to allow the Commit Queue to land a PR in several commits. and removed tsc-agenda Issues and PRs to discuss during the meetings of the TSC. labels Feb 8, 2025
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Feb 8, 2025
@nodejs-github-bot
Copy link
Collaborator

@nodejs-github-bot
Copy link
Collaborator

@anonrig anonrig added the commit-queue Add this label to land a pull request using GitHub Actions. label Feb 8, 2025
@nodejs-github-bot nodejs-github-bot removed the commit-queue Add this label to land a pull request using GitHub Actions. label Feb 8, 2025
@nodejs-github-bot
Copy link
Collaborator

Landed in 0782c91...bf12d72

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

c++ Issues and PRs that require attention from people who are familiar with C++. commit-queue-rebase Add this label to allow the Commit Queue to land a PR in several commits. dependencies Pull requests that update a dependency file. needs-ci PRs that need a full CI run. notable-change PRs with changes that should be highlighted in changelogs. review wanted PRs that need reviews. semver-minor PRs that contain new features and should be released in the next minor version. zlib Issues and PRs related to the zlib subsystem.

Projects

None yet

Development

Successfully merging this pull request may close these issues.