Add span-based Deflate, ZLib and GZip encoder/decoder APIs by iremyux · Pull Request #123145 · dotnet/runtime

iremyux · 2026-01-13T18:46:21Z

This PR introduces new span-based, streamless compression and decompression APIs for Deflate, ZLib, and GZip formats, matching the existing BrotliEncoder/BrotliDecoder pattern.

New APIs

DeflateEncoder / DeflateDecoder
ZLibEncoder / ZLibDecoder
GZipEncoder / GZipDecoder

These classes provide:

Instance-based API for streaming/chunked compression with Compress(), Decompress(), and Flush()
Static one-shot API via TryCompress() and TryDecompress() for simple scenarios
GetMaxCompressedLength() to calculate buffer sizes

Closes #62113
Closes #39327
Closes #44793

src/libraries/System.IO.Compression/src/System/IO/Compression/ZlibEncoderOptions.cs

src/libraries/System.IO.Compression/src/System/IO/Compression/ZlibEncoder.cs

src/libraries/System.IO.Compression/ref/System.IO.Compression.cs

src/libraries/System.IO.Compression/src/System/IO/Compression/ZlibDecoder.cs

src/libraries/System.IO.Compression/src/System/IO/Compression/ZlibEncoder.cs

…mpressedLength

AraHaan · 2026-01-22T23:44:38Z

src/libraries/System.IO.Compression/src/System/IO/Compression/DeflateEncoder.cs

+                CompressionLevel.Fastest => ZLibNative.CompressionLevel.BestSpeed,
+                CompressionLevel.NoCompression => ZLibNative.CompressionLevel.NoCompression,
+                CompressionLevel.SmallestSize => ZLibNative.CompressionLevel.BestCompression,
+                _ => throw new ArgumentOutOfRangeException(nameof(compressionLevel)),


This would fail on valid native compression levels not covered by the CompressionLevel enum. Instead I think it should check if the value is is < -1 or > 9 to throw out of range instead.

Also to add on to the above, now those who want compression levels that just happen to == a value in the CompressionLevel enum will now not be able to use those compression levels either. Perhaps a solution to this is to expose a version of the ctor with CompressionLevel and a version with int that gets casted to ZLibNative.CompressionLevel after a range check.

src/libraries/System.IO.Compression/tests/DeflateZLibGZipEncoderDecoderTests.cs

src/libraries/System.IO.Compression/src/System/IO/Compression/GZipEncoder.cs

Copilot

Pull request overview

Copilot reviewed 34 out of 34 changed files in this pull request and generated 4 comments.

Comments suppressed due to low confidence (1)

src/libraries/System.IO.Compression/tests/Zstandard/ZstandardEncoderDecoderTests.cs:25

ZstandardUtils.WindowLog_Max is platform-dependent (31 on 64-bit, 30 on 32-bit). Hardcoding MaxWindowLog=31 and InvalidWindowLogTooHigh=32 will likely fail on 32-bit test runs. Consider using Environment.Is64BitProcess (or referencing ZstandardUtils.WindowLog_Max) to set MaxWindowLog and the corresponding invalid-too-high value.

        protected override int MinWindowLog => 10;
        protected override int MaxWindowLog => 31;
        protected override int InvalidQualityTooLow => -(1 << 17) - 1;
        protected override int InvalidQualityTooHigh => 23;
        protected override int InvalidWindowLogTooLow => 9;
        protected override int InvalidWindowLogTooHigh => 32;

src/libraries/System.IO.Compression/src/System/IO/Compression/DeflateEncoder.cs

src/libraries/System.IO.Compression/src/System/IO/Compression/ZLibCompressionOptions.cs

src/libraries/System.IO.Compression/tests/Zstandard/ZstandardEncoderDecoderTests.cs

src/libraries/System.IO.Compression/src/System/IO/Compression/DeflateEncoder.cs

src/libraries/System.IO.Compression/tests/CompressionStreamUnitTests.Deflate.cs

src/libraries/System.IO.Compression/tests/CompressionStreamUnitTests.Gzip.cs

src/libraries/System.IO.Compression/tests/CompressionStreamUnitTests.ZLib.cs

src/libraries/System.IO.Compression/tests/ZLibEncoderDecoderTestBase.cs

The file is only used by System.IO.Compression.Tests, so move it from the Common test directory to the test project directly and update the csproj reference accordingly. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

This reverts commit f36746f.

This reverts commit f4b24c2.

Copilot

Pull request overview

Copilot reviewed 30 out of 30 changed files in this pull request and generated 3 comments.

src/libraries/System.IO.Compression/src/System/IO/Compression/DeflateEncoder.cs

src/libraries/System.IO.Compression/src/System/IO/Compression/DeflateDecoder.cs

src/libraries/System.IO.Compression/src/System/IO/Compression/DeflateEncoder.cs

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 30 out of 30 changed files in this pull request and generated 2 comments.

src/libraries/System.IO.Compression/src/System/IO/Compression/GZipEncoder.cs

src/libraries/Common/tests/System/IO/Compression/EncoderDecoderTestBase.cs

rzikm

LGTM! Good job!

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 30 out of 30 changed files in this pull request and generated 2 comments.

src/libraries/System.IO.Compression/src/System/IO/Compression/ZLibCompressionOptions.cs

src/libraries/System.IO.Compression/tests/DeflateEncoderDecoderTests.cs

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 30 out of 30 changed files in this pull request and generated 9 comments.

src/libraries/System.IO.Compression/src/System/IO/Compression/GZipEncoder.cs

+            using var encoder = new GZipEncoder(quality, windowLog);
+            OperationStatus status = encoder.Compress(source, destination, out int consumed, out bytesWritten, isFinalBlock: true);
+
+            return status == OperationStatus.Done && consumed == source.Length;


src/libraries/System.IO.Compression/src/System/IO/Compression/GZipDecoder.cs

+
+            return status == OperationStatus.Done && consumed == source.Length;


src/libraries/Common/src/Interop/Interop.zlib.cs

src/libraries/System.IO.Compression/tests/DeflateEncoderDecoderTests.cs

src/libraries/System.IO.Compression/src/System/IO/Compression/DeflateEncoder.cs

+            using var encoder = new DeflateEncoder(quality, windowLog);
+            OperationStatus status = encoder.Compress(source, destination, out int consumed, out bytesWritten, isFinalBlock: true);
+
+            return status == OperationStatus.Done && consumed == source.Length;


src/libraries/System.IO.Compression/src/System/IO/Compression/DeflateDecoder.cs

+
+            return status == OperationStatus.Done && consumed == source.Length;


src/libraries/System.IO.Compression/src/System/IO/Compression/ZLibDecoder.cs

+
+            return status == OperationStatus.Done && consumed == source.Length;


src/libraries/Common/tests/System/IO/Compression/EncoderDecoderTestBase.cs

src/libraries/System.IO.Compression/src/System/IO/Compression/ZLibEncoder.cs

+            using var encoder = new ZLibEncoder(quality, windowLog);
+            OperationStatus status = encoder.Compress(source, destination, out int consumed, out bytesWritten, isFinalBlock: true);
+
+            return status == OperationStatus.Done && consumed == source.Length;


iremyux and others added 3 commits January 12, 2026 20:46

Add Zlib Encoder and Decoder classes

e402958

Add tests

95ac930

Merge branch 'dotnet:main' into 62113-zlib-encoder-decoder

88acc57

iremyux self-assigned this Jan 13, 2026

iremyux added the area-System.IO.Compression label Jan 13, 2026

AraHaan reviewed Jan 13, 2026

View reviewed changes

src/libraries/System.IO.Compression/src/System/IO/Compression/ZlibEncoderOptions.cs Outdated Show resolved Hide resolved

AraHaan reviewed Jan 13, 2026

View reviewed changes

src/libraries/System.IO.Compression/src/System/IO/Compression/ZlibEncoderOptions.cs Outdated Show resolved Hide resolved

AraHaan reviewed Jan 13, 2026

View reviewed changes

src/libraries/System.IO.Compression/src/System/IO/Compression/ZlibEncoderOptions.cs Outdated Show resolved Hide resolved

AraHaan reviewed Jan 13, 2026

View reviewed changes

src/libraries/System.IO.Compression/src/System/IO/Compression/ZlibEncoder.cs Outdated Show resolved Hide resolved

AraHaan reviewed Jan 13, 2026

View reviewed changes

src/libraries/System.IO.Compression/ref/System.IO.Compression.cs Outdated Show resolved Hide resolved

This was referenced Jan 13, 2026

"We stopped hearing from agent Azure Pipelines 32. Verify the agent machine is running and has a healthy network connection" dotnet/dnceng#1886

Open

Cannot find 'arm64-v8a' device dotnet/dnceng#2284

Closed

rzikm self-requested a review January 14, 2026 11:06

iremyux added 2 commits January 15, 2026 11:28

Use CompressionLevel instead of int

6f1ab80

Use CompressionLevel instead of int

8926759

rzikm reviewed Jan 16, 2026

View reviewed changes

iremyux added 2 commits January 19, 2026 10:51

Introduce seperate encoders and decoders

bb2d89c

Update the tests accordingly

a21a800

iremyux changed the title ~~[WIP] Add span-based ZlibEncoder and ZlibDecoder APIs~~ [WIP] Add span-based Deflate, ZLib and GZip encoder/decoder APIs Jan 19, 2026

iremyux added 4 commits January 19, 2026 11:35

Remove ZlibEncoder and decoder with lower L

4b87dcd

Add ZLibEncoder and Decoder with capital L

aabc3a6

Change GetMaxCompressedLengt's inputLength to long

cfb3d7e

"File is required to end with a single newline character"

f031eed

This was referenced Jan 19, 2026

[android][arm64] System.Net.Sockets.Tests.SendTo_SyncForceNonBlocking.Datagram_UDP_ShouldImplicitlyBindLocalEndpoint fails with NetworkUnreachable #120526

Open

browser-wasm linux LibraryTests build failure - NETSDK1022: Duplicate 'Content' items #123237

Closed

Add Interop layer for zlib's compressBound and use it inside GetMaxCo…

1d18eec

…mpressedLength

build-analysis bot mentioned this pull request Jan 20, 2026

Assertion failures in System.Tests.ConvertTests.TryFromBase64Chars on Mono MiniJIT and Interpreter across multiple Linux and macOS platforms #123389

Closed

AraHaan reviewed Jan 22, 2026

View reviewed changes

rzikm reviewed Jan 23, 2026

View reviewed changes

src/libraries/System.IO.Compression/tests/DeflateZLibGZipEncoderDecoderTests.cs Outdated Show resolved Hide resolved

rzikm reviewed Jan 23, 2026

View reviewed changes

src/libraries/System.IO.Compression/src/System/IO/Compression/GZipEncoder.cs Outdated Show resolved Hide resolved

Copilot AI reviewed Mar 25, 2026

View reviewed changes

rzikm reviewed Mar 26, 2026

View reviewed changes

iremyux and others added 8 commits April 9, 2026 13:53

REmove whitespaces

c415ebd

Move ResolveWindowBits to new CompressionFormatHelper class

9522265

Switch to manual computation inside GetMaxCompressedLength

a425e1f

Add test to verify our manual computation for GetMaxCompressedLength

2d6ac86

Undo wasm/callhelpers-pinvoke.cpp addition

f36746f

Revert "Undo wasm/callhelpers-pinvoke.cpp addition"

bb27a9c

This reverts commit f36746f.

Revert "Add CompressionNative_CompressBound to callhelpers-pinvoke.cpp"

b976f98

This reverts commit f4b24c2.

Copilot AI review requested due to automatic review settings April 13, 2026 17:57

Copilot started reviewing on behalf of iremyux April 13, 2026 17:58 View session

Copilot AI reviewed Apr 13, 2026

View reviewed changes

This was referenced Apr 14, 2026

TestNativeDigits fails for ur-IN on Apple platforms (xunit v3 exposed previously non-running test) #125933

Open

System.Net.NameResolution.Tests DNS failures: Name or service not known #126641

Open

Use ulong arithmetic in GetMaxCompressedLength to prevent overflow

627cb05

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings April 14, 2026 07:36

Copilot started reviewing on behalf of iremyux April 14, 2026 07:37 View session

iremyux requested a review from rzikm April 14, 2026 07:41

Copilot AI reviewed Apr 14, 2026

View reviewed changes

src/libraries/System.IO.Compression/src/System/IO/Compression/GZipEncoder.cs Outdated Show resolved Hide resolved

src/libraries/Common/tests/System/IO/Compression/EncoderDecoderTestBase.cs Show resolved Hide resolved

rzikm approved these changes Apr 14, 2026

View reviewed changes

Add check to Gzip's GetMaxCompressedLength

125cc24

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings April 14, 2026 09:29

Copilot started reviewing on behalf of iremyux April 14, 2026 09:30 View session

Copilot AI reviewed Apr 14, 2026

View reviewed changes

src/libraries/System.IO.Compression/src/System/IO/Compression/ZLibCompressionOptions.cs Show resolved Hide resolved

src/libraries/System.IO.Compression/tests/DeflateEncoderDecoderTests.cs Show resolved Hide resolved

Add comments for windowlog=8

46c3a18

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings April 14, 2026 10:01

Copilot started reviewing on behalf of iremyux April 14, 2026 10:02 View session

Copilot AI reviewed Apr 14, 2026

View reviewed changes


		return status == OperationStatus.Done && consumed == source.Length;

Conversation

iremyux commented Jan 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

AraHaan Jan 22, 2026

Choose a reason for hiding this comment

Uh oh!

AraHaan Feb 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

rzikm left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

iremyux commented Jan 13, 2026 •

edited

Loading

AraHaan Feb 4, 2026 •

edited

Loading