[PATCH 0/2] uncompress: lz4 fixes

[PATCH 0/2] uncompress: lz4 fixes

[Sascha Hauer]

This fixes two bugs in LZ4 decompression. The first is a bug we derived
from Linux and was fixed there back in 2014. The second is a general bug
in uncompress_fd_to_fd() but seems to hit only in the LZ4 case. Both
bugs are relevant only for the uncompress command, but not when LZ4 is
used for compiled in binaries.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
---
Sascha Hauer (2):
      initrd: fix lz4 decompress with initrd
      uncompress: fix prebuffering

 lib/decompress_unlz4.c | 65 +++++++++++++++++++++++++++++++++-----------------
 lib/uncompress.c       |  7 +++---
 2 files changed, 47 insertions(+), 25 deletions(-)
---
base-commit: dbc28aa3aad573f6ed0199c4fb85295b95e3bd11
change-id: 20250522-uncompress-fixes-d94b645acaad

Best regards,
-- 
Sascha Hauer <s.hauer@pengutronix.de>

[PATCH 1/2] initrd: fix lz4 decompress with initrd

[Sascha Hauer]

This is an adoption of the corresponding Linux commit:

| commit 4d4b866aee039d609c0b40e7e5b27204607ce614
| Author: Yinghai Lu <yinghai@kernel.org>
| Date:   Fri Aug 8 14:23:10 2014 -0700
|
| During testing initrd (>2G) support, find decompress/lz4 does not work
| with initrd at all.
|
| decompress_* should support:
| 1. inbuf[]/outbuf[] for kernel preboot.
| 2. inbuf[]/flush() for initramfs
| 3. fill()/flush() for initrd.
|
| in the unlz4 does not handle case 3, as input len is passed as 0, and it
| failed in first try.
|
| Fix that add one extra if (fill) checking, and get out if EOF from the
| fill().
|
| Signed-off-by: Yinghai Lu <yinghai@kernel.org>
| Cc: Kyungsik Lee <kyungsik.lee@lge.com>
| Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
| Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

In barebox case 3 is used when called from uncompress_fd_to_fd() which
is used in the uncompress command. This seems to never have worked with
LZ4 compressed files.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
---
 lib/decompress_unlz4.c | 65 +++++++++++++++++++++++++++++++++-----------------
 1 file changed, 43 insertions(+), 22 deletions(-)

diff --git a/lib/decompress_unlz4.c b/lib/decompress_unlz4.c
index a18e6591e9fd1e63158ac58c6d265df4f002fdf5..2bb30e71bb1ca1d73ae20f815f0a2be93d4ea7da 100644
--- a/lib/decompress_unlz4.c
+++ b/lib/decompress_unlz4.c
@@ -90,13 +90,20 @@ static inline int unlz4(u8 *input, long in_len,
 	if (posp)
 		*posp = 0;
 
-	if (fill)
-		fill(inp, 4);
+	if (fill) {
+		size = fill(inp, 4);
+		if (size < 4) {
+			error("data corrupted");
+			goto exit_2;
+		}
+	}
 
 	chunksize = get_unaligned_le32(inp);
 	if (chunksize == ARCHIVE_MAGICNUMBER) {
-		inp += 4;
-		size -= 4;
+		if (!fill) {
+			inp += 4;
+			size -= 4;
+		}
 	} else {
 		error("invalid header");
 		goto exit_2;
@@ -107,29 +114,44 @@ static inline int unlz4(u8 *input, long in_len,
 
 	for (;;) {
 
-		if (fill)
-			fill(inp, 4);
+		if (fill) {
+			size = fill(inp, 4);
+			if (size == 0)
+				break;
+			if (size < 4) {
+				error("data corrupted");
+				goto exit_2;
+			}
+		}
 
 		chunksize = get_unaligned_le32(inp);
 		if (chunksize == ARCHIVE_MAGICNUMBER) {
-			inp += 4;
-			size -= 4;
+			if (!fill) {
+				inp += 4;
+				size -= 4;
+			}
 			if (posp)
 				*posp += 4;
 			continue;
 		}
-		inp += 4;
-		size -= 4;
+
 
 		if (posp)
 			*posp += 4;
 
-		if (fill) {
+		if (!fill) {
+			inp += 4;
+			size -= 4;
+		} else {
 			if (chunksize > lz4_compressbound(uncomp_chunksize)) {
 				error("chunk length is longer than allocated");
 				goto exit_2;
 			}
-			fill(inp, chunksize);
+			size = fill(inp, chunksize);
+			if (size < chunksize) {
+				error("data corrupted");
+				goto exit_2;
+			}
 		}
 #ifdef PREBOOT
 		if (out_len >= uncomp_chunksize) {
@@ -155,18 +177,17 @@ static inline int unlz4(u8 *input, long in_len,
 		if (posp)
 			*posp += chunksize;
 
-		size -= chunksize;
+		if (!fill) {
+			size -= chunksize;
 
-		if (size == 0)
-			break;
-		else if (size < 0) {
-			error("data corrupted");
-			goto exit_2;
+			if (size == 0)
+				break;
+			else if (size < 0) {
+				error("data corrupted");
+				goto exit_2;
+			}
+			inp += chunksize;
 		}
-
-		inp += chunksize;
-		if (fill)
-			inp = inp_start;
 	}
 
 	ret = 0;

-- 
2.39.5

[PATCH 2/2] uncompress: fix prebuffering

[Sascha Hauer]

When uncompressing from a fill function we prebuffer the first 32 bytes
in order to detect the filetype. This prebuffer is then fed back into
the input stream in uncompress_fill().

This currently fails when uncompress_fill() crosses the prebuffered
size and continues reading from the original fill function. Advance
the buffer in this case to append the data to the buffer instead of
overwriting the data we already copied from the prebuffer.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
---
 lib/uncompress.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/lib/uncompress.c b/lib/uncompress.c
index c284bcc6ba5649419a226af14a78f42142cf5e74..6a4e5a34df8a69d22ae51b9fb7423f7a052ef480 100644
--- a/lib/uncompress.c
+++ b/lib/uncompress.c
@@ -45,6 +45,7 @@ static long uncompress_fill(void *buf, unsigned long len)
 		int now = min(len, uncompress_size);
 
 		memcpy(buf, uncompress_buf, now);
+		uncompress_buf += now;
 		uncompress_size -= now;
 		len -= now;
 		total = now;
@@ -77,17 +78,17 @@ int uncompress(unsigned char *inbuf, long len,
             void(*error)(char *x));
 	int ret;
 	char *err;
+	void *uncompress_buf_free = NULL;
 
 	if (inbuf) {
 		ft = file_detect_compression_type(inbuf, len);
-		uncompress_buf = NULL;
 		uncompress_size = 0;
 	} else {
 		if (!fill)
 			return -EINVAL;
 
 		uncompress_fill_fn = fill;
-		uncompress_buf = xzalloc(32);
+		uncompress_buf_free = uncompress_buf = xzalloc(32);
 		uncompress_size = 32;
 
 		ret = fill(uncompress_buf, 32);
@@ -142,7 +143,7 @@ int uncompress(unsigned char *inbuf, long len,
 	ret = compfn(inbuf, len, fill ? uncompress_fill : NULL,
 			flush, output, pos, error_fn);
 err:
-	free(uncompress_buf);
+	free(uncompress_buf_free);
 
 	return ret;
 }

-- 
2.39.5

Re: [PATCH 0/2] uncompress: lz4 fixes

[Sascha Hauer]

On Thu, 22 May 2025 20:59:03 +0200, Sascha Hauer wrote:
> This fixes two bugs in LZ4 decompression. The first is a bug we derived
> from Linux and was fixed there back in 2014. The second is a general bug
> in uncompress_fd_to_fd() but seems to hit only in the LZ4 case. Both
> bugs are relevant only for the uncompress command, but not when LZ4 is
> used for compiled in binaries.
> 
> 
> [...]

Applied, thanks!

[1/2] initrd: fix lz4 decompress with initrd
      https://git.pengutronix.de/cgit/barebox/commit/?id=29830209ee3e (link may not be stable)
[2/2] uncompress: fix prebuffering
      https://git.pengutronix.de/cgit/barebox/commit/?id=766be5ec3b0a (link may not be stable)

Best regards,
-- 
Sascha Hauer <s.hauer@pengutronix.de>