* [PATCH 1/2] initrd: fix lz4 decompress with initrd
2025-05-22 18:59 [PATCH 0/2] uncompress: lz4 fixes Sascha Hauer
@ 2025-05-22 18:59 ` Sascha Hauer
2025-05-22 18:59 ` [PATCH 2/2] uncompress: fix prebuffering Sascha Hauer
2025-05-26 13:48 ` [PATCH 0/2] uncompress: lz4 fixes Sascha Hauer
2 siblings, 0 replies; 4+ messages in thread
From: Sascha Hauer @ 2025-05-22 18:59 UTC (permalink / raw)
To: BAREBOX
This is an adoption of the corresponding Linux commit:
| commit 4d4b866aee039d609c0b40e7e5b27204607ce614
| Author: Yinghai Lu <yinghai@kernel.org>
| Date: Fri Aug 8 14:23:10 2014 -0700
|
| During testing initrd (>2G) support, find decompress/lz4 does not work
| with initrd at all.
|
| decompress_* should support:
| 1. inbuf[]/outbuf[] for kernel preboot.
| 2. inbuf[]/flush() for initramfs
| 3. fill()/flush() for initrd.
|
| in the unlz4 does not handle case 3, as input len is passed as 0, and it
| failed in first try.
|
| Fix that add one extra if (fill) checking, and get out if EOF from the
| fill().
|
| Signed-off-by: Yinghai Lu <yinghai@kernel.org>
| Cc: Kyungsik Lee <kyungsik.lee@lge.com>
| Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
| Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
In barebox case 3 is used when called from uncompress_fd_to_fd() which
is used in the uncompress command. This seems to never have worked with
LZ4 compressed files.
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
---
lib/decompress_unlz4.c | 65 +++++++++++++++++++++++++++++++++-----------------
1 file changed, 43 insertions(+), 22 deletions(-)
diff --git a/lib/decompress_unlz4.c b/lib/decompress_unlz4.c
index a18e6591e9fd1e63158ac58c6d265df4f002fdf5..2bb30e71bb1ca1d73ae20f815f0a2be93d4ea7da 100644
--- a/lib/decompress_unlz4.c
+++ b/lib/decompress_unlz4.c
@@ -90,13 +90,20 @@ static inline int unlz4(u8 *input, long in_len,
if (posp)
*posp = 0;
- if (fill)
- fill(inp, 4);
+ if (fill) {
+ size = fill(inp, 4);
+ if (size < 4) {
+ error("data corrupted");
+ goto exit_2;
+ }
+ }
chunksize = get_unaligned_le32(inp);
if (chunksize == ARCHIVE_MAGICNUMBER) {
- inp += 4;
- size -= 4;
+ if (!fill) {
+ inp += 4;
+ size -= 4;
+ }
} else {
error("invalid header");
goto exit_2;
@@ -107,29 +114,44 @@ static inline int unlz4(u8 *input, long in_len,
for (;;) {
- if (fill)
- fill(inp, 4);
+ if (fill) {
+ size = fill(inp, 4);
+ if (size == 0)
+ break;
+ if (size < 4) {
+ error("data corrupted");
+ goto exit_2;
+ }
+ }
chunksize = get_unaligned_le32(inp);
if (chunksize == ARCHIVE_MAGICNUMBER) {
- inp += 4;
- size -= 4;
+ if (!fill) {
+ inp += 4;
+ size -= 4;
+ }
if (posp)
*posp += 4;
continue;
}
- inp += 4;
- size -= 4;
+
if (posp)
*posp += 4;
- if (fill) {
+ if (!fill) {
+ inp += 4;
+ size -= 4;
+ } else {
if (chunksize > lz4_compressbound(uncomp_chunksize)) {
error("chunk length is longer than allocated");
goto exit_2;
}
- fill(inp, chunksize);
+ size = fill(inp, chunksize);
+ if (size < chunksize) {
+ error("data corrupted");
+ goto exit_2;
+ }
}
#ifdef PREBOOT
if (out_len >= uncomp_chunksize) {
@@ -155,18 +177,17 @@ static inline int unlz4(u8 *input, long in_len,
if (posp)
*posp += chunksize;
- size -= chunksize;
+ if (!fill) {
+ size -= chunksize;
- if (size == 0)
- break;
- else if (size < 0) {
- error("data corrupted");
- goto exit_2;
+ if (size == 0)
+ break;
+ else if (size < 0) {
+ error("data corrupted");
+ goto exit_2;
+ }
+ inp += chunksize;
}
-
- inp += chunksize;
- if (fill)
- inp = inp_start;
}
ret = 0;
--
2.39.5
^ permalink raw reply [flat|nested] 4+ messages in thread* [PATCH 2/2] uncompress: fix prebuffering
2025-05-22 18:59 [PATCH 0/2] uncompress: lz4 fixes Sascha Hauer
2025-05-22 18:59 ` [PATCH 1/2] initrd: fix lz4 decompress with initrd Sascha Hauer
@ 2025-05-22 18:59 ` Sascha Hauer
2025-05-26 13:48 ` [PATCH 0/2] uncompress: lz4 fixes Sascha Hauer
2 siblings, 0 replies; 4+ messages in thread
From: Sascha Hauer @ 2025-05-22 18:59 UTC (permalink / raw)
To: BAREBOX
When uncompressing from a fill function we prebuffer the first 32 bytes
in order to detect the filetype. This prebuffer is then fed back into
the input stream in uncompress_fill().
This currently fails when uncompress_fill() crosses the prebuffered
size and continues reading from the original fill function. Advance
the buffer in this case to append the data to the buffer instead of
overwriting the data we already copied from the prebuffer.
Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
---
lib/uncompress.c | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/lib/uncompress.c b/lib/uncompress.c
index c284bcc6ba5649419a226af14a78f42142cf5e74..6a4e5a34df8a69d22ae51b9fb7423f7a052ef480 100644
--- a/lib/uncompress.c
+++ b/lib/uncompress.c
@@ -45,6 +45,7 @@ static long uncompress_fill(void *buf, unsigned long len)
int now = min(len, uncompress_size);
memcpy(buf, uncompress_buf, now);
+ uncompress_buf += now;
uncompress_size -= now;
len -= now;
total = now;
@@ -77,17 +78,17 @@ int uncompress(unsigned char *inbuf, long len,
void(*error)(char *x));
int ret;
char *err;
+ void *uncompress_buf_free = NULL;
if (inbuf) {
ft = file_detect_compression_type(inbuf, len);
- uncompress_buf = NULL;
uncompress_size = 0;
} else {
if (!fill)
return -EINVAL;
uncompress_fill_fn = fill;
- uncompress_buf = xzalloc(32);
+ uncompress_buf_free = uncompress_buf = xzalloc(32);
uncompress_size = 32;
ret = fill(uncompress_buf, 32);
@@ -142,7 +143,7 @@ int uncompress(unsigned char *inbuf, long len,
ret = compfn(inbuf, len, fill ? uncompress_fill : NULL,
flush, output, pos, error_fn);
err:
- free(uncompress_buf);
+ free(uncompress_buf_free);
return ret;
}
--
2.39.5
^ permalink raw reply [flat|nested] 4+ messages in thread* Re: [PATCH 0/2] uncompress: lz4 fixes
2025-05-22 18:59 [PATCH 0/2] uncompress: lz4 fixes Sascha Hauer
2025-05-22 18:59 ` [PATCH 1/2] initrd: fix lz4 decompress with initrd Sascha Hauer
2025-05-22 18:59 ` [PATCH 2/2] uncompress: fix prebuffering Sascha Hauer
@ 2025-05-26 13:48 ` Sascha Hauer
2 siblings, 0 replies; 4+ messages in thread
From: Sascha Hauer @ 2025-05-26 13:48 UTC (permalink / raw)
To: BAREBOX, Sascha Hauer
On Thu, 22 May 2025 20:59:03 +0200, Sascha Hauer wrote:
> This fixes two bugs in LZ4 decompression. The first is a bug we derived
> from Linux and was fixed there back in 2014. The second is a general bug
> in uncompress_fd_to_fd() but seems to hit only in the LZ4 case. Both
> bugs are relevant only for the uncompress command, but not when LZ4 is
> used for compiled in binaries.
>
>
> [...]
Applied, thanks!
[1/2] initrd: fix lz4 decompress with initrd
https://git.pengutronix.de/cgit/barebox/commit/?id=29830209ee3e (link may not be stable)
[2/2] uncompress: fix prebuffering
https://git.pengutronix.de/cgit/barebox/commit/?id=766be5ec3b0a (link may not be stable)
Best regards,
--
Sascha Hauer <s.hauer@pengutronix.de>
^ permalink raw reply [flat|nested] 4+ messages in thread