[PATCH 0/4] test: py: add FIT image boot test

[PATCH 0/4] test: py: add FIT image boot test

[Ahmad Fatoum]

This series assembles a FIT image with a compressed barebox, an
uncompressed device tree and a small initramfs.

Then the newly introduced test_fit.py checks that it can be booted
and device tree is fixed up with command line, initrd location,
generic and custom fixups.

It already caught a regression in next that broke initrd loading on
ARM64 & RISC-V and it can also detect the FIT image compression
regression in v2025.05.0 that was fixed by Stefan recently.


Ahmad Fatoum (4):
  ci: pytest: mount virtfs with signed FIT image into VMs
  test: py: add FIT boot test
  ci: pytest: multi_v8_defconfig: pass --runxfail
  ci: container: add device-tree-compiler

 .github/testfs/multi_v7_defconfig-gzipped.its | 57 +++++++++++++++++
 .github/testfs/multi_v8_defconfig-gzipped.its | 57 +++++++++++++++++
 .github/workflows/test-labgrid-pytest.yml     | 22 ++++++-
 scripts/generate_testfs.sh                    | 26 ++++++++
 test/Containerfile                            |  1 +
 test/arm/virt@multi_v7_defconfig.yaml         |  1 +
 test/arm/virt@multi_v8_defconfig.yaml         |  1 +
 test/py/helper.py                             | 15 +++++
 test/py/test_fit.py                           | 63 +++++++++++++++++++
 9 files changed, 242 insertions(+), 1 deletion(-)
 create mode 100644 .github/testfs/multi_v7_defconfig-gzipped.its
 create mode 100644 .github/testfs/multi_v8_defconfig-gzipped.its
 create mode 100755 scripts/generate_testfs.sh
 create mode 100644 test/py/test_fit.py

-- 
2.39.5

[PATCH 1/4] ci: pytest: mount virtfs with signed FIT image into VMs

[Ahmad Fatoum]

This is in preparation for adding a FIT boot pytest.

We intentionally don't use scripts/make_fit.py as its compression is
broken and we want to sign the FIT image, so we can test that in future
too.

Signed-off-by: Ahmad Fatoum <a.fatoum@barebox.org>
---
 .github/testfs/multi_v7_defconfig-gzipped.its | 57 +++++++++++++++++++
 .github/testfs/multi_v8_defconfig-gzipped.its | 57 +++++++++++++++++++
 .github/workflows/test-labgrid-pytest.yml     | 20 ++++++-
 scripts/generate_testfs.sh                    | 26 +++++++++
 4 files changed, 159 insertions(+), 1 deletion(-)
 create mode 100644 .github/testfs/multi_v7_defconfig-gzipped.its
 create mode 100644 .github/testfs/multi_v8_defconfig-gzipped.its
 create mode 100755 scripts/generate_testfs.sh

diff --git a/.github/testfs/multi_v7_defconfig-gzipped.its b/.github/testfs/multi_v7_defconfig-gzipped.its
new file mode 100644
index 000000000000..11595c39baa1
--- /dev/null
+++ b/.github/testfs/multi_v7_defconfig-gzipped.its
@@ -0,0 +1,57 @@
+/dts-v1/;
+
+/ {
+    description = "fitImage for testing";
+    #address-cells = <1>;
+    images {
+        kernel-1 {
+            description = "barebox image";
+            data = /incbin/("barebox-dt-2nd.img.gz");
+            type = "kernel";
+            arch = "arm";
+            os = "linux";
+            compression = "gzip";
+            hash-1 {
+                algo = "sha256";
+            };
+        };
+        fdt-qemu-virt32.dtb {
+            description = "Flattened Device Tree blob";
+            data = /incbin/("arch/arm/dts/qemu-virt32.dtb");
+            type = "flat_dt";
+            arch = "arm";
+            compression = "none";
+            hash-1 {
+                algo = "sha256";
+            };
+        };
+        ramdisk-1 {
+            description = "initramfs";
+            data = /incbin/("ramdisk.cpio.gz");
+            type = "ramdisk";
+            arch = "arm";
+            os = "linux";
+            compression = "none";
+            hash-1 {
+                algo = "sha256";
+            };
+        };
+    };
+
+    configurations {
+        default = "conf-qemu-virt32.dtb";
+        conf-qemu-virt32.dtb {
+            description = "Qemu Virt32";
+            kernel = "kernel-1";
+            fdt = "fdt-qemu-virt32.dtb";
+            ramdisk = "ramdisk-1";
+            compatible = "linux,dummy-virt";
+            signature-1 {
+                algo = "sha256,rsa2048";
+                sign-images = "kernel", "fdt", "ramdisk";
+                signer-name = "None";
+                signer-version = "None";
+            };
+        };
+    };
+};
diff --git a/.github/testfs/multi_v8_defconfig-gzipped.its b/.github/testfs/multi_v8_defconfig-gzipped.its
new file mode 100644
index 000000000000..5ce678eec9a8
--- /dev/null
+++ b/.github/testfs/multi_v8_defconfig-gzipped.its
@@ -0,0 +1,57 @@
+/dts-v1/;
+
+/ {
+    description = "fitImage for testing";
+    #address-cells = <1>;
+    images {
+        kernel-1 {
+            description = "barebox image";
+            data = /incbin/("barebox-dt-2nd.img.gz");
+            type = "kernel";
+            arch = "arm64";
+            os = "linux";
+            compression = "gzip";
+            hash-1 {
+                algo = "sha256";
+            };
+        };
+        fdt-qemu-virt64.dtb {
+            description = "Flattened Device Tree blob";
+            data = /incbin/("arch/arm/dts/qemu-virt64.dtb");
+            type = "flat_dt";
+            arch = "arm64";
+            compression = "none";
+            hash-1 {
+                algo = "sha256";
+            };
+        };
+        ramdisk-1 {
+            description = "initramfs";
+            data = /incbin/("ramdisk.cpio.gz");
+            type = "ramdisk";
+            arch = "arm64";
+            os = "linux";
+            compression = "none";
+            hash-1 {
+                algo = "sha256";
+            };
+        };
+    };
+
+    configurations {
+        default = "conf-qemu-virt64.dtb";
+        conf-qemu-virt64.dtb {
+            description = "Qemu Virt64";
+            kernel = "kernel-1";
+            fdt = "fdt-qemu-virt64.dtb";
+            ramdisk = "ramdisk-1";
+            compatible = "linux,dummy-virt";
+            signature-1 {
+                algo = "sha256,rsa2048";
+                sign-images = "kernel", "fdt", "ramdisk";
+                signer-name = "None";
+                signer-version = "None";
+            };
+        };
+    };
+};
diff --git a/.github/workflows/test-labgrid-pytest.yml b/.github/workflows/test-labgrid-pytest.yml
index d4043de961b2..64618582a940 100644
--- a/.github/workflows/test-labgrid-pytest.yml
+++ b/.github/workflows/test-labgrid-pytest.yml
@@ -80,6 +80,20 @@ jobs:
         if [ ${{matrix.arch}} = "riscv" ]; then
           cp /usr/share/qemu/opensbi-riscv32-generic-fw_dynamic.bin ${KBUILD_OUTPUT}/
         fi
+
+    - name: Populate testfs
+      run: |
+        export KBUILD_OUTPUT=build-${{matrix.arch}}-${{matrix.defconfig}}
+        export KBUILD_DEFCONFIG=${{matrix.defconfig}}
+
+        for i in ${{matrix.lgenv}}; do
+          grep -wq 'testfs' "$i" || continue
+
+          # Just use already built dtc
+          export PATH="$PATH:${KBUILD_OUTPUT}/scripts/dtc/"
+          exec scripts/generate_testfs.sh
+        done
+
     - name: labgrid-pytest
       run: |
         export KBUILD_OUTPUT=build-${{matrix.arch}}-${{matrix.defconfig}}
@@ -87,10 +101,14 @@ jobs:
         for i in ${{matrix.lgenv}}; do
           grep -wq '\(QEMUDriver\|ExternalConsoleDriver\):' "$i" || continue
 
+          if grep -wq 'testfs' "$i"; then
+            extraargs="--fs testfs=${KBUILD_OUTPUT}/testfs"
+          fi
+
           cfg=$(basename $i .yaml)
           echo "Testing $cfg"
           labgrid-pytest --lg-env $i test/py --verbosity=1 \
-          --junitxml=$cfg.tests.xml --lg-log=log/$cfg
+          --junitxml=$cfg.tests.xml --lg-log=log/$cfg $extraargs
         done
 
     - name: Publish Test Results
diff --git a/scripts/generate_testfs.sh b/scripts/generate_testfs.sh
new file mode 100755
index 000000000000..c5d24f7e1af8
--- /dev/null
+++ b/scripts/generate_testfs.sh
@@ -0,0 +1,26 @@
+#!/bin/sh
+
+MKIMAGE=${MKIMAGE:-mkimage}
+KGZIP=${KGZIP:-gzip}
+
+set -e
+
+if [ -z "${KBUILD_OUTPUT}" ] || [ -z "${KBUILD_DEFCONFIG}" ] ; then
+	2>&1 echo "KBUILD_OUTPUT and KBUILD_DEFCONFIG must be set"
+	exit 1
+fi
+
+rm -rf "${KBUILD_OUTPUT}/testfs/"
+mkdir -p ${KBUILD_OUTPUT}/testfs
+
+cat ${KBUILD_OUTPUT}/images/barebox-dt-2nd.img | \
+  ${KGZIP} -n -f -9 >${KBUILD_OUTPUT}/barebox-dt-2nd.img.gz
+
+cp .github/testfs/${KBUILD_DEFCONFIG}-gzipped.its ${KBUILD_OUTPUT}/
+
+find COPYING LICENSES/ | cpio -o -H newc | ${KGZIP} \
+  > ${KBUILD_OUTPUT}/ramdisk.cpio.gz
+
+${MKIMAGE} -G $PWD/test/self/development_rsa2048.pem -r \
+      -f ${KBUILD_OUTPUT}/${KBUILD_DEFCONFIG}-gzipped.its \
+	 ${KBUILD_OUTPUT}/testfs/barebox-gzipped.fit
-- 
2.39.5

[PATCH 2/4] test: py: add FIT boot test

[Ahmad Fatoum]

This test verifies that FIT image can be booted and device tree is fixed
up with command line, initrd location, generic and custom fixups.

It already caught a regression in next that broke initrd loading on
ARM64 & RISC-V.

Cc: Stefan Kerkmann <s.kerkmann@pengutronix.de>
Signed-off-by: Ahmad Fatoum <a.fatoum@barebox.org>
---
 test/arm/virt@multi_v7_defconfig.yaml |  1 +
 test/arm/virt@multi_v8_defconfig.yaml |  1 +
 test/py/helper.py                     | 15 +++++++
 test/py/test_fit.py                   | 63 +++++++++++++++++++++++++++
 4 files changed, 80 insertions(+)
 create mode 100644 test/py/test_fit.py

diff --git a/test/arm/virt@multi_v7_defconfig.yaml b/test/arm/virt@multi_v7_defconfig.yaml
index 203f17bfc7f0..637d132d3967 100644
--- a/test/arm/virt@multi_v7_defconfig.yaml
+++ b/test/arm/virt@multi_v7_defconfig.yaml
@@ -14,6 +14,7 @@ targets:
       BareboxTestStrategy: {}
     features:
       - virtio-mmio
+      - testfs
 images:
   barebox-dt-2nd.img: !template "$LG_BUILDDIR/images/barebox-dt-2nd.img"
 imports:
diff --git a/test/arm/virt@multi_v8_defconfig.yaml b/test/arm/virt@multi_v8_defconfig.yaml
index 42ce10328d7c..d018af06aa53 100644
--- a/test/arm/virt@multi_v8_defconfig.yaml
+++ b/test/arm/virt@multi_v8_defconfig.yaml
@@ -15,6 +15,7 @@ targets:
     features:
       - virtio-mmio
       - network
+      - testfs
     runner:
       tuxmake_arch: arm64
 images:
diff --git a/test/py/helper.py b/test/py/helper.py
index 4a68e83669ba..cfcc9c337fec 100644
--- a/test/py/helper.py
+++ b/test/py/helper.py
@@ -30,6 +30,21 @@ def get_config(command):
     return options
 
 
+def of_get_property(barebox, path):
+    node, prop = os.path.split(path)
+
+    stdout = barebox.run_check(f"of_dump -p {node}")
+    for line in stdout:
+        if line == '{prop};':
+            return True
+
+        prefix = f'{prop} = '
+        if line.startswith(prefix):
+            # Also drop the semicolon
+            return line[len(prefix):-1]
+    return False
+
+
 def skip_disabled(config, *options):
     if bool(config):
         undefined = list(filterfalse(config.__contains__, options))
diff --git a/test/py/test_fit.py b/test/py/test_fit.py
new file mode 100644
index 000000000000..c53a1ece1445
--- /dev/null
+++ b/test/py/test_fit.py
@@ -0,0 +1,63 @@
+# SPDX-License-Identifier: GPL-2.0-or-later
+
+import re
+import pytest
+from .helper import of_get_property
+
+
+def fit_name(suffix):
+    return f"/mnt/9p/testfs/barebox-{suffix}.fit"
+
+
+def generate_bootscript(barebox, image, name="test"):
+    barebox.run_check(f"echo -o /env/boot/{name} '#!/bin/sh'")
+    barebox.run_check(f"echo -a /env/boot/{name}  bootm {image}")
+    return name
+
+
+def test_fit(barebox, env, target, barebox_config):
+    if 'testfs' not in env.get_target_features():
+        pytest.xfail("testfs feature not specified")
+
+    _, _, returncode = barebox.run("ls /mnt/9p/testfs")
+    if returncode != 0:
+        pytest.xfail("skipping test due to missing --fs testfs=")
+
+    barebox.run_check(f"ls {fit_name('gzipped')}")
+
+    # Sanity check, this is only fixed up on first boot
+    assert of_get_property(barebox, "/chosen/barebox-version") is False
+    [ver] = barebox.run_check("echo barebox-$global.version")
+    assert ver.startswith('barebox-2')
+
+    barebox.run_check("of_property -s /chosen barebox,boot-count '<0x0>'")
+    assert of_get_property(barebox, "/chosen/barebox,boot-count") == '<0x0>'
+
+    barebox.run_check("of_property -fs /chosen barebox,boot-count '<0x1>'")
+    assert of_get_property(barebox, "/chosen/barebox,boot-count") == '<0x0>'
+
+    barebox.run_check("global linux.bootargs.testarg=barebox.chainloaded")
+
+    boottarget = generate_bootscript(barebox, fit_name('gzipped'))
+    barebox.boot(boottarget)
+    target.deactivate(barebox)
+    target.activate(barebox)
+
+    assert of_get_property(barebox, "/chosen/barebox-version") == f'"{ver}"', \
+           "/chosen/barebox-version suggests we did not chainload"
+
+    assert of_get_property(barebox, "/chosen/barebox,boot-count") == '<0x1>', \
+           "/chosen/barebox,boot-count suggests we got bultin DT"
+
+    # Check that command line arguments were fixed up
+    bootargs = of_get_property(barebox, "/chosen/bootargs")
+    assert "barebox.chainloaded" in bootargs
+
+    initrd_start = of_get_property(barebox, "/chosen/linux,initrd-start")
+    initrd_end = of_get_property(barebox, "/chosen/linux,initrd-end")
+
+    addr_regex = r"<(0x[0-9a-f]{1,8} ?)+>"
+    assert re.search(addr_regex, initrd_start), \
+        f"initrd start {initrd_start} malformed"
+    assert re.search(addr_regex, initrd_end), \
+        f"initrd end {initrd_end} malformed"
-- 
2.39.5

[PATCH 3/4] ci: pytest: multi_v8_defconfig: pass --runxfail

[Ahmad Fatoum]

We currently use xfail in test_network.py to skip the network tests if
the networking feature is not enabled. For ARMv8, the CI sets up
networking and the feature is set, so pass it --runxfail, so unintended
xfails don't creep in.

Cc: Jonas Rebmann <jre@pengutronix.de>
Signed-off-by: Ahmad Fatoum <a.fatoum@barebox.org>
---
 .github/workflows/test-labgrid-pytest.yml | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/.github/workflows/test-labgrid-pytest.yml b/.github/workflows/test-labgrid-pytest.yml
index 64618582a940..38bd7126bd86 100644
--- a/.github/workflows/test-labgrid-pytest.yml
+++ b/.github/workflows/test-labgrid-pytest.yml
@@ -26,6 +26,7 @@ jobs:
           - ARCH: arm
             lgenv: test/arm/multi_v8_defconfig.yaml
             defconfig: multi_v8_defconfig
+            lgargs: --runxfail
 
           - ARCH: mips
             lgenv: test/mips/qemu-malta_defconfig.yaml
@@ -101,8 +102,9 @@ jobs:
         for i in ${{matrix.lgenv}}; do
           grep -wq '\(QEMUDriver\|ExternalConsoleDriver\):' "$i" || continue
 
+          extraargs="${{matrix.lgargs}}"
           if grep -wq 'testfs' "$i"; then
-            extraargs="--fs testfs=${KBUILD_OUTPUT}/testfs"
+            extraargs="${extraargs} --fs testfs=${KBUILD_OUTPUT}/testfs"
           fi
 
           cfg=$(basename $i .yaml)
-- 
2.39.5

[PATCH 4/4] ci: container: add device-tree-compiler

[Ahmad Fatoum]

For use with mkimage, we need dtc in the container image as well.
This allows us to avoid having to change PATH to point at our vendored
version.

Signed-off-by: Ahmad Fatoum <a.fatoum@barebox.org>
---
 test/Containerfile | 1 +
 1 file changed, 1 insertion(+)

diff --git a/test/Containerfile b/test/Containerfile
index 2da786c69925..644dc63ad292 100644
--- a/test/Containerfile
+++ b/test/Containerfile
@@ -51,6 +51,7 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
 	python3-yaml \
 	virtualenv \
 	sudo \
+	device-tree-compiler \
 	u-boot-tools \
 	yamllint \
 	yq \
-- 
2.39.5