Re: [PATCH 3/7] test: move dm-verity testdata generation to fixture

[Jonas Rebmann <jre@pengutronix.de>]

Hi all,

On 2025-09-26 12:14, Jonas Rebmann wrote:
> Simplify barebox integration test setup by moving logic away from
> scripts/ and .github/.
> 
> Instead of generating testdata in separate scripts, they should be
> implemented as testfs fixtures which are automatically ran as part of
> the test suite.
> 
> Includes error handling and cleanup.
> 
> Signed-off-by: Jonas Rebmann <jre@pengutronix.de>
> ---
>   scripts/generate_testfs.sh | 44 ---------------------------------------
>   test/py/test_dm.py         | 51 ++++++++++++++++++++++++++++++++++++++++++++--
>   2 files changed, 49 insertions(+), 46 deletions(-)
> 
> diff --git a/scripts/generate_testfs.sh b/scripts/generate_testfs.sh
> index 1c358ff846..3c200bd401 100755
> --- a/scripts/generate_testfs.sh
> +++ b/scripts/generate_testfs.sh
> @@ -28,47 +28,3 @@ generate_fit()
>   	       ${KBUILD_OUTPUT}/testfs/barebox-gzipped.fit
>   }
>   [ -f .github/testfs/${KBUILD_DEFCONFIG}-gzipped.its ] && generate_fit

It took me some time to figure out why this patch broke CI.

With the set -e in this script, this last line of fit generation logic
should have been

   [ -f .github/testfs/${KBUILD_DEFCONFIG}-gzipped.its ] && generate_fit || true

given the intent here is instead of erroring out, to just skip fit
generation if the its does not exist.

The only reason CI didn't break when Tobias added mipsel and riscv, for
which no .github/testfs/${KBUILD_DEFCONFIG}-gzipped.its exist, in commit
739a1a7855f5 ("ci: pytest: Open up testfs to more consumers than the FIT
test"), is that the github workflow (incorrectly) invokes the script
with exec. This is why only when this check became the last line of the
script, the error return code propagated into github ci.

I would like to drop scripts/generate_testfs.sh entirely in v2,
migrating the generate_fit logic to a pytest fixture too.

The reason I haven't yet is mainly that there we cannot rely on
KBUILD_DEFCONFIG to select the test data.

I thought about selecting the its from something in barebox_config or
something in strategy.qemu instead. Any suggestions?

And I'd like to also move those testfs files out of .github. I would
like to consider the pytest tests something more than just artifacts of
the github workflow.

As this excursion into cleanup around the integration tests is getting a
bit more involved than I hoped, I'm happy about feedback already while
I'm working on v2.

> -
> -alias pad128k="dd if=/dev/zero bs=128k count=1 status=none"
> -
> -generate_dm_verity()
> -{
> -    work=$(mktemp -d)
> -    cd ${work}
> -
> -    # Create two dummy files; use lots of padding to make sure that
> -    # when we alter the contents of 'english' in root-bad, 'latin' is
> -    # still be readable, as their contents wont (a) share the same
> -    # hash block and (b) the block cache layer won't accedentally read
> -    # the invalid block.
> -
> -    pad128k  >latin
> -    echo -n "veritas vos liberabit" >>latin
> -    pad128k >>latin
> -
> -    pad128k  >english
> -    echo -n "truth will set you free" >>english
> -    pad128k >>english
> -
> -    truncate -s 1M good.fat
> -    mkfs.vfat good.fat
> -    mcopy -i good.fat latin english ::
> -
> -    veritysetup format \
> -		--root-hash-file=good.hash \
> -		good.fat good.verity
> -
> -    sed 's/truth will set you free/LIAR LIAR PANTS ON FIRE/' \
> -	<good.fat >bad.fat
> -
> -    cd -
> -    cp \
> -	${work}/good.fat \
> -	${work}/good.verity \
> -	${work}/good.hash \
> -	${work}/bad.fat \
> -	${KBUILD_OUTPUT}/testfs
> -
> -    rm -rf ${work}
> -}
> -generate_dm_verity
> diff --git a/test/py/test_dm.py b/test/py/test_dm.py
> index 8bdacf7b4e..837881f4af 100644
> --- a/test/py/test_dm.py
> +++ b/test/py/test_dm.py
> @@ -4,10 +4,57 @@ import re
>   import pytest
>   from .helper import of_get_property
>   
> +import os, subprocess, shutil
>   
> +def pad128k(f):
> +    f.write(b"\0" * 128 * 1024)
>   
> -def test_dm_verity(barebox, testfs):
> -    barebox.run_check("cd /mnt/9p/testfs")
> +@pytest.fixture(scope="module")
> +def dm_testdata(testfs):
> +    path = os.path.join(testfs, "dm")
> +    os.makedirs(path, exist_ok=True)
> +    cwd = os.getcwd()
> +    os.chdir(path)
> +
> +    with open("latin", "wb") as f:
> +        pad128k(f)
> +        f.write(b"veritas vos liberabit")
> +        pad128k(f)
> +
> +    with open("english", "wb") as f:
> +        pad128k(f)
> +        f.write(b"truth will set you free")
> +        pad128k(f)
> +
> +    try:
> +        subprocess.run(["truncate", "-s", "1M", "good.fat"], check=True)
> +        subprocess.run(["mkfs.vfat", "good.fat"], check=True)
> +        subprocess.run(["mcopy", "-i", "good.fat", "latin", "english", "::"], check=True)
> +
> +        subprocess.run([
> +            "veritysetup", "format",
> +            "--root-hash-file=good.hash",
> +            "good.fat", "good.verity"
> +        ], check=True)
> +    except FileNotFoundError as e:
> +        pytest.skip(f"Skip dm tests due to missing dependency: {e}")
> +
> +    with open("good.fat", "rb") as f: data = f.read()
> +    with open("bad.fat", "wb") as f:
> +        f.write(data.replace(
> +            b"truth will set you free",
> +            b"LIAR LIAR PANTS ON FIRE"
> +        ))
> +
> +    os.chdir(cwd)
> +
> +    yield path
> +
> +    shutil.rmtree(path)
> +
> +
> +def test_dm_verity(barebox, dm_testdata):
> +    barebox.run_check("cd /mnt/9p/testfs/dm")
>   
>       # Since commands run in a subshell, export the root hash in a
>       # global, so that we can access it from subsequent commands
> 

Regards,
Jonas

-- 
Pengutronix e.K.                           | Jonas Rebmann               |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-9    |