[PATCH 0/4] ci: fix sandbox allyesconfig build

[PATCH 0/4] ci: fix sandbox allyesconfig build

[Ahmad Fatoum]

With these changes applied, the most recent container image can be used
to run the pytest tests for allyesconfig.

Ahmad Fatoum (4):
  sandbox: env: switch from CONFIG_DEFAULT_ENVIRONMENT_PATH to bbenv-y
  test: bthread: change comparison condition for switches vs. yields
  ci: pytest: rename build directories
  ci: pytest: enable console and libc allocator for allyesconfig

 .github/workflows/test-labgrid-pytest.yml        | 16 ++++++++++------
 arch/sandbox/board/Makefile                      |  1 +
 arch/sandbox/board/board.c                       |  3 +++
 .../{env => defaultenv-sandbox}/network/eth0     |  0
 .../{env => defaultenv-sandbox}/nv/autoboot      |  0
 arch/sandbox/configs/sandbox_defconfig           |  2 +-
 test/py/test_bthread.py                          |  3 ++-
 7 files changed, 17 insertions(+), 8 deletions(-)
 rename arch/sandbox/board/{env => defaultenv-sandbox}/network/eth0 (100%)
 rename arch/sandbox/board/{env => defaultenv-sandbox}/nv/autoboot (100%)

-- 
2.39.5

[PATCH 1/4] sandbox: env: switch from CONFIG_DEFAULT_ENVIRONMENT_PATH to bbenv-y

[Ahmad Fatoum]

By moving it out of the config, the sandbox default environment is always
there, even when building allyesconfig.

This introduces no functional change for sandbox_defconfig, but
is useful as we start running tests against allyesconfig and want
the same environment (autoboot abortion by default) in any case.

Signed-off-by: Ahmad Fatoum <a.fatoum@pengutronix.de>
---
 arch/sandbox/board/Makefile                                 | 1 +
 arch/sandbox/board/board.c                                  | 3 +++
 arch/sandbox/board/{env => defaultenv-sandbox}/network/eth0 | 0
 arch/sandbox/board/{env => defaultenv-sandbox}/nv/autoboot  | 0
 arch/sandbox/configs/sandbox_defconfig                      | 2 +-
 5 files changed, 5 insertions(+), 1 deletion(-)
 rename arch/sandbox/board/{env => defaultenv-sandbox}/network/eth0 (100%)
 rename arch/sandbox/board/{env => defaultenv-sandbox}/nv/autoboot (100%)

diff --git a/arch/sandbox/board/Makefile b/arch/sandbox/board/Makefile
index 029bfe7ff163..2b6346f6098d 100644
--- a/arch/sandbox/board/Makefile
+++ b/arch/sandbox/board/Makefile
@@ -12,6 +12,7 @@ obj-y += dev-random.o
 obj-y += watchdog.o
 obj-$(CONFIG_CMD_SANDBOX_CPUINFO) += cpuinfo.o
 obj-$(CONFIG_LED) += led.o
+bbenv-y += defaultenv-sandbox
 
 extra-y += barebox.lds
 
diff --git a/arch/sandbox/board/board.c b/arch/sandbox/board/board.c
index c8d1c99897f5..58a7b8c96f62 100644
--- a/arch/sandbox/board/board.c
+++ b/arch/sandbox/board/board.c
@@ -15,6 +15,7 @@
  */
 
 #include <common.h>
+#include <envfs.h>
 #include <driver.h>
 #include <malloc.h>
 #include <mach/linux.h>
@@ -59,6 +60,8 @@ static int devices_init(struct device *dev)
 
 	platform_device_register(&devrandom_device);
 
+	defaultenv_append_directory(defaultenv_sandbox);
+
 	return 0;
 }
 
diff --git a/arch/sandbox/board/env/network/eth0 b/arch/sandbox/board/defaultenv-sandbox/network/eth0
similarity index 100%
rename from arch/sandbox/board/env/network/eth0
rename to arch/sandbox/board/defaultenv-sandbox/network/eth0
diff --git a/arch/sandbox/board/env/nv/autoboot b/arch/sandbox/board/defaultenv-sandbox/nv/autoboot
similarity index 100%
rename from arch/sandbox/board/env/nv/autoboot
rename to arch/sandbox/board/defaultenv-sandbox/nv/autoboot
diff --git a/arch/sandbox/configs/sandbox_defconfig b/arch/sandbox/configs/sandbox_defconfig
index 251af8d0ef31..4cc9d9982c1a 100644
--- a/arch/sandbox/configs/sandbox_defconfig
+++ b/arch/sandbox/configs/sandbox_defconfig
@@ -9,7 +9,7 @@ CONFIG_PARTITION_DISK_EFI=y
 CONFIG_DEFAULT_COMPRESSION_GZIP=y
 CONFIG_DEFAULT_ENVIRONMENT_GENERIC_NEW=y
 CONFIG_DEFAULT_ENVIRONMENT_GENERIC_NEW_REBOOT_MODE=y
-CONFIG_DEFAULT_ENVIRONMENT_PATH="arch/sandbox/board/env"
+CONFIG_DEFAULT_ENVIRONMENT_PATH=""
 CONFIG_STATE=y
 CONFIG_STATE_CRYPTO=y
 CONFIG_RESET_SOURCE=y
-- 
2.39.5

[PATCH 2/4] test: bthread: change comparison condition for switches vs. yields

[Ahmad Fatoum]

The code was written with the assumption that isolated yields from test
thread to main thread and back will always be at least as fast as
unrestricted scheduling.

This assumption held so far, but breaks down in sandbox, because it's
highly influenced by the system's CPU pressure.

Just check that we can schedule @1000Hz at least in both cases.

Signed-off-by: Ahmad Fatoum <a.fatoum@pengutronix.de>
---
 test/py/test_bthread.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/test/py/test_bthread.py b/test/py/test_bthread.py
index 6e7b4ba5007e..84fb9f830b0c 100644
--- a/test/py/test_bthread.py
+++ b/test/py/test_bthread.py
@@ -20,4 +20,5 @@ def test_bthread(barebox, barebox_config):
     switches = int(barebox.run_check("bthread -c")[0].split()[0])
     yields   = int(barebox.run_check("bthread -t")[0].split()[0])
 
-    assert yields < switches
+    assert yields > 1000
+    assert yields > 1000
-- 
2.39.5

Re: [PATCH 2/4] test: bthread: change comparison condition for switches vs. yields

[Bastian Krause]

Hi Ahmad,

On 3/27/25 3:47 PM, Ahmad Fatoum wrote:
> The code was written with the assumption that isolated yields from test
> thread to main thread and back will always be at least as fast as
> unrestricted scheduling.
> 
> This assumption held so far, but breaks down in sandbox, because it's
> highly influenced by the system's CPU pressure.
> 
> Just check that we can schedule @1000Hz at least in both cases.
> 
> Signed-off-by: Ahmad Fatoum <a.fatoum@pengutronix.de>
> ---
>   test/py/test_bthread.py | 3 ++-
>   1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/test/py/test_bthread.py b/test/py/test_bthread.py
> index 6e7b4ba5007e..84fb9f830b0c 100644
> --- a/test/py/test_bthread.py
> +++ b/test/py/test_bthread.py
> @@ -20,4 +20,5 @@ def test_bthread(barebox, barebox_config):
>       switches = int(barebox.run_check("bthread -c")[0].split()[0])
>       yields   = int(barebox.run_check("bthread -t")[0].split()[0])
>   
> -    assert yields < switches
> +    assert yields > 1000
> +    assert yields > 1000

I guess one of these asserts should involve "switches", right?

Regards,
Bastian

-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

Re: [PATCH 2/4] test: bthread: change comparison condition for switches vs. yields

[Ahmad Fatoum]

Hello Bastian,

On 4/1/25 10:53, Bastian Krause wrote:
> Hi Ahmad,
> 
> On 3/27/25 3:47 PM, Ahmad Fatoum wrote:
>> The code was written with the assumption that isolated yields from test
>> thread to main thread and back will always be at least as fast as
>> unrestricted scheduling.
>>
>> This assumption held so far, but breaks down in sandbox, because it's
>> highly influenced by the system's CPU pressure.
>>
>> Just check that we can schedule @1000Hz at least in both cases.
>>
>> Signed-off-by: Ahmad Fatoum <a.fatoum@pengutronix.de>
>> ---
>>   test/py/test_bthread.py | 3 ++-
>>   1 file changed, 2 insertions(+), 1 deletion(-)
>>
>> diff --git a/test/py/test_bthread.py b/test/py/test_bthread.py
>> index 6e7b4ba5007e..84fb9f830b0c 100644
>> --- a/test/py/test_bthread.py
>> +++ b/test/py/test_bthread.py
>> @@ -20,4 +20,5 @@ def test_bthread(barebox, barebox_config):
>>       switches = int(barebox.run_check("bthread -c")[0].split()[0])
>>       yields   = int(barebox.run_check("bthread -t")[0].split()[0])
>>   -    assert yields < switches
>> +    assert yields > 1000
>> +    assert yields > 1000
> 
> I guess one of these asserts should involve "switches", right?

You guessed exactly right. I Cc'd you on the fixup.

Thanks,
Ahmad

> 
> Regards,
> Bastian
> 

-- 
Pengutronix e.K.                  |                             |
Steuerwalder Str. 21              | http://www.pengutronix.de/  |
31137 Hildesheim, Germany         | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686  | Fax:   +49-5121-206917-5555 |

Re: [PATCH 2/4] test: bthread: change comparison condition for switches vs. yields

[Bastian Krause]

On 4/1/25 11:34 AM, Ahmad Fatoum wrote:
> Hello Bastian,
> 
> On 4/1/25 10:53, Bastian Krause wrote:
>> Hi Ahmad,
>>
>> On 3/27/25 3:47 PM, Ahmad Fatoum wrote:
>>> The code was written with the assumption that isolated yields from test
>>> thread to main thread and back will always be at least as fast as
>>> unrestricted scheduling.
>>>
>>> This assumption held so far, but breaks down in sandbox, because it's
>>> highly influenced by the system's CPU pressure.
>>>
>>> Just check that we can schedule @1000Hz at least in both cases.
>>>
>>> Signed-off-by: Ahmad Fatoum <a.fatoum@pengutronix.de>
>>> ---
>>>    test/py/test_bthread.py | 3 ++-
>>>    1 file changed, 2 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/test/py/test_bthread.py b/test/py/test_bthread.py
>>> index 6e7b4ba5007e..84fb9f830b0c 100644
>>> --- a/test/py/test_bthread.py
>>> +++ b/test/py/test_bthread.py
>>> @@ -20,4 +20,5 @@ def test_bthread(barebox, barebox_config):
>>>        switches = int(barebox.run_check("bthread -c")[0].split()[0])
>>>        yields   = int(barebox.run_check("bthread -t")[0].split()[0])
>>>    -    assert yields < switches
>>> +    assert yields > 1000
>>> +    assert yields > 1000
>>
>> I guess one of these asserts should involve "switches", right?
> 
> You guessed exactly right. I Cc'd you on the fixup.

Thanks!

Regards,
Bastian

> 
> Thanks,
> Ahmad
> 
>>
>> Regards,
>> Bastian
>>
> 


-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

[PATCH 3/4] ci: pytest: rename build directories

[Ahmad Fatoum]

Naming the build directory after the arch means that multiple jobs will
each create a build-arm directory with different contents.

Let's factor in the name of the defconfig too, so it's
immediately clear what's what.

Signed-off-by: Ahmad Fatoum <a.fatoum@pengutronix.de>
---
 .github/workflows/test-labgrid-pytest.yml | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/.github/workflows/test-labgrid-pytest.yml b/.github/workflows/test-labgrid-pytest.yml
index e1ccac67932f..bfb022dc7778 100644
--- a/.github/workflows/test-labgrid-pytest.yml
+++ b/.github/workflows/test-labgrid-pytest.yml
@@ -65,19 +65,20 @@ jobs:
 
     - name: Build
       run: |
+        export KBUILD_OUTPUT=build-${{matrix.arch}}-${{matrix.defconfig}}
         export ARCH=${{matrix.arch}}
 
-        ./MAKEALL -O build-${{matrix.arch}} -l "" -v 0 \
+        ./MAKEALL -O ${KBUILD_OUTPUT} -l "" -v 0 \
                 -k common/boards/configs/enable_self_test.config \
                 -k common/boards/configs/disable_target_tools.config \
                 ${{matrix.defconfig}}
 
         if [ ${{matrix.arch}} = "riscv" ]; then
-          cp /usr/share/qemu/opensbi-riscv32-generic-fw_dynamic.bin build-${{matrix.arch}}
+          cp /usr/share/qemu/opensbi-riscv32-generic-fw_dynamic.bin ${KBUILD_OUTPUT}/
         fi
     - name: labgrid-pytest
       run: |
-        export KBUILD_OUTPUT=build-${{matrix.arch}}
+        export KBUILD_OUTPUT=build-${{matrix.arch}}-${{matrix.defconfig}}
 
         for i in ${{matrix.lgenv}}; do
           grep -wq '\(QEMUDriver\|ExternalConsoleDriver\):' "$i" || continue
-- 
2.39.5

[PATCH 4/4] ci: pytest: enable console and libc allocator for allyesconfig

[Ahmad Fatoum]

These two option are already set in the pytest YAML, but we build and
run tests in two separate stages and thus they are never applied.

Combining build and test into one doesn't play nicely with our
structure, because we build e.g. multi_v7_defconfig once and then run
multiple tests on it.

Let's workaround this in the build stage for now.

Signed-off-by: Ahmad Fatoum <a.fatoum@pengutronix.de>
---
 .github/workflows/test-labgrid-pytest.yml | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)

diff --git a/.github/workflows/test-labgrid-pytest.yml b/.github/workflows/test-labgrid-pytest.yml
index bfb022dc7778..315868779bee 100644
--- a/.github/workflows/test-labgrid-pytest.yml
+++ b/.github/workflows/test-labgrid-pytest.yml
@@ -67,11 +67,14 @@ jobs:
       run: |
         export KBUILD_OUTPUT=build-${{matrix.arch}}-${{matrix.defconfig}}
         export ARCH=${{matrix.arch}}
+        export KCONFIG_ADD="common/boards/configs/enable_self_test.config \
+                            common/boards/configs/disable_target_tools.config"
 
-        ./MAKEALL -O ${KBUILD_OUTPUT} -l "" -v 0 \
-                -k common/boards/configs/enable_self_test.config \
-                -k common/boards/configs/disable_target_tools.config \
-                ${{matrix.defconfig}}
+        if [ ${{matrix.arch}} = sandbox ] && [ ${{matrix.defconfig}} = allyesconfig ]; then
+          KCONFIG_ADD="${KCONFIG_ADD} CONFIG_CONSOLE_DISABLE_INPUT=n CONFIG_MALLOC_LIBC=y"
+        fi
+
+        ./MAKEALL -O ${KBUILD_OUTPUT} -l "" -v 0 ${{matrix.defconfig}}
 
         if [ ${{matrix.arch}} = "riscv" ]; then
           cp /usr/share/qemu/opensbi-riscv32-generic-fw_dynamic.bin ${KBUILD_OUTPUT}/
-- 
2.39.5

Re: [PATCH 0/4] ci: fix sandbox allyesconfig build

[Sascha Hauer]

On Thu, 27 Mar 2025 15:47:38 +0100, Ahmad Fatoum wrote:
> With these changes applied, the most recent container image can be used
> to run the pytest tests for allyesconfig.
> 
> Ahmad Fatoum (4):
>   sandbox: env: switch from CONFIG_DEFAULT_ENVIRONMENT_PATH to bbenv-y
>   test: bthread: change comparison condition for switches vs. yields
>   ci: pytest: rename build directories
>   ci: pytest: enable console and libc allocator for allyesconfig
> 
> [...]

Applied, thanks!

[1/4] sandbox: env: switch from CONFIG_DEFAULT_ENVIRONMENT_PATH to bbenv-y
      https://git.pengutronix.de/cgit/barebox/commit/?id=1faaaabc6faf (link may not be stable)
[2/4] test: bthread: change comparison condition for switches vs. yields
      https://git.pengutronix.de/cgit/barebox/commit/?id=ea6cdcd699dc (link may not be stable)
[3/4] ci: pytest: rename build directories
      https://git.pengutronix.de/cgit/barebox/commit/?id=8052dd98ab6a (link may not be stable)
[4/4] ci: pytest: enable console and libc allocator for allyesconfig
      https://git.pengutronix.de/cgit/barebox/commit/?id=bc13b2175583 (link may not be stable)

Best regards,
-- 
Sascha Hauer <s.hauer@pengutronix.de>