mail archive of the barebox mailing list
 help / color / mirror / Atom feed
* [PATCH v2 00/13] add barebox in-tree testing infrastructure
@ 2021-05-31  6:55 Ahmad Fatoum
  2021-05-31  6:55 ` [PATCH v2 01/13] kbuild: add install target Ahmad Fatoum
                   ` (12 more replies)
  0 siblings, 13 replies; 21+ messages in thread
From: Ahmad Fatoum @ 2021-05-31  6:55 UTC (permalink / raw)
  To: barebox

v1 is here:
https://lore.barebox.org/barebox/20210412071651.8769-1-a.fatoum@pengutronix.de/

v1 -> v2:
  - drop already merged patches
  - use objcopy to swap image byte order instead of writing something new (Jules)
  - always run tests from source directory, not build directory when
    building out-of-tree (honour KBUILD_OUTPUT, build symlink in that case)
  - add some more sample tests
  - add test/riscv/sifive_defconfig.yaml
  
Ahmad Fatoum (13):
  kbuild: add install target
  kbuild: add ARCH={i386, x86_64} as aliases for x86
  kbuild: add ARCH=um alias for sandbox
  MIPS: qemu-malta: generate swapped image as part of multi-image build
  openrisc: set default KBUILD_IMAGE
  Documentation: boards: RISC-V: update TinyEMU support
  test: add basic barebox self-test infrastructure
  test: self: port Linux printf kselftest
  test: add labgrid configs for some emulated targets
  test: add first sample tests
  test: add emulate.pl, a runner for barebox on emulated targets
  test: self: run selftests as part of the pytest suite
  test: add bthread test

 Documentation/boards/emulated.rst             |  70 +++
 Documentation/boards/mips/qemu-malta.rst      |  16 +-
 Documentation/boards/riscv.rst                |  20 +-
 Documentation/boards/riscv/barebox-virt32.cfg |   7 +
 Documentation/boards/riscv/barebox-virt64.cfg |   7 +
 Kconfig                                       |   1 +
 Makefile                                      |  28 +-
 arch/openrisc/Makefile                        |   2 +
 commands/Makefile                             |   1 +
 commands/selftest.c                           |  88 +++
 common/startup.c                              |   4 +
 images/.gitignore                             |   1 +
 images/Makefile                               |   7 +-
 images/Makefile.malta                         |  10 +-
 include/bselftest.h                           |  74 +++
 include/stdlib.h                              |   5 +
 test/.gitignore                               |   1 +
 test/Kconfig                                  |   8 +
 test/Makefile                                 |   1 +
 test/__init__.py                              |   0
 test/arm/a15@vexpress_defconfig.yaml          |  23 +
 test/arm/a9@vexpress_defconfig.yaml           |  23 +
 test/arm/qemu_virt64_defconfig.yaml           |  27 +
 test/arm/vexpress_defconfig.yaml              |   1 +
 test/arm/virt@vexpress_defconfig.yaml         |  25 +
 test/conftest.py                              |  34 ++
 test/emulate.pl                               | 509 ++++++++++++++++++
 test/kconfig/base.cfg                         |   4 +
 test/kconfig/full.cfg                         |   2 +
 test/kconfig/virtio-pci.cfg                   |   6 +
 test/mips/be@qemu-malta_defconfig.yaml        |  25 +
 test/mips/le@qemu-malta_defconfig.yaml        |  28 +
 test/mips/qemu-malta_defconfig.yaml           |   1 +
 test/openrisc/generic_defconfig.yaml          |  23 +
 test/py/__init__.py                           |   0
 test/py/helper.py                             |  38 ++
 test/py/test_bselftests.py                    |   8 +
 test/py/test_bthread.py                       |  23 +
 test/py/test_shell.py                         |  37 ++
 test/riscv/qemu@virt32_defconfig.yaml         |  30 ++
 test/riscv/qemu@virt64_defconfig.yaml         |  30 ++
 test/riscv/sifive_defconfig.yaml              |  28 +
 test/riscv/tinyemu@virt32_defconfig.yaml      |  25 +
 test/riscv/tinyemu@virt64_defconfig.yaml      |  25 +
 test/riscv/virt32_defconfig.yaml              |   1 +
 test/riscv/virt64_defconfig.yaml              |   1 +
 test/sandbox/sandbox_defconfig.yaml           |  12 +
 test/self/Kconfig                             |  39 ++
 test/self/Makefile                            |   4 +
 test/self/core.c                              |  22 +
 test/self/printf.c                            | 302 +++++++++++
 test/x86/efi_defconfig.yaml                   |   1 +
 test/x86/pc@efi_defconfig.yaml                |  34 ++
 test/x86/q35@efi_defconfig.yaml               |  34 ++
 test/x86/virtio@efi_defconfig.yaml            |  35 ++
 55 files changed, 1786 insertions(+), 25 deletions(-)
 create mode 100644 Documentation/boards/emulated.rst
 create mode 100644 Documentation/boards/riscv/barebox-virt32.cfg
 create mode 100644 Documentation/boards/riscv/barebox-virt64.cfg
 create mode 100644 commands/selftest.c
 create mode 100644 include/bselftest.h
 create mode 100644 test/.gitignore
 create mode 100644 test/Kconfig
 create mode 100644 test/Makefile
 create mode 100644 test/__init__.py
 create mode 100644 test/arm/a15@vexpress_defconfig.yaml
 create mode 100644 test/arm/a9@vexpress_defconfig.yaml
 create mode 100644 test/arm/qemu_virt64_defconfig.yaml
 create mode 120000 test/arm/vexpress_defconfig.yaml
 create mode 100644 test/arm/virt@vexpress_defconfig.yaml
 create mode 100644 test/conftest.py
 create mode 100755 test/emulate.pl
 create mode 100644 test/kconfig/base.cfg
 create mode 100644 test/kconfig/full.cfg
 create mode 100644 test/kconfig/virtio-pci.cfg
 create mode 100644 test/mips/be@qemu-malta_defconfig.yaml
 create mode 100644 test/mips/le@qemu-malta_defconfig.yaml
 create mode 120000 test/mips/qemu-malta_defconfig.yaml
 create mode 100644 test/openrisc/generic_defconfig.yaml
 create mode 100644 test/py/__init__.py
 create mode 100644 test/py/helper.py
 create mode 100644 test/py/test_bselftests.py
 create mode 100644 test/py/test_bthread.py
 create mode 100644 test/py/test_shell.py
 create mode 100644 test/riscv/qemu@virt32_defconfig.yaml
 create mode 100644 test/riscv/qemu@virt64_defconfig.yaml
 create mode 100644 test/riscv/sifive_defconfig.yaml
 create mode 100644 test/riscv/tinyemu@virt32_defconfig.yaml
 create mode 100644 test/riscv/tinyemu@virt64_defconfig.yaml
 create mode 120000 test/riscv/virt32_defconfig.yaml
 create mode 120000 test/riscv/virt64_defconfig.yaml
 create mode 100644 test/sandbox/sandbox_defconfig.yaml
 create mode 100644 test/self/Kconfig
 create mode 100644 test/self/Makefile
 create mode 100644 test/self/core.c
 create mode 100644 test/self/printf.c
 create mode 120000 test/x86/efi_defconfig.yaml
 create mode 100644 test/x86/pc@efi_defconfig.yaml
 create mode 100644 test/x86/q35@efi_defconfig.yaml
 create mode 100644 test/x86/virtio@efi_defconfig.yaml

-- 
2.29.2


_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox


^ permalink raw reply	[flat|nested] 21+ messages in thread

* [PATCH v2 01/13] kbuild: add install target
  2021-05-31  6:55 [PATCH v2 00/13] add barebox in-tree testing infrastructure Ahmad Fatoum
@ 2021-05-31  6:55 ` Ahmad Fatoum
  2021-05-31  6:55 ` [PATCH v2 02/13] kbuild: add ARCH={i386, x86_64} as aliases for x86 Ahmad Fatoum
                   ` (11 subsequent siblings)
  12 siblings, 0 replies; 21+ messages in thread
From: Ahmad Fatoum @ 2021-05-31  6:55 UTC (permalink / raw)
  To: barebox; +Cc: Ahmad Fatoum

For CI, it would be useful to have the barebox build install the
artifacts into a directory. Add an install target that does this.

Example usage: make install INSTALL_PATH=install/

Unlike Linux, we don't set INSTALL_PATH to a default value, because
most barebox-enabled boards don't have barebox in a file system.

Signed-off-by: Ahmad Fatoum <a.fatoum@pengutronix.de>
---
 Makefile        | 14 ++++++++++++++
 images/Makefile |  5 +++++
 2 files changed, 19 insertions(+)

diff --git a/Makefile b/Makefile
index ec1770042549..65e3f6f80c72 100644
--- a/Makefile
+++ b/Makefile
@@ -880,6 +880,20 @@ ifndef CONFIG_PBL_IMAGE
 	$(call cmd,check_file_size,$@,$(CONFIG_BAREBOX_MAX_IMAGE_SIZE))
 endif
 
+install:
+ifeq ($(INSTALL_PATH),)
+	@echo 'error: INSTALL_PATH undefined' >&2
+	@exit 1
+endif
+ifdef CONFIG_PBL_IMAGE
+	$(Q)$(MAKE) $(build)=images __images_install
+	@install -t "$(INSTALL_PATH)" barebox.bin
+else
+	@install -t "$(INSTALL_PATH)" $(KBUILD_IMAGE)
+endif
+
+PHONY += install
+
 # By default the uImage load address is 2MB below CONFIG_TEXT_BASE,
 # leaving space for the compressed PBL image at 1MB below CONFIG_TEXT_BASE.
 UIMAGE_BASE ?= $(shell printf "0x%08x" $$(($(CONFIG_TEXT_BASE) - 0x200000)))
diff --git a/images/Makefile b/images/Makefile
index ee1347f6b6bd..c185982c17ed 100644
--- a/images/Makefile
+++ b/images/Makefile
@@ -204,6 +204,11 @@ images: $(image-y-path) $(flash-link) $(flash-list) FORCE
 	@echo "images built:"
 	@for i in $(image-y); do echo $$i; done
 
+__images_install: images
+	@for i in $(image-y-path); do install -t "$(INSTALL_PATH)" $$i; done
+
+PHONY += __images_install
+
 $(flash-link): $(link-dest) FORCE
 	$(call if_changed,ln)
 
-- 
2.29.2


_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox


^ permalink raw reply	[flat|nested] 21+ messages in thread

* [PATCH v2 02/13] kbuild: add ARCH={i386, x86_64} as aliases for x86
  2021-05-31  6:55 [PATCH v2 00/13] add barebox in-tree testing infrastructure Ahmad Fatoum
  2021-05-31  6:55 ` [PATCH v2 01/13] kbuild: add install target Ahmad Fatoum
@ 2021-05-31  6:55 ` Ahmad Fatoum
  2021-05-31  6:55 ` [PATCH v2 03/13] kbuild: add ARCH=um alias for sandbox Ahmad Fatoum
                   ` (10 subsequent siblings)
  12 siblings, 0 replies; 21+ messages in thread
From: Ahmad Fatoum @ 2021-05-31  6:55 UTC (permalink / raw)
  To: barebox; +Cc: Ahmad Fatoum

Linux already does this. Follow suit so we can interoperate with tools
that assume Linux-like Kbuild structure.

Signed-off-by: Ahmad Fatoum <a.fatoum@pengutronix.de>
---
 Makefile | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/Makefile b/Makefile
index 65e3f6f80c72..3b7dbd4c772d 100644
--- a/Makefile
+++ b/Makefile
@@ -343,6 +343,14 @@ ifeq ($(ARCH),arm64)
        SRCARCH := arm
 endif
 
+ifeq ($(ARCH),i386)
+       SRCARCH := x86
+endif
+
+ifeq ($(ARCH),x86_64)
+       SRCARCH := x86
+endif
+
 # Support ARCH=ppc for backward compatibility
 ifeq ($(ARCH),ppc)
        SRCARCH := powerpc
-- 
2.29.2


_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox


^ permalink raw reply	[flat|nested] 21+ messages in thread

* [PATCH v2 03/13] kbuild: add ARCH=um alias for sandbox
  2021-05-31  6:55 [PATCH v2 00/13] add barebox in-tree testing infrastructure Ahmad Fatoum
  2021-05-31  6:55 ` [PATCH v2 01/13] kbuild: add install target Ahmad Fatoum
  2021-05-31  6:55 ` [PATCH v2 02/13] kbuild: add ARCH={i386, x86_64} as aliases for x86 Ahmad Fatoum
@ 2021-05-31  6:55 ` Ahmad Fatoum
  2021-05-31  6:55 ` [PATCH v2 04/13] MIPS: qemu-malta: generate swapped image as part of multi-image build Ahmad Fatoum
                   ` (9 subsequent siblings)
  12 siblings, 0 replies; 21+ messages in thread
From: Ahmad Fatoum @ 2021-05-31  6:55 UTC (permalink / raw)
  To: barebox; +Cc: Ahmad Fatoum

ARCH=um (User Mode) is the Linux kernel counterpart to our sandbox
architecture. Add um as an alias, so we can interoperate with tools that
handle ARCH=um specially.

Signed-off-by: Ahmad Fatoum <a.fatoum@pengutronix.de>
---
 Makefile | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/Makefile b/Makefile
index 3b7dbd4c772d..a1411386f1b3 100644
--- a/Makefile
+++ b/Makefile
@@ -356,6 +356,10 @@ ifeq ($(ARCH),ppc)
        SRCARCH := powerpc
 endif
 
+ifeq ($(ARCH),um)
+       SRCARCH := sandbox
+endif
+
 KCONFIG_CONFIG	?= .config
 
 export KCONFIG_CONFIG
-- 
2.29.2


_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox


^ permalink raw reply	[flat|nested] 21+ messages in thread

* [PATCH v2 04/13] MIPS: qemu-malta: generate swapped image as part of multi-image build
  2021-05-31  6:55 [PATCH v2 00/13] add barebox in-tree testing infrastructure Ahmad Fatoum
                   ` (2 preceding siblings ...)
  2021-05-31  6:55 ` [PATCH v2 03/13] kbuild: add ARCH=um alias for sandbox Ahmad Fatoum
@ 2021-05-31  6:55 ` Ahmad Fatoum
  2021-05-31  6:55 ` [PATCH v2 05/13] openrisc: set default KBUILD_IMAGE Ahmad Fatoum
                   ` (8 subsequent siblings)
  12 siblings, 0 replies; 21+ messages in thread
From: Ahmad Fatoum @ 2021-05-31  6:55 UTC (permalink / raw)
  To: barebox; +Cc: Ahmad Fatoum

Having to manually swap the words in the MIPS Malta image for QEMU
little endian emulation is annoying.

Have the multi-image build for Malta generate a second .swapped
image that can be readily used if needed.

Signed-off-by: Ahmad Fatoum <a.fatoum@pengutronix.de>
---
 Documentation/boards/mips/qemu-malta.rst | 16 ++++------------
 images/.gitignore                        |  1 +
 images/Makefile                          |  2 +-
 images/Makefile.malta                    | 10 +++++++++-
 4 files changed, 15 insertions(+), 14 deletions(-)

diff --git a/Documentation/boards/mips/qemu-malta.rst b/Documentation/boards/mips/qemu-malta.rst
index e188ae8c642a..fd37d5edb229 100644
--- a/Documentation/boards/mips/qemu-malta.rst
+++ b/Documentation/boards/mips/qemu-malta.rst
@@ -10,31 +10,23 @@ QEMU run string:
 
   qemu-system-mips -nodefaults -M malta -m 256 \
       -device VGA -serial stdio -monitor null \
-      -bios barebox-flash-image
+      -bios ./images/barebox-qemu-malta.img
 
 
 Little-endian mode
 ------------------
 
-Running little-endian Malta is a bit tricky.
 In little-endian mode the 32bit words in the boot flash image are swapped,
 a neat trick which allows bi-endian firmware.
 
-You have to swap words of ``zbarebox.bin`` image, e.g.:
-
-.. code-block:: sh
-
-  echo arch/mips/pbl/zbarebox.bin \
-      | cpio --create \
-      | cpio --extract --swap --unconditional
-
-QEMU run string:
+The barebox build generates a second ``./images/barebox-qemu-malta.img.swapped``
+image that can be used in this case, e.g.:
 
 .. code-block:: sh
 
   qemu-system-mipsel -nodefaults -M malta -m 256 \
       -device VGA -serial stdio -monitor null \
-      -bios barebox-flash-image
+      -bios ./images/barebox-qemu-malta.img.swapped
 
 
 Using GXemul
diff --git a/images/.gitignore b/images/.gitignore
index eafdb44b5bdd..3a9a77dad16e 100644
--- a/images/.gitignore
+++ b/images/.gitignore
@@ -32,3 +32,4 @@ barebox.sum
 *.mvebu1img
 *.stm32
 *.nmon
+*.swapped
diff --git a/images/Makefile b/images/Makefile
index c185982c17ed..cc330d957597 100644
--- a/images/Makefile
+++ b/images/Makefile
@@ -218,5 +218,5 @@ $(flash-list): $(image-y-path)
 clean-files := *.pbl *.pblb *.map start_*.imximg *.img barebox.z start_*.kwbimg \
 	start_*.kwbuartimg *.socfpgaimg *.mlo *.t20img *.t20img.cfg *.t30img \
 	*.t30img.cfg *.t124img *.t124img.cfg *.mlospi *.mlo *.mxsbs *.mxssd \
-	start_*.simximg start_*.usimximg *.zynqimg *.image
+	start_*.simximg start_*.usimximg *.zynqimg *.image *.swapped
 clean-files += pbl.lds
diff --git a/images/Makefile.malta b/images/Makefile.malta
index 5739ec464092..96d7b86b1187 100644
--- a/images/Makefile.malta
+++ b/images/Makefile.malta
@@ -1,3 +1,11 @@
+quiet_cmd_bswap32_image = BSWAP4  $@
+      cmd_bswap32_image = cp $< $@ && \
+			  truncate -s %4 $@ && \
+			  objcopy -I binary --reverse-byte=4 $@
+
+$(obj)/%.img.swapped: $(obj)/%.img FORCE
+	$(call if_changed,bswap32_image)
+
 pblb-$(CONFIG_BOARD_QEMU_MALTA) += start_qemu_malta
 FILE_barebox-qemu-malta.img = start_qemu_malta.pblb
-image-$(CONFIG_BOARD_QEMU_MALTA) += barebox-qemu-malta.img
+image-$(CONFIG_BOARD_QEMU_MALTA) += barebox-qemu-malta.img barebox-qemu-malta.img.swapped
-- 
2.29.2


_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox


^ permalink raw reply	[flat|nested] 21+ messages in thread

* [PATCH v2 05/13] openrisc: set default KBUILD_IMAGE
  2021-05-31  6:55 [PATCH v2 00/13] add barebox in-tree testing infrastructure Ahmad Fatoum
                   ` (3 preceding siblings ...)
  2021-05-31  6:55 ` [PATCH v2 04/13] MIPS: qemu-malta: generate swapped image as part of multi-image build Ahmad Fatoum
@ 2021-05-31  6:55 ` Ahmad Fatoum
  2021-05-31  6:55 ` [PATCH v2 06/13] Documentation: boards: RISC-V: update TinyEMU support Ahmad Fatoum
                   ` (7 subsequent siblings)
  12 siblings, 0 replies; 21+ messages in thread
From: Ahmad Fatoum @ 2021-05-31  6:55 UTC (permalink / raw)
  To: barebox; +Cc: Ahmad Fatoum

For non-multi-image configuration, KBUILD_IMAGE is taken as the image to
install. It was so far not set for openRISC. Define it, so it can be
used by the make install stage, e.g. to collect artifacts after a CI
run.

Signed-off-by: Ahmad Fatoum <a.fatoum@pengutronix.de>
---
 arch/openrisc/Makefile | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/arch/openrisc/Makefile b/arch/openrisc/Makefile
index 72d7fa3d53fe..1776f56df97b 100644
--- a/arch/openrisc/Makefile
+++ b/arch/openrisc/Makefile
@@ -28,3 +28,5 @@ dts := arch/openrisc/dts
 
 %.dtb: scripts
 	$(Q)$(MAKE) $(build)=$(dts) $(dts)/$@
+
+KBUILD_IMAGE := barebox
-- 
2.29.2


_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox


^ permalink raw reply	[flat|nested] 21+ messages in thread

* [PATCH v2 06/13] Documentation: boards: RISC-V: update TinyEMU support
  2021-05-31  6:55 [PATCH v2 00/13] add barebox in-tree testing infrastructure Ahmad Fatoum
                   ` (4 preceding siblings ...)
  2021-05-31  6:55 ` [PATCH v2 05/13] openrisc: set default KBUILD_IMAGE Ahmad Fatoum
@ 2021-05-31  6:55 ` Ahmad Fatoum
  2021-05-31  6:55 ` [PATCH v2 07/13] test: add basic barebox self-test infrastructure Ahmad Fatoum
                   ` (6 subsequent siblings)
  12 siblings, 0 replies; 21+ messages in thread
From: Ahmad Fatoum @ 2021-05-31  6:55 UTC (permalink / raw)
  To: barebox; +Cc: Ahmad Fatoum

Adding the tinyemu files as separate files and including them has the
nice benefit that temu users can just use these files directly instead
of copy pasting. While at it, update the section as input, as support
was added meanwhile.

Signed-off-by: Ahmad Fatoum <a.fatoum@pengutronix.de>
---
 Documentation/boards/riscv.rst                | 20 +++++++++----------
 Documentation/boards/riscv/barebox-virt32.cfg |  7 +++++++
 Documentation/boards/riscv/barebox-virt64.cfg |  7 +++++++
 3 files changed, 24 insertions(+), 10 deletions(-)
 create mode 100644 Documentation/boards/riscv/barebox-virt32.cfg
 create mode 100644 Documentation/boards/riscv/barebox-virt64.cfg

diff --git a/Documentation/boards/riscv.rst b/Documentation/boards/riscv.rst
index 59cdc00a99e6..955a6fbb6bd1 100644
--- a/Documentation/boards/riscv.rst
+++ b/Documentation/boards/riscv.rst
@@ -41,25 +41,25 @@ TinyEMU
 -------
 
 TinyEMU can emulate a qemu-virt like machine with a RISC-V 32-, 64-
-and 128-bit CPU. It can run barebox with this sample configuration::
+and 128-bit CPU. It can run 32-bit barebox with this sample configuration:
 
-  /* temu barebox-virt64.cfg */
-  {
-      version: 1,
-      machine: "riscv64",
-      memory_size: 256,
-      bios: "bbl64.bin",
-      kernel: "./images/barebox-dt-2nd.img",
-  }
+.. literalinclude:: riscv/barebox-virt32.cfg
+
+as well as 64-bit barebox with this configuration:
+
+.. literalinclude:: riscv/barebox-virt64.cfg
 
 ``barebox-dt-2nd.img`` can be generated like with Qemu. Graphical
-output is also supported, but virtio input support is still missing.
+output and input are also supported.
 To activate add::
 
     display0: { device: "simplefb", width: 800, height: 600 },
+    input_device: "virtio",
 
 into the config file.
 
+See https://barebox.org/jsbarebox/?graphic=1 for a live example.
+
 Erizo
 -----
 
diff --git a/Documentation/boards/riscv/barebox-virt32.cfg b/Documentation/boards/riscv/barebox-virt32.cfg
new file mode 100644
index 000000000000..5f0eb34eee9c
--- /dev/null
+++ b/Documentation/boards/riscv/barebox-virt32.cfg
@@ -0,0 +1,7 @@
+{
+    version: 1,
+    machine: "riscv32",
+    memory_size: 256,
+    bios: "bbl32.bin",
+    kernel: "images/barebox-dt-2nd.img",
+}
diff --git a/Documentation/boards/riscv/barebox-virt64.cfg b/Documentation/boards/riscv/barebox-virt64.cfg
new file mode 100644
index 000000000000..45e1cd830802
--- /dev/null
+++ b/Documentation/boards/riscv/barebox-virt64.cfg
@@ -0,0 +1,7 @@
+{
+    version: 1,
+    machine: "riscv64",
+    memory_size: 256,
+    bios: "bbl64.bin",
+    kernel: "images/barebox-dt-2nd.img",
+}
-- 
2.29.2


_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox


^ permalink raw reply	[flat|nested] 21+ messages in thread

* [PATCH v2 07/13] test: add basic barebox self-test infrastructure
  2021-05-31  6:55 [PATCH v2 00/13] add barebox in-tree testing infrastructure Ahmad Fatoum
                   ` (5 preceding siblings ...)
  2021-05-31  6:55 ` [PATCH v2 06/13] Documentation: boards: RISC-V: update TinyEMU support Ahmad Fatoum
@ 2021-05-31  6:55 ` Ahmad Fatoum
  2021-05-31  6:55 ` [PATCH v2 08/13] test: self: port Linux printf kselftest Ahmad Fatoum
                   ` (5 subsequent siblings)
  12 siblings, 0 replies; 21+ messages in thread
From: Ahmad Fatoum @ 2021-05-31  6:55 UTC (permalink / raw)
  To: barebox; +Cc: Ahmad Fatoum

Self tests is code written to run within barebox to exercise
functionality. They offer flexibility to test specific units of barebox
instead of the program as a whole. Add a very simple infrastructure
for registering and executing self-tests. This is based on the Linux
kselftest modules. We don't utilize modules for this, however, because
we only have module support on ARM, but we need a generic solution.

Selftests can be enabled individually and even tested without shell
support to allow tests to happen for size-restricted barebox images
as well.

Signed-off-by: Ahmad Fatoum <a.fatoum@pengutronix.de>
---
 Kconfig             |  1 +
 Makefile            |  2 +-
 commands/Makefile   |  1 +
 commands/selftest.c | 88 +++++++++++++++++++++++++++++++++++++++++++++
 common/startup.c    |  4 +++
 include/bselftest.h | 74 ++++++++++++++++++++++++++++++++++++++
 test/Kconfig        |  8 +++++
 test/Makefile       |  1 +
 test/self/Kconfig   | 33 +++++++++++++++++
 test/self/Makefile  |  3 ++
 test/self/core.c    | 22 ++++++++++++
 11 files changed, 236 insertions(+), 1 deletion(-)
 create mode 100644 commands/selftest.c
 create mode 100644 include/bselftest.h
 create mode 100644 test/Kconfig
 create mode 100644 test/Makefile
 create mode 100644 test/self/Kconfig
 create mode 100644 test/self/Makefile
 create mode 100644 test/self/core.c

diff --git a/Kconfig b/Kconfig
index 29c32463fb43..7c4cf36881b4 100644
--- a/Kconfig
+++ b/Kconfig
@@ -15,3 +15,4 @@ source "lib/Kconfig"
 source "crypto/Kconfig"
 source "firmware/Kconfig"
 source "scripts/Kconfig"
+source "test/Kconfig"
diff --git a/Makefile b/Makefile
index a1411386f1b3..e856d8672b71 100644
--- a/Makefile
+++ b/Makefile
@@ -581,7 +581,7 @@ endif
 include $(srctree)/scripts/Makefile.lib
 
 # Objects we will link into barebox / subdirs we need to visit
-common-y		:= common/ drivers/ commands/ lib/ crypto/ net/ fs/ firmware/
+common-y		:= common/ drivers/ commands/ lib/ crypto/ net/ fs/ firmware/ test/
 
 include $(srctree)/arch/$(SRCARCH)/Makefile
 
diff --git a/commands/Makefile b/commands/Makefile
index 447349fd157d..4b45d266fd56 100644
--- a/commands/Makefile
+++ b/commands/Makefile
@@ -130,5 +130,6 @@ obj-$(CONFIG_CMD_SEED)		+= seed.o
 obj-$(CONFIG_CMD_IP_ROUTE_GET)  += ip-route-get.o
 obj-$(CONFIG_CMD_BTHREAD)	+= bthread.o
 obj-$(CONFIG_CMD_UBSAN)		+= ubsan.o
+obj-$(CONFIG_CMD_SELFTEST)	+= selftest.o
 
 UBSAN_SANITIZE_ubsan.o := y
diff --git a/commands/selftest.c b/commands/selftest.c
new file mode 100644
index 000000000000..a10f1467fece
--- /dev/null
+++ b/commands/selftest.c
@@ -0,0 +1,88 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+#define pr_fmt(fmt) "selftest: " fmt
+
+#include <common.h>
+#include <command.h>
+#include <getopt.h>
+#include <bselftest.h>
+#include <complete.h>
+
+static int run_selftest(const char *match, bool list)
+{
+	struct selftest *test;
+	int matches = 0;
+	int err = 0;
+
+	list_for_each_entry(test, &selftests, list) {
+		if (list) {
+			printf("%s\n", test->name);
+			matches++;
+			continue;
+		}
+
+		if (match && strcmp(test->name, match))
+			continue;
+
+		err |= test->func();
+		matches++;
+	}
+
+	if (!matches) {
+		if (match) {
+			printf("No tests matching '%s' found.\n", match);
+			return -EINVAL;
+		}
+
+		printf("No tests found.\n");
+	}
+
+	return err;
+}
+
+static int do_selftest(int argc, char *argv[])
+{
+	bool list = false;
+	int i, err = 0;
+	int opt;
+
+	while((opt = getopt(argc, argv, "l")) > 0) {
+		switch(opt) {
+		case 'l':
+			list = true;
+			break;
+		default:
+			return COMMAND_ERROR_USAGE;
+		}
+	}
+
+	if (optind == argc) {
+		err = run_selftest(NULL, list);
+	} else {
+		for (i = optind; i < argc; i++) {
+			err = run_selftest(argv[i], list);
+			if (err)
+				goto out;
+		}
+	}
+
+out:
+	return err ? COMMAND_ERROR : COMMAND_SUCCESS;
+}
+
+BAREBOX_CMD_HELP_START(selftest)
+BAREBOX_CMD_HELP_TEXT("Run enabled barebox self-tests")
+BAREBOX_CMD_HELP_TEXT("If run without arguments, all tests are run")
+BAREBOX_CMD_HELP_TEXT("")
+BAREBOX_CMD_HELP_TEXT("Options:")
+BAREBOX_CMD_HELP_OPT ("-l",     "list available tests")
+BAREBOX_CMD_HELP_END
+
+BAREBOX_CMD_START(selftest)
+	.cmd		= do_selftest,
+	BAREBOX_CMD_DESC("Run selftests")
+	BAREBOX_CMD_OPTS("[-l] [tests..]")
+	BAREBOX_CMD_GROUP(CMD_GRP_MISC)
+	BAREBOX_CMD_COMPLETE(empty_complete)
+	BAREBOX_CMD_HELP(cmd_selftest_help)
+BAREBOX_CMD_END
diff --git a/common/startup.c b/common/startup.c
index 080feebf05fd..d170cb8a7c5a 100644
--- a/common/startup.c
+++ b/common/startup.c
@@ -37,6 +37,7 @@
 #include <linux/ctype.h>
 #include <watchdog.h>
 #include <glob.h>
+#include <bselftest.h>
 
 extern initcall_t __barebox_initcalls_start[], __barebox_early_initcalls_end[],
 		  __barebox_initcalls_end[];
@@ -417,6 +418,9 @@ void __noreturn start_barebox(void)
 
 	pr_debug("initcalls done\n");
 
+	if (IS_ENABLED(CONFIG_SELFTEST_AUTORUN))
+		selftests_run();
+
 	if (barebox_main)
 		barebox_main();
 
diff --git a/include/bselftest.h b/include/bselftest.h
new file mode 100644
index 000000000000..21eeba0526ef
--- /dev/null
+++ b/include/bselftest.h
@@ -0,0 +1,74 @@
+/* SPDX-License-Identifier: GPL-2.0+ */
+#ifndef __BSELFTEST_H
+#define __BSELFTEST_H
+
+#include <linux/compiler.h>
+#include <linux/list.h>
+#include <init.h>
+
+enum bselftest_group {
+	BSELFTEST_core
+};
+
+struct selftest {
+	enum bselftest_group group;
+	const char *name;
+	int (*func)(void);
+	struct list_head list;
+};
+
+static inline int selftest_report(unsigned int total_tests, unsigned int failed_tests,
+				  unsigned int skipped_tests)
+{
+	if (failed_tests == 0) {
+		if (skipped_tests) {
+			pr_info("skipped %u tests\n", skipped_tests);
+			pr_info("remaining %u tests passed\n", total_tests);
+		} else
+			pr_info("all %u tests passed\n", total_tests);
+	} else
+		pr_err("failed %u out of %u tests\n", failed_tests, total_tests);
+
+	return failed_tests ? -EINVAL : 0;
+}
+
+extern struct list_head selftests;
+
+#define BSELFTEST_GLOBALS()			\
+static unsigned int total_tests __initdata;	\
+static unsigned int failed_tests __initdata;	\
+static unsigned int skipped_tests __initdata
+
+#ifdef CONFIG_SELFTEST
+#define __bselftest_initcall(func) late_initcall(func)
+void selftests_run(void);
+#else
+#define __bselftest_initcall(func)
+static inline void selftests_run(void)
+{
+}
+#endif
+
+#define bselftest(_group, _func)				\
+	static int __bselftest_##_func(void)			\
+	{							\
+		total_tests = failed_tests = skipped_tests = 0;	\
+		_func();					\
+		return selftest_report(total_tests,		\
+				failed_tests,			\
+				skipped_tests);			\
+	}							\
+	static __maybe_unused					\
+	int __init _func##_bselftest_register(void)		\
+	{							\
+		static struct selftest this = {			\
+			.group = BSELFTEST_##_group,		\
+			.name = KBUILD_MODNAME,			\
+			.func = __bselftest_##_func,		\
+		};						\
+		list_add_tail(&this.list, &selftests);		\
+		return 0;					\
+	}							\
+	__bselftest_initcall(_func##_bselftest_register);
+
+#endif
diff --git a/test/Kconfig b/test/Kconfig
new file mode 100644
index 000000000000..eece702e68aa
--- /dev/null
+++ b/test/Kconfig
@@ -0,0 +1,8 @@
+menuconfig TEST
+	bool "Testing"
+
+if TEST
+
+source "test/self/Kconfig"
+
+endif
diff --git a/test/Makefile b/test/Makefile
new file mode 100644
index 000000000000..1b9eb2171a82
--- /dev/null
+++ b/test/Makefile
@@ -0,0 +1 @@
+obj-y += self/
diff --git a/test/self/Kconfig b/test/self/Kconfig
new file mode 100644
index 000000000000..720abeffc51b
--- /dev/null
+++ b/test/self/Kconfig
@@ -0,0 +1,33 @@
+# SPDX-License-Identifier: GPL-2.0
+
+config SELFTEST
+	bool "Self-tests"
+	help
+	  Configures support for in-barebox testing
+
+if SELFTEST
+
+config CMD_SELFTEST
+	bool "selftest command"
+	depends on COMMAND_SUPPORT
+	help
+	  Command to run enabled barebox self-tests.
+	  If run without arguments, all tests are run
+
+	  Usage: selftest [-l] [tests...]
+
+	  Options:
+	    -l     list available tests
+
+config SELFTEST_AUTORUN
+	bool "Run self-tests on startup"
+	help
+	  Self tests are run automatically after initcalls are done,
+	  but before barebox_main (shell or board-specific startup).
+
+config SELFTEST_ENABLE_ALL
+	bool "Enable all self-tests"
+	help
+	  Selects all self-tests compatible with current configuration
+
+endif
diff --git a/test/self/Makefile b/test/self/Makefile
new file mode 100644
index 000000000000..78f738c8e210
--- /dev/null
+++ b/test/self/Makefile
@@ -0,0 +1,3 @@
+# SPDX-License-Identifier: GPL-2.0
+
+obj-$(CONFIG_SELFTEST) += core.o
diff --git a/test/self/core.c b/test/self/core.c
new file mode 100644
index 000000000000..5a309fc8f18d
--- /dev/null
+++ b/test/self/core.c
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+#define pr_fmt(fmt) "bselftest: " fmt
+
+#include <common.h>
+#include <bselftest.h>
+
+LIST_HEAD(selftests);
+
+int selftests_run(void)
+{
+	struct selftest *test;
+	int err = 0;
+
+	pr_notice("Configured tests will run now\n");
+
+	list_for_each_entry(test, &selftests, list)
+		err |= test->func();
+
+	if (err)
+		pr_err("Some selftests failed\n");
+}
-- 
2.29.2


_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox


^ permalink raw reply	[flat|nested] 21+ messages in thread

* [PATCH v2 08/13] test: self: port Linux printf kselftest
  2021-05-31  6:55 [PATCH v2 00/13] add barebox in-tree testing infrastructure Ahmad Fatoum
                   ` (6 preceding siblings ...)
  2021-05-31  6:55 ` [PATCH v2 07/13] test: add basic barebox self-test infrastructure Ahmad Fatoum
@ 2021-05-31  6:55 ` Ahmad Fatoum
  2021-05-31  6:55 ` [PATCH v2 09/13] test: add labgrid configs for some emulated targets Ahmad Fatoum
                   ` (4 subsequent siblings)
  12 siblings, 0 replies; 21+ messages in thread
From: Ahmad Fatoum @ 2021-05-31  6:55 UTC (permalink / raw)
  To: barebox; +Cc: Ahmad Fatoum

Port over the Linux v5.11 selftest for printf sans the parts we don't
support. This can be used to catch regressions if changes affecting the
printf code are made.

Signed-off-by: Ahmad Fatoum <a.fatoum@pengutronix.de>
---
 include/stdlib.h   |   5 +
 test/self/Kconfig  |   6 +
 test/self/Makefile |   1 +
 test/self/core.c   |   2 +-
 test/self/printf.c | 302 +++++++++++++++++++++++++++++++++++++++++++++
 5 files changed, 315 insertions(+), 1 deletion(-)
 create mode 100644 test/self/printf.c

diff --git a/include/stdlib.h b/include/stdlib.h
index d4056087243d..8eb419e111f0 100644
--- a/include/stdlib.h
+++ b/include/stdlib.h
@@ -25,4 +25,9 @@ static inline u32 random32(void)
 	return ret;
 }
 
+static inline u32 prandom_u32_max(u32 ep_ro)
+{
+	return (u32)(((u64) random32() * ep_ro) >> 32);
+}
+
 #endif /* __STDLIB_H */
diff --git a/test/self/Kconfig b/test/self/Kconfig
index 720abeffc51b..73dc6c7b4f03 100644
--- a/test/self/Kconfig
+++ b/test/self/Kconfig
@@ -27,7 +27,13 @@ config SELFTEST_AUTORUN
 
 config SELFTEST_ENABLE_ALL
 	bool "Enable all self-tests"
+	select SELFTEST_PRINTF
 	help
 	  Selects all self-tests compatible with current configuration
 
+config SELFTEST_PRINTF
+	bool "printf selftest"
+	help
+	  Tests barebox vsnprintf() functionality
+
 endif
diff --git a/test/self/Makefile b/test/self/Makefile
index 78f738c8e210..b4aa49d6f817 100644
--- a/test/self/Makefile
+++ b/test/self/Makefile
@@ -1,3 +1,4 @@
 # SPDX-License-Identifier: GPL-2.0
 
 obj-$(CONFIG_SELFTEST) += core.o
+obj-$(CONFIG_SELFTEST_PRINTF) += printf.o
diff --git a/test/self/core.c b/test/self/core.c
index 5a309fc8f18d..caa4c27f6def 100644
--- a/test/self/core.c
+++ b/test/self/core.c
@@ -7,7 +7,7 @@
 
 LIST_HEAD(selftests);
 
-int selftests_run(void)
+void selftests_run(void)
 {
 	struct selftest *test;
 	int err = 0;
diff --git a/test/self/printf.c b/test/self/printf.c
new file mode 100644
index 000000000000..52fe6ac0faf9
--- /dev/null
+++ b/test/self/printf.c
@@ -0,0 +1,302 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Test cases for printf facility.
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <common.h>
+#include <bselftest.h>
+#include <linux/kernel.h>
+#include <module.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <linux/string.h>
+#include <errno.h>
+
+#include <linux/bitmap.h>
+
+#define BUF_SIZE 256
+#define PAD_SIZE 16
+#define FILL_CHAR '$'
+
+BSELFTEST_GLOBALS();
+
+static char *test_buffer __initdata;
+static char *alloced_buffer __initdata;
+
+static int __printf(4, 0) __init
+do_test(int bufsize, const char *expect, int elen,
+	const char *fmt, va_list ap)
+{
+	va_list aq;
+	int ret, written;
+
+	total_tests++;
+
+	memset(alloced_buffer, FILL_CHAR, BUF_SIZE + 2*PAD_SIZE);
+	va_copy(aq, ap);
+	ret = vsnprintf(test_buffer, bufsize, fmt, aq);
+	va_end(aq);
+
+	if (ret != elen) {
+		pr_warn("vsnprintf(buf, %d, \"%s\", ...) returned %d, expected %d\n",
+			bufsize, fmt, ret, elen);
+		pr_warn("vsnprintf(buf, %d, \"%s\", ...) wrote '%s', expected '%.*s'\n",
+			bufsize, fmt, test_buffer, ret, expect);
+		return 1;
+	}
+
+	if (memchr_inv(alloced_buffer, FILL_CHAR, PAD_SIZE)) {
+		pr_warn("vsnprintf(buf, %d, \"%s\", ...) wrote before buffer\n", bufsize, fmt);
+		return 1;
+	}
+
+	if (!bufsize) {
+		if (memchr_inv(test_buffer, FILL_CHAR, BUF_SIZE + PAD_SIZE)) {
+			pr_warn("vsnprintf(buf, 0, \"%s\", ...) wrote to buffer\n",
+				fmt);
+			return 1;
+		}
+		return 0;
+	}
+
+	written = min(bufsize-1, elen);
+	if (test_buffer[written]) {
+		pr_warn("vsnprintf(buf, %d, \"%s\", ...) did not nul-terminate buffer\n",
+			bufsize, fmt);
+		return 1;
+	}
+
+	if (memchr_inv(test_buffer + written + 1, FILL_CHAR, BUF_SIZE + PAD_SIZE - (written + 1))) {
+		pr_warn("vsnprintf(buf, %d, \"%s\", ...) wrote beyond the nul-terminator\n",
+			bufsize, fmt);
+		return 1;
+	}
+
+	if (memcmp(test_buffer, expect, written)) {
+		pr_warn("vsnprintf(buf, %d, \"%s\", ...) wrote '%s', expected '%.*s'\n",
+			bufsize, fmt, test_buffer, written, expect);
+		return 1;
+	}
+	return 0;
+}
+
+static void __printf(3, 4) __init
+__test(const char *expect, int elen, const char *fmt, ...)
+{
+	va_list ap;
+	int rand;
+	char *p;
+
+	if (elen >= BUF_SIZE) {
+		pr_err("error in test suite: expected output length %d too long. Format was '%s'.\n",
+		       elen, fmt);
+		failed_tests++;
+		return;
+	}
+
+	va_start(ap, fmt);
+
+	/*
+	 * Every fmt+args is subjected to four tests: Three where we
+	 * tell vsnprintf varying buffer sizes (plenty, not quite
+	 * enough and 0), and then we also test that bvasprintf would
+	 * be able to print it as expected.
+	 */
+	failed_tests += do_test(BUF_SIZE, expect, elen, fmt, ap);
+	rand = 1 + prandom_u32_max(elen+1);
+	/* Since elen < BUF_SIZE, we have 1 <= rand <= BUF_SIZE. */
+	failed_tests += do_test(rand, expect, elen, fmt, ap);
+	failed_tests += do_test(0, expect, elen, fmt, ap);
+
+	p = bvasprintf(fmt, ap);
+	if (p) {
+		total_tests++;
+		if (memcmp(p, expect, elen+1)) {
+			pr_warn("bvasprintf(..., \"%s\", ...) returned '%s', expected '%s'\n",
+				fmt, p, expect);
+			failed_tests++;
+		}
+		kfree(p);
+	}
+	va_end(ap);
+}
+
+#define test(expect, fmt, ...)					\
+	__test(expect, strlen(expect), fmt, ##__VA_ARGS__)
+
+static void __init
+test_basic(void)
+{
+	/* Work around annoying "warning: zero-length gnu_printf format string". */
+	char nul = '\0';
+
+	test("", &nul);
+	test("100%", "100%%");
+	test("xxx%yyy", "xxx%cyyy", '%');
+	__test("xxx\0yyy", 7, "xxx%cyyy", '\0');
+}
+
+static void __init
+test_number(void)
+{
+	signed char val;
+
+	test("0x1234abcd  ", "%#-12x", 0x1234abcd);
+	test("  0x1234abcd", "%#12x", 0x1234abcd);
+	test("0|001| 12|+123| 1234|-123|-1234", "%d|%03d|%3d|%+d|% d|%+d|% d", 0, 1, 12, 123, 1234, -123, -1234);
+	test("0|1|1|32768|65535", "%hu|%hu|%hu|%hu|%hu", 0, 1, 65537, 32768, -1);
+	test("0|1|1|-32768|-1", "%hd|%hd|%hd|%hd|%hd", 0, 1, 65537, 32768, -1);
+	test("2015122420151225", "%ho%ho%#ho", 1037, 5282, -11627);
+
+	test("2015122420151225", "%ho%ho%#ho", 1037, 5282, -11627);
+
+	/*
+	 * POSIX/C99: »The result of converting zero with an explicit
+	 * precision of zero shall be no characters.« Hence the output
+	 * from the below test should really be "00|0||| ". However,
+	 * the kernel's printf also produces a single 0 in that
+	 * case. This test case simply documents the current
+	 * behaviour.
+	 */
+	test("00|0|0|0|0", "%.2d|%.1d|%.0d|%.*d|%1.0d", 0, 0, 0, 0, 0, 0);
+
+	val = -16;
+	test("0xfffffff0|0xf0|0xf0", "%#02x|%#02x|%#02x", val, val & 0xff, (u8)val);
+
+	/* On some platforms, test failure here indicates a misaligned stack */
+	test("0x0807060504030201", "0x%016llx", 0x0807060504030201ULL);
+}
+
+static void __init
+test_string(void)
+{
+	test("", "%s%.0s", "", "123");
+	test("ABCD|abc|123", "%s|%.3s|%.*s", "ABCD", "abcdef", 3, "123456");
+	test("1  |  2|3  |  4|5  ", "%-3s|%3s|%-*s|%*s|%*s", "1", "2", 3, "3", 3, "4", -3, "5");
+	test("1234      ", "%-10.4s", "123456");
+	test("      1234", "%10.4s", "123456");
+}
+
+#if BITS_PER_LONG == 64
+
+#define PTR_WIDTH 16
+#define PTR ((void *)0xffff0123456789abUL)
+#define PTR_STR "ffff0123456789ab"
+#define PTR_VAL_NO_CRNG "(____ptrval____)"
+#define ZEROS "00000000"	/* hex 32 zero bits */
+#define ONES "ffffffff"		/* hex 32 one bits */
+
+#else
+
+#define PTR_WIDTH 8
+#define PTR ((void *)0x456789ab)
+#define PTR_STR "456789ab"
+#define PTR_VAL_NO_CRNG "(ptrval)"
+#define ZEROS ""
+#define ONES ""
+
+#endif	/* BITS_PER_LONG == 64 */
+
+/*
+ * NULL pointers aren't hashed.
+ */
+static void __init
+null_pointer(void)
+{
+	test(ZEROS "00000000", "%p", NULL);
+	test(ZEROS "00000000", "%px", NULL);
+}
+
+/*
+ * Error pointers aren't hashed.
+ */
+static void __init
+error_pointer(void)
+{
+	test(ONES "fffffff5", "%p", ERR_PTR(-11));
+	test(ONES "fffffff5", "%px", ERR_PTR(-11));
+}
+
+#define PTR_INVALID ((void *)0x000000ab)
+
+static void __init
+invalid_pointer(void)
+{
+	test(ZEROS "000000ab", "%px", PTR_INVALID);
+}
+
+static void __init
+ip4(void)
+{
+	IPaddr_t ip = cpu_to_be32(0x7f000001);
+
+	test("127.0.0.1", "%pI4", &ip);
+}
+
+static void __init
+ip(void)
+{
+	ip4();
+}
+
+static void __init
+uuid(void)
+{
+	const char uuid[16] = {0x0, 0x1, 0x2, 0x3, 0x4, 0x5, 0x6, 0x7,
+			       0x8, 0x9, 0xa, 0xb, 0xc, 0xd, 0xe, 0xf};
+
+	if (!IS_ENABLED(CONFIG_PRINTF_UUID))
+		return;
+
+	test("00010203-0405-0607-0809-0a0b0c0d0e0f", "%pUb", uuid);
+	test("00010203-0405-0607-0809-0A0B0C0D0E0F", "%pUB", uuid);
+	test("03020100-0504-0706-0809-0a0b0c0d0e0f", "%pUl", uuid);
+	test("03020100-0504-0706-0809-0A0B0C0D0E0F", "%pUL", uuid);
+}
+
+static void __init
+errptr(void)
+{
+	test("error 1234", "%pe", ERR_PTR(-1234));
+	test(sizeof(void *) == 8 ? "00000000000004d2" : "000004d2", "%pe", ERR_PTR(1234));
+
+	/* Check that %pe with a non-ERR_PTR gets treated as ordinary %p. */
+	BUILD_BUG_ON(IS_ERR(PTR));
+
+	if (!IS_ENABLED(CONFIG_ERRNO_MESSAGES))
+		return;
+	test("(Operation not permitted)", "(%pe)", ERR_PTR(-EPERM));
+	test("Requested probe deferral", "%pe", ERR_PTR(-EPROBE_DEFER));
+}
+
+static void __init
+test_pointer(void)
+{
+	null_pointer();
+	error_pointer();
+	invalid_pointer();
+	ip();
+	uuid();
+	errptr();
+}
+
+static void __init test_printf(void)
+{
+	alloced_buffer = malloc(BUF_SIZE + 2*PAD_SIZE);
+	if (!alloced_buffer)
+		return;
+	test_buffer = alloced_buffer + PAD_SIZE;
+
+	test_basic();
+	test_number();
+	test_string();
+	test_pointer();
+
+	free(alloced_buffer);
+}
+
+bselftest(core, test_printf);
+MODULE_AUTHOR("Rasmus Villemoes <linux@rasmusvillemoes.dk>");
+MODULE_LICENSE("GPL");
-- 
2.29.2


_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox

^ permalink raw reply	[flat|nested] 21+ messages in thread

* [PATCH v2 09/13] test: add labgrid configs for some emulated targets
  2021-05-31  6:55 [PATCH v2 00/13] add barebox in-tree testing infrastructure Ahmad Fatoum
                   ` (7 preceding siblings ...)
  2021-05-31  6:55 ` [PATCH v2 08/13] test: self: port Linux printf kselftest Ahmad Fatoum
@ 2021-05-31  6:55 ` Ahmad Fatoum
  2021-06-02 11:27   ` Rouven Czerwinski
  2021-05-31  6:55 ` [PATCH v2 10/13] test: add first sample tests Ahmad Fatoum
                   ` (3 subsequent siblings)
  12 siblings, 1 reply; 21+ messages in thread
From: Ahmad Fatoum @ 2021-05-31  6:55 UTC (permalink / raw)
  To: barebox; +Cc: Ahmad Fatoum

There are various ways barebox can run under emulation, some of them are
described in the documentation and some more in the git history.

Make running the emulators more convenient to use by collecting some
known-good emulator invocations in a machine readable format.

These YAML files can be parsed by labgrid for running tests or executed
by a new test/emulate.pl runner script that will be added in a follow-up
commit.

Using labgrid for this will allow using the same test suite for physical
targets as well in future.

Signed-off-by: Ahmad Fatoum <a.fatoum@pengutronix.de>
---
 test/arm/a15@vexpress_defconfig.yaml     | 23 ++++++++++++++++
 test/arm/a9@vexpress_defconfig.yaml      | 23 ++++++++++++++++
 test/arm/qemu_virt64_defconfig.yaml      | 27 ++++++++++++++++++
 test/arm/vexpress_defconfig.yaml         |  1 +
 test/arm/virt@vexpress_defconfig.yaml    | 25 +++++++++++++++++
 test/kconfig/virtio-pci.cfg              |  6 ++++
 test/mips/be@qemu-malta_defconfig.yaml   | 25 +++++++++++++++++
 test/mips/le@qemu-malta_defconfig.yaml   | 28 +++++++++++++++++++
 test/mips/qemu-malta_defconfig.yaml      |  1 +
 test/openrisc/generic_defconfig.yaml     | 23 ++++++++++++++++
 test/riscv/qemu@virt32_defconfig.yaml    | 30 ++++++++++++++++++++
 test/riscv/qemu@virt64_defconfig.yaml    | 30 ++++++++++++++++++++
 test/riscv/sifive_defconfig.yaml         | 28 +++++++++++++++++++
 test/riscv/tinyemu@virt32_defconfig.yaml | 25 +++++++++++++++++
 test/riscv/tinyemu@virt64_defconfig.yaml | 25 +++++++++++++++++
 test/riscv/virt32_defconfig.yaml         |  1 +
 test/riscv/virt64_defconfig.yaml         |  1 +
 test/sandbox/sandbox_defconfig.yaml      | 12 ++++++++
 test/x86/efi_defconfig.yaml              |  1 +
 test/x86/pc@efi_defconfig.yaml           | 34 +++++++++++++++++++++++
 test/x86/q35@efi_defconfig.yaml          | 34 +++++++++++++++++++++++
 test/x86/virtio@efi_defconfig.yaml       | 35 ++++++++++++++++++++++++
 22 files changed, 438 insertions(+)
 create mode 100644 test/arm/a15@vexpress_defconfig.yaml
 create mode 100644 test/arm/a9@vexpress_defconfig.yaml
 create mode 100644 test/arm/qemu_virt64_defconfig.yaml
 create mode 120000 test/arm/vexpress_defconfig.yaml
 create mode 100644 test/arm/virt@vexpress_defconfig.yaml
 create mode 100644 test/kconfig/virtio-pci.cfg
 create mode 100644 test/mips/be@qemu-malta_defconfig.yaml
 create mode 100644 test/mips/le@qemu-malta_defconfig.yaml
 create mode 120000 test/mips/qemu-malta_defconfig.yaml
 create mode 100644 test/openrisc/generic_defconfig.yaml
 create mode 100644 test/riscv/qemu@virt32_defconfig.yaml
 create mode 100644 test/riscv/qemu@virt64_defconfig.yaml
 create mode 100644 test/riscv/sifive_defconfig.yaml
 create mode 100644 test/riscv/tinyemu@virt32_defconfig.yaml
 create mode 100644 test/riscv/tinyemu@virt64_defconfig.yaml
 create mode 120000 test/riscv/virt32_defconfig.yaml
 create mode 120000 test/riscv/virt64_defconfig.yaml
 create mode 100644 test/sandbox/sandbox_defconfig.yaml
 create mode 120000 test/x86/efi_defconfig.yaml
 create mode 100644 test/x86/pc@efi_defconfig.yaml
 create mode 100644 test/x86/q35@efi_defconfig.yaml
 create mode 100644 test/x86/virtio@efi_defconfig.yaml

diff --git a/test/arm/a15@vexpress_defconfig.yaml b/test/arm/a15@vexpress_defconfig.yaml
new file mode 100644
index 000000000000..7ed93ba0a19f
--- /dev/null
+++ b/test/arm/a15@vexpress_defconfig.yaml
@@ -0,0 +1,23 @@
+targets:
+  main:
+    drivers:
+      QEMUDriver:
+        qemu_bin: qemu
+        machine: vexpress-a15
+        cpu: cortex-a15
+        memory: 1024M
+        bios: barebox-vexpress-ca15.img
+        extra_args: ''
+      BareboxDriver:
+        prompt: 'barebox@[^:]+:[^ ]+ '
+        bootstring: 'commandline:'
+      ShellDriver:
+        prompt: 'root@\w+:[^ ]+ '
+        login_prompt: ' login: '
+        login_timeout: 90
+        username: 'root'
+      BareboxStrategy: {}
+images:
+  barebox-vexpress-ca15.img: $LG_BUILDDIR/images/barebox-vexpress-ca15.img
+tools:
+  qemu: /usr/bin/qemu-system-arm
diff --git a/test/arm/a9@vexpress_defconfig.yaml b/test/arm/a9@vexpress_defconfig.yaml
new file mode 100644
index 000000000000..88e09ca052e6
--- /dev/null
+++ b/test/arm/a9@vexpress_defconfig.yaml
@@ -0,0 +1,23 @@
+targets:
+  main:
+    drivers:
+      QEMUDriver:
+        qemu_bin: qemu
+        machine: vexpress-a9
+        cpu: cortex-a9
+        memory: 1024M
+        bios: barebox-vexpress-ca9.img
+        extra_args: ''
+      BareboxDriver:
+        prompt: 'barebox@[^:]+:[^ ]+ '
+        bootstring: 'commandline:'
+      ShellDriver:
+        prompt: 'root@\w+:[^ ]+ '
+        login_prompt: ' login: '
+        login_timeout: 90
+        username: 'root'
+      BareboxStrategy: {}
+images:
+  barebox-vexpress-ca9.img: $LG_BUILDDIR/images/barebox-vexpress-ca9.img
+tools:
+  qemu: /usr/bin/qemu-system-arm
diff --git a/test/arm/qemu_virt64_defconfig.yaml b/test/arm/qemu_virt64_defconfig.yaml
new file mode 100644
index 000000000000..2dfaaedef61a
--- /dev/null
+++ b/test/arm/qemu_virt64_defconfig.yaml
@@ -0,0 +1,27 @@
+targets:
+  main:
+    drivers:
+      QEMUDriver:
+        qemu_bin: qemu
+        machine: virt
+        cpu: cortex-a57
+        memory: 1024M
+        kernel: barebox-dt-2nd.img
+        extra_args: ''
+      BareboxDriver:
+        prompt: 'barebox@[^:]+:[^ ]+ '
+        bootstring: 'commandline:'
+      ShellDriver:
+        prompt: 'root@\w+:[^ ]+ '
+        login_prompt: ' login: '
+        login_timeout: 90
+        username: 'root'
+      BareboxStrategy: {}
+    features:
+      - virtio-mmio
+    runner:
+      tuxmake_arch: arm64
+images:
+  barebox-dt-2nd.img: $LG_BUILDDIR/images/barebox-dt-2nd.img
+tools:
+  qemu: /usr/bin/qemu-system-aarch64
diff --git a/test/arm/vexpress_defconfig.yaml b/test/arm/vexpress_defconfig.yaml
new file mode 120000
index 000000000000..732f51b19dc9
--- /dev/null
+++ b/test/arm/vexpress_defconfig.yaml
@@ -0,0 +1 @@
+a9@vexpress_defconfig.yaml
\ No newline at end of file
diff --git a/test/arm/virt@vexpress_defconfig.yaml b/test/arm/virt@vexpress_defconfig.yaml
new file mode 100644
index 000000000000..9c2e727c9cb5
--- /dev/null
+++ b/test/arm/virt@vexpress_defconfig.yaml
@@ -0,0 +1,25 @@
+targets:
+  main:
+    drivers:
+      QEMUDriver:
+        qemu_bin: qemu
+        machine: virt
+        cpu: cortex-a7
+        memory: 1024M
+        kernel: barebox-dt-2nd.img
+        extra_args: ''
+      BareboxDriver:
+        prompt: 'barebox@[^:]+:[^ ]+ '
+        bootstring: 'commandline:'
+      ShellDriver:
+        prompt: 'root@\w+:[^ ]+ '
+        login_prompt: ' login: '
+        login_timeout: 90
+        username: 'root'
+      BareboxStrategy: {}
+    features:
+      - virtio-mmio
+images:
+  barebox-dt-2nd.img: $LG_BUILDDIR/images/barebox-dt-2nd.img
+tools:
+  qemu: /usr/bin/qemu-system-arm
diff --git a/test/kconfig/virtio-pci.cfg b/test/kconfig/virtio-pci.cfg
new file mode 100644
index 000000000000..78237b8fca26
--- /dev/null
+++ b/test/kconfig/virtio-pci.cfg
@@ -0,0 +1,6 @@
+CONFIG_VIRTIO_MENU=y
+CONFIG_VIRTIO_PCI=y
+CONFIG_VIRTIO_CONSOLE=y
+CONFIG_VIRTIO_BLK=y
+CONFIG_HWRNG=y
+CONFIG_HW_RANDOM_VIRTIO=y
diff --git a/test/mips/be@qemu-malta_defconfig.yaml b/test/mips/be@qemu-malta_defconfig.yaml
new file mode 100644
index 000000000000..5217a6ec2a13
--- /dev/null
+++ b/test/mips/be@qemu-malta_defconfig.yaml
@@ -0,0 +1,25 @@
+targets:
+  main:
+    drivers:
+      QEMUDriver:
+        qemu_bin: qemu
+        machine: malta
+        cpu: M14Kc
+        memory: 256M
+        bios: barebox-qemu-malta.img
+        extra_args: ''
+      BareboxDriver:
+        prompt: 'barebox@[^:]+:[^ ]+ '
+        bootstring: 'commandline:'
+      ShellDriver:
+        prompt: 'root@\w+:[^ ]+ '
+        login_prompt: ' login: '
+        login_timeout: 90
+        username: 'root'
+      BareboxStrategy: {}
+    features:
+      - virtio-pci
+images:
+  barebox-qemu-malta.img: $LG_BUILDDIR/images/barebox-qemu-malta.img
+tools:
+  qemu: /usr/bin/qemu-system-mips
diff --git a/test/mips/le@qemu-malta_defconfig.yaml b/test/mips/le@qemu-malta_defconfig.yaml
new file mode 100644
index 000000000000..3b0f557dde5b
--- /dev/null
+++ b/test/mips/le@qemu-malta_defconfig.yaml
@@ -0,0 +1,28 @@
+targets:
+  main:
+    drivers:
+      QEMUDriver:
+        qemu_bin: qemu
+        machine: malta
+        cpu: M14Kc
+        memory: 256M
+        bios: barebox-qemu-malta.img.swapped
+        extra_args: ''
+      BareboxDriver:
+        prompt: 'barebox@[^:]+:[^ ]+ '
+        bootstring: 'commandline:'
+      ShellDriver:
+        prompt: 'root@\w+:[^ ]+ '
+        login_prompt: ' login: '
+        login_timeout: 90
+        username: 'root'
+      BareboxStrategy: {}
+    features:
+      - virtio-pci
+    runner:
+      kconfig_add:
+        - CONFIG_CPU_LITTLE_ENDIAN=y
+images:
+  barebox-qemu-malta.img.swapped: $LG_BUILDDIR/images/barebox-qemu-malta.img.swapped
+tools:
+  qemu: /usr/bin/qemu-system-mipsel
diff --git a/test/mips/qemu-malta_defconfig.yaml b/test/mips/qemu-malta_defconfig.yaml
new file mode 120000
index 000000000000..481b6e547782
--- /dev/null
+++ b/test/mips/qemu-malta_defconfig.yaml
@@ -0,0 +1 @@
+be@qemu-malta_defconfig.yaml
\ No newline at end of file
diff --git a/test/openrisc/generic_defconfig.yaml b/test/openrisc/generic_defconfig.yaml
new file mode 100644
index 000000000000..ece54da1c191
--- /dev/null
+++ b/test/openrisc/generic_defconfig.yaml
@@ -0,0 +1,23 @@
+targets:
+  main:
+    drivers:
+      QEMUDriver:
+        qemu_bin: qemu
+        machine: or1k-sim
+        cpu: or1200
+        memory: 256M
+        kernel: barebox
+        extra_args: ''
+      BareboxDriver:
+        prompt: 'barebox@[^:]+:[^ ]+ '
+        bootstring: 'commandline:'
+      ShellDriver:
+        prompt: 'root@\w+:[^ ]+ '
+        login_prompt: ' login: '
+        login_timeout: 90
+        username: 'root'
+      BareboxStrategy: {}
+images:
+  barebox: $LG_BUILDDIR/barebox
+tools:
+  qemu: /usr/bin/qemu-system-or1k
diff --git a/test/riscv/qemu@virt32_defconfig.yaml b/test/riscv/qemu@virt32_defconfig.yaml
new file mode 100644
index 000000000000..d879ac6880a7
--- /dev/null
+++ b/test/riscv/qemu@virt32_defconfig.yaml
@@ -0,0 +1,30 @@
+targets:
+  main:
+    drivers:
+      QEMUDriver:
+        qemu_bin: qemu
+        machine: virt
+        cpu: rv32
+        memory: 256M
+        kernel: barebox-dt-2nd.img
+        bios: opensbi-riscv32-generic-fw_dynamic.bin
+        extra_args: ''
+      BareboxDriver:
+        prompt: 'barebox@[^:]+:[^ ]+ '
+        bootstring: 'commandline:'
+      ShellDriver:
+        prompt: 'root@\w+:[^ ]+ '
+        login_prompt: ' login: '
+        login_timeout: 90
+        username: 'root'
+      BareboxStrategy: {}
+    features:
+      - virtio-mmio
+    runner:
+      download:
+        opensbi-riscv32-generic-fw_dynamic.bin: https://github.com/qemu/qemu/blob/v5.2.0/pc-bios/opensbi-riscv32-generic-fw_dynamic.bin?raw=true
+images:
+  barebox-dt-2nd.img: $LG_BUILDDIR/images/barebox-dt-2nd.img
+  opensbi-riscv32-generic-fw_dynamic.bin: $LG_BUILDDIR/opensbi-riscv32-generic-fw_dynamic.bin
+tools:
+  qemu: /usr/bin/qemu-system-riscv32
diff --git a/test/riscv/qemu@virt64_defconfig.yaml b/test/riscv/qemu@virt64_defconfig.yaml
new file mode 100644
index 000000000000..5102f9067e1b
--- /dev/null
+++ b/test/riscv/qemu@virt64_defconfig.yaml
@@ -0,0 +1,30 @@
+targets:
+  main:
+    drivers:
+      QEMUDriver:
+        qemu_bin: qemu
+        machine: virt
+        cpu: rv64
+        memory: 256M
+        kernel: barebox-dt-2nd.img
+        bios: opensbi-riscv64-generic-fw_dynamic.bin
+        extra_args: ''
+      BareboxDriver:
+        prompt: 'barebox@[^:]+:[^ ]+ '
+        bootstring: 'commandline:'
+      ShellDriver:
+        prompt: 'root@\w+:[^ ]+ '
+        login_prompt: ' login: '
+        login_timeout: 90
+        username: 'root'
+      BareboxStrategy: {}
+    features:
+      - virtio-mmio
+    runner:
+      download:
+        opensbi-riscv64-generic-fw_dynamic.bin: https://github.com/qemu/qemu/blob/v5.2.0/pc-bios/opensbi-riscv64-generic-fw_dynamic.bin?raw=true
+images:
+  barebox-dt-2nd.img: $LG_BUILDDIR/images/barebox-dt-2nd.img
+  opensbi-riscv64-generic-fw_dynamic.bin: $LG_BUILDDIR/opensbi-riscv64-generic-fw_dynamic.bin
+tools:
+  qemu: /usr/bin/qemu-system-riscv64
diff --git a/test/riscv/sifive_defconfig.yaml b/test/riscv/sifive_defconfig.yaml
new file mode 100644
index 000000000000..534d7b8b044a
--- /dev/null
+++ b/test/riscv/sifive_defconfig.yaml
@@ -0,0 +1,28 @@
+targets:
+  main:
+    drivers:
+      QEMUDriver:
+        qemu_bin: qemu
+        machine: sifive_u
+        cpu: sifive-u54
+        memory: 256M
+        kernel: barebox-hifive-unleashed.img
+        bios: opensbi-riscv64-generic-fw_dynamic.bin
+        extra_args: ''
+      BareboxDriver:
+        prompt: 'barebox@[^:]+:[^ ]+ '
+        bootstring: 'commandline:'
+      ShellDriver:
+        prompt: 'root@\w+:[^ ]+ '
+        login_prompt: ' login: '
+        login_timeout: 90
+        username: 'root'
+      BareboxStrategy: {}
+    runner:
+      download:
+        opensbi-riscv64-generic-fw_dynamic.bin: https://github.com/qemu/qemu/blob/v5.2.0/pc-bios/opensbi-riscv64-generic-fw_dynamic.bin?raw=true
+images:
+  barebox-hifive-unleashed.img: $LG_BUILDDIR/images/barebox-hifive-unleashed.img
+  opensbi-riscv64-generic-fw_dynamic.bin: $LG_BUILDDIR/opensbi-riscv64-generic-fw_dynamic.bin
+tools:
+  qemu: /usr/bin/qemu-system-riscv64
diff --git a/test/riscv/tinyemu@virt32_defconfig.yaml b/test/riscv/tinyemu@virt32_defconfig.yaml
new file mode 100644
index 000000000000..807a8ce47145
--- /dev/null
+++ b/test/riscv/tinyemu@virt32_defconfig.yaml
@@ -0,0 +1,25 @@
+targets:
+  main:
+    drivers:
+      TinyEMUDriver:
+        temu_bin: temu
+        config: ./Documentation/boards/riscv/barebox-virt32.cfg
+        image: barebox-dt-2nd.img
+      BareboxDriver:
+        prompt: 'barebox@[^:]+:[^ ]+ '
+        bootstring: 'commandline:'
+      ShellDriver:
+        prompt: 'root@\w+:[^ ]+ '
+        login_prompt: ' login: '
+        login_timeout: 90
+        username: 'root'
+      BareboxStrategy: {}
+    features:
+      - virtio-mmio
+    runner:
+      download:
+        bbl32.bin: https://barebox.org/jsbarebox/bbl32.bin
+images:
+  barebox-dt-2nd.img: $LG_BUILDDIR/images/barebox-dt-2nd.img
+tools:
+  temu: /usr/local/bin/temu
diff --git a/test/riscv/tinyemu@virt64_defconfig.yaml b/test/riscv/tinyemu@virt64_defconfig.yaml
new file mode 100644
index 000000000000..58277d1ce0dc
--- /dev/null
+++ b/test/riscv/tinyemu@virt64_defconfig.yaml
@@ -0,0 +1,25 @@
+targets:
+  main:
+    drivers:
+      TinyEMUDriver:
+        temu_bin: temu
+        config: ./Documentation/boards/riscv/barebox-virt64.cfg
+        image: barebox-dt-2nd.img
+      BareboxDriver:
+        prompt: 'barebox@[^:]+:[^ ]+ '
+        bootstring: 'commandline:'
+      ShellDriver:
+        prompt: 'root@\w+:[^ ]+ '
+        login_prompt: ' login: '
+        login_timeout: 90
+        username: 'root'
+      BareboxStrategy: {}
+    features:
+      - virtio-mmio
+    runner:
+      download:
+        bbl64.bin: https://barebox.org/jsbarebox/bbl64.bin
+images:
+  barebox-dt-2nd.img: $LG_BUILDDIR/images/barebox-dt-2nd.img
+tools:
+  temu: /usr/local/bin/temu
diff --git a/test/riscv/virt32_defconfig.yaml b/test/riscv/virt32_defconfig.yaml
new file mode 120000
index 000000000000..e9d7237f5339
--- /dev/null
+++ b/test/riscv/virt32_defconfig.yaml
@@ -0,0 +1 @@
+qemu@virt32_defconfig.yaml
\ No newline at end of file
diff --git a/test/riscv/virt64_defconfig.yaml b/test/riscv/virt64_defconfig.yaml
new file mode 120000
index 000000000000..ab419d5e7fb4
--- /dev/null
+++ b/test/riscv/virt64_defconfig.yaml
@@ -0,0 +1 @@
+qemu@virt64_defconfig.yaml
\ No newline at end of file
diff --git a/test/sandbox/sandbox_defconfig.yaml b/test/sandbox/sandbox_defconfig.yaml
new file mode 100644
index 000000000000..68826e8b5830
--- /dev/null
+++ b/test/sandbox/sandbox_defconfig.yaml
@@ -0,0 +1,12 @@
+targets:
+  main:
+    drivers:
+      NativeExecutableDriver:
+        command: ./barebox
+        image: barebox
+    runner:
+      tuxmake_arch: um
+images:
+  barebox: $LG_BUILDDIR/barebox
+tools:
+  qemu: /usr/local/bin/temu
diff --git a/test/x86/efi_defconfig.yaml b/test/x86/efi_defconfig.yaml
new file mode 120000
index 000000000000..942dc259d56b
--- /dev/null
+++ b/test/x86/efi_defconfig.yaml
@@ -0,0 +1 @@
+pc@efi_defconfig.yaml
\ No newline at end of file
diff --git a/test/x86/pc@efi_defconfig.yaml b/test/x86/pc@efi_defconfig.yaml
new file mode 100644
index 000000000000..923b0680fff2
--- /dev/null
+++ b/test/x86/pc@efi_defconfig.yaml
@@ -0,0 +1,34 @@
+targets:
+  main:
+    drivers:
+      QEMUDriver:
+        qemu_bin: qemu
+        machine: pc
+        cpu: Nehalem
+        memory: 1024M
+        kernel: barebox.efi
+        bios: OVMF.fd
+        extra_args: ''
+      BareboxDriver:
+        prompt: 'barebox@[^:]+:[^ ]+ '
+        bootstring: 'commandline:'
+      ShellDriver:
+        prompt: 'root@\w+:[^ ]+ '
+        login_prompt: ' login: '
+        login_timeout: 90
+        username: 'root'
+      BareboxStrategy: {}
+    features:
+      - pci
+    runner:
+      tuxmake_arch: x86_64
+      kconfig_add:
+        - CONFIG_DRIVER_SERIAL_NS16550=y
+        - CONFIG_CONSOLE_ACTIVATE_FIRST=y # avoid duplicate output
+      download:
+        OVMF.fd: /usr/share/qemu/OVMF.fd
+images:
+  barebox.efi: $LG_BUILDDIR/barebox.efi
+  OVMF.fd: $LG_BUILDDIR/OVMF.fd
+tools:
+  qemu: /usr/bin/qemu-system-x86_64
diff --git a/test/x86/q35@efi_defconfig.yaml b/test/x86/q35@efi_defconfig.yaml
new file mode 100644
index 000000000000..1cfbd9fd88e3
--- /dev/null
+++ b/test/x86/q35@efi_defconfig.yaml
@@ -0,0 +1,34 @@
+targets:
+  main:
+    drivers:
+      QEMUDriver:
+        qemu_bin: qemu
+        machine: q35
+        cpu: Nehalem
+        memory: 1024M
+        kernel: barebox.efi
+        bios: OVMF.fd
+        extra_args: -global ICH9-LPC.noreboot=false
+      BareboxDriver:
+        prompt: 'barebox@[^:]+:[^ ]+ '
+        bootstring: 'commandline:'
+      ShellDriver:
+        prompt: 'root@\w+:[^ ]+ '
+        login_prompt: ' login: '
+        login_timeout: 90
+        username: 'root'
+      BareboxStrategy: {}
+    features:
+      - pci
+    runner:
+        tuxmake_arch: x86_64
+        kconfig_add:
+            - CONFIG_DRIVER_SERIAL_NS16550=y
+            - CONFIG_CONSOLE_ACTIVATE_FIRST=y # avoid duplicate output
+        download:
+            OVMF.fd: /usr/share/qemu/OVMF.fd
+images:
+  barebox.efi: $LG_BUILDDIR/barebox.efi
+  OVMF.fd: $LG_BUILDDIR/OVMF.fd
+tools:
+  qemu: /usr/bin/qemu-system-x86_64
diff --git a/test/x86/virtio@efi_defconfig.yaml b/test/x86/virtio@efi_defconfig.yaml
new file mode 100644
index 000000000000..3c9a9ae51e23
--- /dev/null
+++ b/test/x86/virtio@efi_defconfig.yaml
@@ -0,0 +1,35 @@
+targets:
+  main:
+    drivers:
+      QEMUDriver:
+        qemu_bin: qemu
+        machine: pc
+        cpu: Nehalem
+        memory: 1024M
+        kernel: barebox.efi
+        bios: OVMF.fd
+        extra_args: ''
+      BareboxDriver:
+        prompt: 'barebox@[^:]+:[^ ]+ '
+        bootstring: 'commandline:'
+      ShellDriver:
+        prompt: 'root@\w+:[^ ]+ '
+        login_prompt: ' login: '
+        login_timeout: 90
+        username: 'root'
+      BareboxStrategy: {}
+    features:
+      - virtio-pci
+    runner:
+      tuxmake_arch: x86_64
+      kconfig_add:
+        - test/kconfig/virtio-pci.cfg
+        - CONFIG_DRIVER_SERIAL_NS16550=y
+        - CONFIG_CONSOLE_ACTIVATE_FIRST=y # avoid duplicate output
+      download:
+        OVMF.fd: /usr/share/qemu/OVMF.fd
+images:
+  barebox.efi: $LG_BUILDDIR/barebox.efi
+  OVMF.fd: $LG_BUILDDIR/OVMF.fd
+tools:
+  qemu: /usr/bin/qemu-system-x86_64
-- 
2.29.2


_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox


^ permalink raw reply	[flat|nested] 21+ messages in thread

* [PATCH v2 10/13] test: add first sample tests
  2021-05-31  6:55 [PATCH v2 00/13] add barebox in-tree testing infrastructure Ahmad Fatoum
                   ` (8 preceding siblings ...)
  2021-05-31  6:55 ` [PATCH v2 09/13] test: add labgrid configs for some emulated targets Ahmad Fatoum
@ 2021-05-31  6:55 ` Ahmad Fatoum
  2021-06-02 11:33   ` Rouven Czerwinski
  2021-05-31  6:55 ` [PATCH v2 11/13] test: add emulate.pl, a runner for barebox on emulated targets Ahmad Fatoum
                   ` (2 subsequent siblings)
  12 siblings, 1 reply; 21+ messages in thread
From: Ahmad Fatoum @ 2021-05-31  6:55 UTC (permalink / raw)
  To: barebox; +Cc: Ahmad Fatoum

The test can be run manually with e.g.

  labgrid-pytest --lg-env test/arm/qemu_virt64_defconfig.yaml test/py

Signed-off-by: Ahmad Fatoum <a.fatoum@pengutronix.de>
---
 test/.gitignore       |  1 +
 test/__init__.py      |  0
 test/conftest.py      | 34 ++++++++++++++++++++++++++++++++++
 test/py/__init__.py   |  0
 test/py/helper.py     | 38 ++++++++++++++++++++++++++++++++++++++
 test/py/test_shell.py | 37 +++++++++++++++++++++++++++++++++++++
 6 files changed, 110 insertions(+)
 create mode 100644 test/.gitignore
 create mode 100644 test/__init__.py
 create mode 100644 test/conftest.py
 create mode 100644 test/py/__init__.py
 create mode 100644 test/py/helper.py
 create mode 100644 test/py/test_shell.py

diff --git a/test/.gitignore b/test/.gitignore
new file mode 100644
index 000000000000..bee8a64b79a9
--- /dev/null
+++ b/test/.gitignore
@@ -0,0 +1 @@
+__pycache__
diff --git a/test/__init__.py b/test/__init__.py
new file mode 100644
index 000000000000..e69de29bb2d1
diff --git a/test/conftest.py b/test/conftest.py
new file mode 100644
index 000000000000..5acc1a99e18e
--- /dev/null
+++ b/test/conftest.py
@@ -0,0 +1,34 @@
+import pytest
+import os
+from .py import helper
+
+
+@pytest.fixture(scope='function')
+def barebox(strategy, target):
+    strategy.transition('barebox')
+    return target.get_driver('BareboxDriver')
+
+
+@pytest.fixture(scope='function')
+def shell(strategy, target):
+    strategy.transition('shell')
+    return target.get_driver('ShellDriver')
+
+
+@pytest.fixture(scope="session")
+def barebox_config(strategy, target):
+    strategy.transition('barebox')
+    command = target.get_driver("BareboxDriver")
+    return helper.get_config(command)
+
+def pytest_configure(config):
+    if 'LG_BUILDDIR' not in os.environ:
+        if 'KBUILD_OUTPUT' in os.environ:
+            os.environ['LG_BUILDDIR'] = os.environ['KBUILD_OUTPUT']
+        elif os.path.isdir('build'):
+            os.environ['LG_BUILDDIR'] = os.path.realpath('build')
+        else:
+            os.environ['LG_BUILDDIR'] = os.getcwd()
+
+    if os.environ['LG_BUILDDIR'] is not None:
+        os.environ['LG_BUILDDIR'] = os.path.realpath(os.environ['LG_BUILDDIR'])
diff --git a/test/py/__init__.py b/test/py/__init__.py
new file mode 100644
index 000000000000..e69de29bb2d1
diff --git a/test/py/helper.py b/test/py/helper.py
new file mode 100644
index 000000000000..4a68e83669ba
--- /dev/null
+++ b/test/py/helper.py
@@ -0,0 +1,38 @@
+from labgrid.driver import BareboxDriver
+import pytest
+import os
+from itertools import filterfalse
+
+
+def get_config(command):
+    """Returns the enabled config options of barebox, either from
+    a running instance if supported or by looking into .config
+    in the build directory.
+    Args:
+        command (BareboxDriver): An instance of the BareboxDriver
+    Returns:
+        list: list of the enabled config options
+    """
+    assert isinstance(command, BareboxDriver)
+
+    out, err, returncode = command.run("cat /env/data/config")
+    if returncode != 0:
+        try:
+            with open(os.environ['LG_BUILDDIR'] + "/.config") as f:
+                out = f.read().splitlines()
+        except OSError:
+            return set()
+
+    options = set()
+    for line in out:
+        if line and line.startswith("CONFIG_"):
+            options.add(line.split('=')[0])
+    return options
+
+
+def skip_disabled(config, *options):
+    if bool(config):
+        undefined = list(filterfalse(config.__contains__, options))
+
+        if bool(undefined):
+            pytest.skip("skipping test due to disabled " + (",".join(undefined)) + " dependency")
diff --git a/test/py/test_shell.py b/test/py/test_shell.py
new file mode 100644
index 000000000000..0d2dfe38c5dd
--- /dev/null
+++ b/test/py/test_shell.py
@@ -0,0 +1,37 @@
+import pytest
+from .helper import *
+
+
+def test_barebox_true(barebox, barebox_config):
+    skip_disabled(barebox_config, "CONFIG_CMD_TRUE")
+
+    _, _, returncode = barebox.run('true')
+    assert returncode == 0
+
+
+def test_barebox_false(barebox, barebox_config):
+    skip_disabled(barebox_config, "CONFIG_CMD_FALSE")
+
+    _, _, returncode = barebox.run('false')
+    assert returncode == 1
+
+def test_barebox_md5sum(barebox, barebox_config):
+    skip_disabled(barebox_config, "CONFIG_CMD_MD5SUM", "CONFIG_CMD_ECHO")
+
+    barebox.run_check("echo -o md5 test")
+    out = barebox.run_check("md5sum md5")
+    assert out == ["d8e8fca2dc0f896fd7cb4cb0031ba249  md5"]
+
+def test_barebox_version(barebox, barebox_config):
+    skip_disabled(barebox_config, "CONFIG_CMD_VERSION")
+
+    stdout, _, returncode = barebox.run('version')
+    assert 'barebox' in stdout[1]
+    assert returncode == 0
+
+def test_barebox_no_err(barebox, barebox_config):
+    skip_disabled(barebox_config, "CONFIG_CMD_DMESG")
+
+    # TODO extend by err once all qemu platforms conform
+    stdout, _, _ = barebox.run('dmesg -l crit,alert,emerg')
+    assert stdout == []
-- 
2.29.2


_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox


^ permalink raw reply	[flat|nested] 21+ messages in thread

* [PATCH v2 11/13] test: add emulate.pl, a runner for barebox on emulated targets
  2021-05-31  6:55 [PATCH v2 00/13] add barebox in-tree testing infrastructure Ahmad Fatoum
                   ` (9 preceding siblings ...)
  2021-05-31  6:55 ` [PATCH v2 10/13] test: add first sample tests Ahmad Fatoum
@ 2021-05-31  6:55 ` Ahmad Fatoum
  2021-05-31  6:55 ` [PATCH v2 12/13] test: self: run selftests as part of the pytest suite Ahmad Fatoum
  2021-05-31  6:55 ` [PATCH v2 13/13] test: add bthread test Ahmad Fatoum
  12 siblings, 0 replies; 21+ messages in thread
From: Ahmad Fatoum @ 2021-05-31  6:55 UTC (permalink / raw)
  To: barebox; +Cc: Ahmad Fatoum

This script is a wrapper around tuxmake, qemu and labgrid-pytest.

It parses the same YAML files, labgrid-pytest uses and instructs
tuxmake to build and collect the needed images.

By default, it will start an interactive emulator session, but with
--test, it can also run labgrid-pytest instead.

The script has some knowledge of QEMU options to make common tasks like
passing images straight-forward.

Script is written with both manual use and CI in mind.

Signed-off-by: Ahmad Fatoum <a.fatoum@pengutronix.de>
---
 Documentation/boards/emulated.rst |  70 ++++
 test/emulate.pl                   | 509 ++++++++++++++++++++++++++++++
 test/kconfig/base.cfg             |   0
 test/kconfig/full.cfg             |   0
 4 files changed, 579 insertions(+)
 create mode 100644 Documentation/boards/emulated.rst
 create mode 100755 test/emulate.pl
 create mode 100644 test/kconfig/base.cfg
 create mode 100644 test/kconfig/full.cfg

diff --git a/Documentation/boards/emulated.rst b/Documentation/boards/emulated.rst
new file mode 100644
index 000000000000..37f5038560a6
--- /dev/null
+++ b/Documentation/boards/emulated.rst
@@ -0,0 +1,70 @@
+Emulated targets
+================
+
+Some targets barebox runs on are virtualized by emulators like QEMU, which
+allows basic testing of barebox functionality without the actual hardware.
+
+Generic DT image
+----------------
+
+Supported for ARM and RISC-V. It generates a barebox image that can
+be booted with the Linux kernel booting convention, which makes
+it suitable to be passed as argument to the QEMU ``-kernel`` option.
+The device tree is either the QEMU internal device tree or a device
+tree supplied by QEMU's ``-dtb`` option. The former can be very useful
+with :ref:`virtio_sect`, because QEMU will fix up the virtio mmio
+regions into the device tree and barebox will discover the devices
+automatically, analogously to what it does with VirtIO over PCI.
+
+test/emulate.pl
+---------------
+
+The ``emulate.pl`` script shipped with barebox can be used to easily
+start VMs. It reads a number of YAML files in ``test/``, which describe
+some virtualized targets that barebox is known to run on.
+Controlled by command line options, these targets are built with
+tuxmake if available and loaded into the emulator for either interactive
+use or for automated testing with Labgrid ``QEMUDriver``.
+
+.. _tuxmake: https://pypi.org/project/tuxmake/
+.. _Labgrid: https://labgrid.org
+
+Install dependencies for interactive use::
+
+  cpan YAML::Tiny # or use e.g. libyaml-tiny-perl on debian
+  pip3 install tuxmake # optional
+
+Example usage::
+
+  # Switch to barebox source directory
+  cd barebox
+
+  # emulate ARM virt with an image built by vexpress_defconfig
+  ARCH=arm ./test/emulate.pl virt@vexpress_defconfig
+
+  # build all MIPS targets known to emulate.pl and exit
+  ARCH=mips ./test/emulate.pl --no-emulate
+
+The script can also be used with a precompiled barebox tree::
+
+  # Switch to build directory
+  export KBUILD_OUTPUT=build
+
+  # run a barebox image built outside tuxmake with an emulated vexpress-ca15
+  ARCH=x86 ./test/emulate.pl efi_defconfig --no-tuxmake
+
+  # run tests instead of starting emulator interactively
+  ARCH=x86 ./test/emulate.pl efi_defconfig --no-tuxmake --test
+
+``emulate.pl`` also has some knowledge on paravirtualized devices::
+
+  # Run target and pass a block device (here /dev/virtioblk0)
+  ARCH=riscv ./test/emulate.pl --blk=rootfs.ext4 virt64_defconfig
+
+``emulate.pl`` if needed command line options can be passed directly
+to the emulator/``pytest`` as well by placing them behind ``--``::
+
+  # appends -device ? to the command line. Add -n to see the final result
+  ARCH=riscv ./test/emulate.pl virt64_defconfig -- -device ?
+
+For a complete listing of options run ``./test/emulate.pl -h``.
diff --git a/test/emulate.pl b/test/emulate.pl
new file mode 100755
index 000000000000..6eb166ee1111
--- /dev/null
+++ b/test/emulate.pl
@@ -0,0 +1,509 @@
+#!/usr/bin/env perl
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2021 Ahmad Fatoum
+
+use strict;
+use warnings;
+
+use Cwd;
+use File::Basename;
+use File::Spec;
+use File::Temp 'tempdir';
+use Getopt::Long;
+use List::Util 'first';
+use Pod::Usage;
+use YAML::Tiny;
+
+my @QEMU_INTERACTIVE_OPTS = qw(-serial mon:stdio -trace file=/dev/null);
+
+my %targets;
+
+for my $arch (glob dirname(__FILE__) . "/*/") {
+    for my $cfg (glob "$arch/*.yaml") {
+	my $linkdest = readlink $cfg // '';
+
+	my $yaml = YAML::Tiny->read($cfg);
+
+	defined $yaml && exists $yaml->[0]{targets} && exists $yaml->[0]{targets}{main}
+	    or die "Invalid structure for $cfg\n";
+
+	my $path = File::Spec->abs2rel($cfg, getcwd);
+	$cfg = basename($cfg);
+	$cfg =~ s/\.yaml$//;
+	$linkdest =~ s{^.*?([^/]+)\.yaml$}{$1};
+
+	$targets{basename $arch}{$cfg} = $yaml->[0]{targets}{main};
+	$targets{basename $arch}{$cfg}{path} = $path;
+	$targets{basename $arch}{$cfg}{tools} = $yaml->[0]{tools};
+	$targets{basename $arch}{$cfg}{images} = $yaml->[0]{images};
+	$targets{basename $arch}{$cfg}{alias} = $linkdest if $linkdest && $linkdest ne $cfg;
+    }
+}
+
+my %arch_aliases = (arm64 => 'arm', x86_64 => 'x86', i386 => 'x86', um => 'sandbox');
+
+my ($dryrun, $help, @blks, $rng, $list, $shell, $runtime, @kconfig_add, $artifacts);
+my ($tuxmake, $emulate, $clean, $extraconsoles, $test) = (1, 1, 1, 0, 0);
+my ($kconfig_base, $kconfig_full) = (1, 0);
+
+my @OPTS;
+
+if (defined (my $idx = first { $ARGV[$_] eq '--' } 0 .. @ARGV - 1)) {
+    @OPTS = splice @ARGV, 1 + $idx;
+}
+
+GetOptions(
+	'help|?|h' => \$help,
+	'dryrun|n' => \$dryrun,
+	'list|l' => \$list,
+	'tuxmake!' => \$tuxmake,
+	'artifacts=s' => \$artifacts,
+	'runtime=s' => \$runtime,
+	'blk=s@' => \@blks,
+	'console+' => \$extraconsoles,
+	'rng' => \$rng,
+	'emulate!' => \$emulate,
+	'clean!' => \$clean,
+	'shell' => \$shell,
+	'kconfig-base!' => \$kconfig_base,
+	'kconfig-full!' => \$kconfig_full,
+	'kconfig-add|K=s@' => \@kconfig_add,
+	'test' => \$test,
+) or pod2usage(2);
+pod2usage(1) if $help;
+
+my @ARCH = split /\s/, $ENV{ARCH} // '';
+@ARCH = @ARCH ? @ARCH : keys %targets;
+
+my $success = 0;
+
+for my $arch (@ARCH) {
+	my @targets = @ARGV ? @ARGV : keys %{$targets{$arch}};
+	@targets != 1 && !$tuxmake
+		and die "can't use --no-tuxmake with more than one config\n";
+
+	unless (defined $targets{$arch}) {
+		die "Unknown ARCH=$arch. Supported values:\n",
+		join(', ', keys %targets), "\n";
+	}
+
+	for my $config (@targets) {
+	        $arch = $arch_aliases{$arch} // $arch;
+
+		$config = fileparse($config, ".yaml");
+
+		unless (defined $targets{$arch}{$config}) {
+		    next;
+		}
+
+		if ($list) {
+		    print "ARCH=$arch $config\n";
+		    $success += 1;
+		    next;
+		}
+
+		if (defined $targets{$arch}{$config}{alias}) {
+		    next if grep { /^$targets{$arch}{$config}{alias}$/ } @targets;
+		    $config = $targets{$arch}{$config}{alias};
+		    redo;
+		}
+
+		print "# $config\n" if $dryrun;
+		$success += process($arch, $config);
+	}
+}
+
+$success > 0
+	or die "No suitable config found. $0 -l to list all built-in.\n";
+
+sub process {
+	my ($ARCH, $defconfig, %keys) = @_;
+	my $target = $targets{$ARCH}{$defconfig};
+
+	if (!exists ($target->{runner}{tuxmake_arch})) {
+	    $target->{runner}{tuxmake_arch} = $ARCH;
+	}
+
+	my $dir;
+
+	$dir = tempdir("bareboxbuild-XXXXXXXXXX", TMPDIR => 1, CLEANUP => $clean);
+	report('mkdir', $dir);
+
+	my %opts = (
+	    target => $target, builddir => $tuxmake ? $dir : getcwd,
+	    defconfig => $defconfig,
+	    extra_opts => [map { s/\{config\}/$defconfig/gr  } @OPTS],
+	);
+
+	build(%opts) if $tuxmake;
+
+	while (my ($k, $v) = each %{$target->{runner}{download}}) {
+	    if ($v =~ m{^[/.]}) {
+		symlink_force($v, "$dir/$k");
+	    } else {
+		vsystem('curl', '-L', '--create-dirs', '-o', "$dir/$k", $v) == 0
+		    or die "Failed to download resource `$v': $?\n";
+	    }
+
+	    symlink_force("$dir/$k", "$k") unless $tuxmake;
+	}
+
+	if ($shell) {
+		pushd($dir);
+		system($ENV{SHELL} // "/bin/sh");
+		popd();
+	}
+
+	return 1 unless $emulate;
+
+	if ($tuxmake) {
+	    $ENV{KBUILD_OUTPUT} = $dir;
+	    print "export KBUILD_OUTPUT=$ENV{KBUILD_OUTPUT}\n" if $dryrun;
+	}
+
+	my $success = $test ? test(%opts) : emulate(%opts);
+
+        report("rm", "-rd", $dir) if $clean && $tuxmake;
+
+	print "\n\n" if $dryrun;
+	return $success;
+}
+
+sub build {
+    my %args = @_;
+    my ($runner, $dir) = ($args{target}{runner}, $args{builddir});
+
+    $args{defconfig} =~ s/[^@]+@//g;
+
+    my @TUXMAKE_ARGS = ('-a', $runner->{tuxmake_arch}, '-k', $args{defconfig});
+
+    if (defined $runner->{kconfig_add}) {
+	for my $cfg (@{$runner->{kconfig_add}}) {
+	    push @TUXMAKE_ARGS, "--kconfig-add=$cfg"
+	}
+    }
+
+    push @TUXMAKE_ARGS, "--kconfig-add=test/kconfig/base.cfg" if $kconfig_base || $kconfig_full;
+    push @TUXMAKE_ARGS, "--kconfig-add=test/kconfig/full.cfg" if $kconfig_full;
+
+    for (@kconfig_add) {
+	push @TUXMAKE_ARGS, "--kconfig-add=$_";
+    }
+
+    push @TUXMAKE_ARGS, "--runtime=$runtime" if $runtime;
+
+    vsystem('tuxmake', @TUXMAKE_ARGS, '-b', $dir, '-o',
+	$artifacts // "$dir/artifacts", 'default') == 0
+	or die "Error building: $?\n";
+}
+
+sub emulate {
+    my %args = @_;
+    my %target = %{$args{target}};
+    my @OPTS = @{$args{extra_opts}};
+
+    if (defined $target{drivers}{QEMUDriver}) {
+	    my %qemu = %{$target{drivers}{QEMUDriver}};
+	    my ($kernel, $bios, $dtb);
+	    my $qemu_virtio;
+	    my $i;
+
+	    $kernel = abs_configpath($qemu{kernel}, \%args);
+	    $bios   = abs_configpath($qemu{bios},   \%args);
+	    $dtb    = abs_configpath($qemu{dtb},    \%args);
+
+	    my @cmdline = ($target{tools}{$qemu{qemu_bin}},
+		'-M', $qemu{machine}, '-cpu', $qemu{cpu}, '-m', $qemu{memory});
+
+	    push @cmdline, '-kernel', $kernel if defined $kernel;
+	    push @cmdline, '-bios', $bios if defined $bios;
+	    push @cmdline, '-dtb',  $dtb if defined $dtb;
+
+	    push @cmdline, @QEMU_INTERACTIVE_OPTS;
+	    for (split /\s/, $qemu{extra_args}) {
+		push @cmdline, $_;
+	    }
+
+	    if (has_feature(\%target, 'virtio-pci')) {
+		    $qemu_virtio = 'pci,disable-legacy=on,disable-modern=off';
+		    push @{$target{features}}, 'pci';
+		    push @{$target{features}}, 'virtio';
+	    } elsif (has_feature(\%target, 'virtio-mmio')) {
+		    $qemu_virtio = 'device';
+		    push @{$target{features}}, 'virtio';
+	    }
+
+	    $i = 0;
+	    for my $blk (@blks) {
+		if (has_feature(\%target, 'virtio')) {
+		    $blk = rel2abs($blk);
+		    push @cmdline,
+		    '-drive', "if=none,file=$blk,format=raw,id=hd$i",
+		    '-device', "virtio-blk-$qemu_virtio,drive=hd$i";
+		} else {
+		    die "--blk unsupported for target\n";
+		}
+	    }
+
+	    # note that barebox doesn't yet support multiple virtio consoles
+	    if ($extraconsoles) {
+		$i = 0;
+
+		if (defined $qemu_virtio) {
+		    push @cmdline,
+		    '-device', "virtio-serial-$qemu_virtio",
+		    '-chardev', "pty,id=virtcon$i",
+		    '-device', "virtconsole,chardev=virtcon$i,name=console.virtcon$i";
+
+		    $i++;
+		}
+
+		if ($i < $extraconsoles) {
+		    # ns16550 serial driver only works with x86 so far
+		    if (has_feature(\%target, 'pci')) {
+			for (; $i < $extraconsoles; $i++) {
+			    push @cmdline,
+			    '-chardev', "pty,id=pcicon$i",
+			    '-device', "pci-serial,chardev=pcicon$i";
+			}
+		    } else {
+			warn "barebox currently supports only a single extra virtio console\n";
+		    }
+		}
+	    }
+
+	    if (defined $rng) {
+		if (has_feature(\%target, 'virtio')) {
+		    push @cmdline, '-device', "virtio-rng-$qemu_virtio";
+		} else {
+		    die "--rng unsupported for target\n";
+		}
+	    }
+
+	    pushd($args{builddir}) if $tuxmake;
+
+	    vsystem(@cmdline, @OPTS) == 0 or die "Error running emulator: $?\n";
+
+    } elsif (defined $target{drivers}{TinyEMUDriver}) {
+	    my %temu = %{$target{drivers}{TinyEMUDriver}};
+	    my $TEMU_CFG;
+	    my $i = 0;
+
+	    if (exists $temu{config}) {
+		$temu{config} = rel2abs($temu{config});
+		open(my $fh, '<', "$temu{config}")
+		    or die "Could not open file '$temu{config}': $!";
+		$TEMU_CFG = do { local $/; <$fh> };
+	    }
+
+	    print ("$temu{config}\n");
+
+	    defined $TEMU_CFG or die "Unknown tinyemu-config\n";
+
+	    open(my $fh, '>', "$args{builddir}/tinyemu.cfg")
+		or die "Could not create file 'tinyemu.cfg': $!";
+	    print $fh $TEMU_CFG;
+	    print "cat >'tinyemu.cfg' <<EOF\n$TEMU_CFG\nEOF\n" if $dryrun;
+
+	    for my $blk (@blks) {
+		$blk = rel2abs($blk);
+		$TEMU_CFG =~ s[\}(?!.*\})][drive$i: { file: "$blk" },\n}]ms
+	    }
+
+	    die "--console unsupported for target\n" if $extraconsoles;
+	    die "--rng unsupported for target\n" if defined $rng;
+
+	    pushd($args{builddir}) if $tuxmake;
+
+	    vsystem($temu{temu_bin}, "tinyemu.cfg", @OPTS) == 0
+		or die "Error running emulator: $?\n";
+    } elsif (defined $target{drivers}{NativeExecutableDriver}) {
+	    my %exec = %{$target{drivers}{NativeExecutableDriver}};
+
+	    pushd($args{builddir}) if $tuxmake;
+
+	    vsystem($exec{command}, @OPTS) == 0 or die "Error running emulator: $?\n";
+    }
+
+    popd() if $tuxmake;
+
+    return 1;
+}
+
+sub test {
+    my %args = @_;
+    my @OPTS = @{$args{extra_opts}};
+    my $pytest;
+    my $dir;
+
+    unless (defined $args{target}{drivers}{QEMUDriver}) {
+	warn "$args{target}{path}: Skipping test, no QEMUDriver\n";
+	return 0;
+    }
+
+    $pytest = `sh -c 'command -v labgrid-pytest'` ? 'labgrid-pytest' : 'pytest';
+
+    vsystem($pytest, '--lg-env', "$args{target}{path}", "test/py",
+	'--lg-log', @OPTS) == 0 or die "Error running 'labgrid-pytest': $?\n";
+
+    return 1;
+}
+
+sub has_feature {
+    defined first { lc($_) eq $_[1] } @{$_[0]->{features}}
+}
+
+sub report {
+    print join(' ', map { /\s/ ? "'$_'" : $_  } @_), "\n" if $dryrun;
+    0;
+}
+
+sub vsystem {
+    if ($dryrun) {
+	return report(@_);
+    } else {
+	my $ret = system @_;
+	warn "vsystem: $!\n" if $ret == -1;
+	return $ret >> 8;
+    }
+}
+
+sub rel2abs {
+    File::Spec->rel2abs(@_)
+}
+
+sub abs_configpath {
+    my ($path, $args) = @_;
+    my $LG_BUILDDIR;
+
+    return unless defined $path;
+    $path = $args->{target}{images}{$path};
+    return unless defined $path;
+
+    if (exists $ENV{KBUILD_OUTPUT}) {
+	$LG_BUILDDIR = $ENV{KBUILD_OUTPUT};
+    } elsif (-d 'build') {
+	$LG_BUILDDIR = 'build';
+    } else {
+	$LG_BUILDDIR = getcwd();
+    }
+
+    $path =~ s/\$LG_BUILDDIR\b/$LG_BUILDDIR/g;
+
+    return rel2abs($path, $args->{builddir})
+}
+
+sub symlink_force {
+    unlink($_[1]);
+    symlink($_[0], $_[1]);
+}
+
+my @oldcwd;
+
+sub pushd {
+	my ($old, $new) = (getcwd, shift);
+	report ("pushd", $new);
+	push @oldcwd, $old;
+	chdir $new;
+	return $old;
+};
+
+sub popd {
+	report("popd");
+	chdir pop(@oldcwd)
+};
+
+__END__
+
+=head1 NAME
+
+emulate.pl - Build and run barebox under an emulator
+
+=head1 SYNOPSIS
+
+[ARCH=arch] emulate.pl [options] [defconfigs...] [--] [qemu/pytest-opts]
+
+=head1 OPTIONS
+
+Must be run from either barebox source or build directory.
+
+=over 8
+
+=item B<-?>, B<-h>, B<--help>
+
+Print a help message and exits
+
+=item B<-n>, B<--dryrun>
+
+Print commands and exit
+
+=item B<-l>, B<--list>
+
+Filter input with list of known targets
+
+=item B<--no-tuxmake>
+
+Don't rerun tuxkmake. Assumes current working directory is finished build directory
+
+=item B<--artifacts>=%s
+
+Destination directory for the tuxmake artifacts. By default, this is
+the artifacts/ subdirectory in the temporary build directory and is
+not persisted (unless --no-clean is specified).
+
+=item B<--blk>=%s
+
+Pass block device to emulated barebox. Can be specified more than once
+
+=item B<--console>
+
+Pass one Virt I/O console to emulated barebox.
+
+=item B<--rng>
+
+instantiate Virt I/O random number generator
+
+=item B<--no-emulate>
+
+Don't emulate anything and exit directly after build
+
+=item B<--no-clean>
+
+Don't delete temporary working directory after
+
+=item B<--shell>
+
+Open shell in temporary working directory, after build, but before emulation
+
+=item B<--test>
+
+Instead of starting emulator interactively, run it under labgrid-pytest with
+the in-tree python tests.
+
+=item B<--runtime>=%s
+
+Runtime to use for the tuxmake builds. By default, builds are
+run natively on the build host.
+Supported: null, podman-local, podman, docker, docker-local.
+
+=item B<--no-kconfig-base>
+
+Don't apply test/kconfig/base.cfg. This may lead to more tests being
+skipped.
+
+=item B<--kconfig-full>
+
+Applies test/kconfig/full.cfg on top of base.cfg. This enables as much as
+possible to avoid skipping tests for disabled functionality.
+
+=item B<--kconfig_add>=%s, B<-K>=%s
+
+Extra kconfig fragments, merged on top of the defconfig and Kconfig
+fragments described by the YAML. In tree configuration fragment
+(e.g. `test/kconfig/virtio-pci.config`), path to local file, URL,
+`CONFIG_*=[y|m|n]`, or `# CONFIG_* is not set` are supported.
+Can be specified multiple times, and will be merged in the order given.
+
+=back
+
+=cut
diff --git a/test/kconfig/base.cfg b/test/kconfig/base.cfg
new file mode 100644
index 000000000000..e69de29bb2d1
diff --git a/test/kconfig/full.cfg b/test/kconfig/full.cfg
new file mode 100644
index 000000000000..e69de29bb2d1
-- 
2.29.2


_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox


^ permalink raw reply	[flat|nested] 21+ messages in thread

* [PATCH v2 12/13] test: self: run selftests as part of the pytest suite
  2021-05-31  6:55 [PATCH v2 00/13] add barebox in-tree testing infrastructure Ahmad Fatoum
                   ` (10 preceding siblings ...)
  2021-05-31  6:55 ` [PATCH v2 11/13] test: add emulate.pl, a runner for barebox on emulated targets Ahmad Fatoum
@ 2021-05-31  6:55 ` Ahmad Fatoum
  2021-06-02 11:36   ` Rouven Czerwinski
  2021-05-31  6:55 ` [PATCH v2 13/13] test: add bthread test Ahmad Fatoum
  12 siblings, 1 reply; 21+ messages in thread
From: Ahmad Fatoum @ 2021-05-31  6:55 UTC (permalink / raw)
  To: barebox; +Cc: Ahmad Fatoum

We don't want to enable self tests in the normal configs as they may,
in future, bloat size needlessly. Enable it instead in the base.cfg
fragment and add a test that verifies the selftest command
runs without errors.

Selftests can be run on startup for CONFIG_SHELL_NONE systems. This is
not implemented here. For such systems the test will be skipped
as CONFIG_CMD_SELFTEST won't be defined. To manually skip with
emulate.pl, add --no-kconfig-base.

Signed-off-by: Ahmad Fatoum <a.fatoum@pengutronix.de>
---
 test/kconfig/base.cfg      | 4 ++++
 test/py/test_bselftests.py | 8 ++++++++
 2 files changed, 12 insertions(+)
 create mode 100644 test/py/test_bselftests.py

diff --git a/test/kconfig/base.cfg b/test/kconfig/base.cfg
index e69de29bb2d1..6a9f68349816 100644
--- a/test/kconfig/base.cfg
+++ b/test/kconfig/base.cfg
@@ -0,0 +1,4 @@
+CONFIG_TEST=y
+CONFIG_SELFTEST=y
+CONFIG_CMD_SELFTEST=y
+CONFIG_SELFTEST_ENABLE_ALL=y
diff --git a/test/py/test_bselftests.py b/test/py/test_bselftests.py
new file mode 100644
index 000000000000..48e9d38c12fc
--- /dev/null
+++ b/test/py/test_bselftests.py
@@ -0,0 +1,8 @@
+import pytest
+from .helper import *
+
+def test_bselftest(barebox, barebox_config):
+    skip_disabled(barebox_config, "CONFIG_CMD_SELFTEST")
+
+    _, _, returncode = barebox.run('selftest')
+    assert returncode == 0
-- 
2.29.2


_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox


^ permalink raw reply	[flat|nested] 21+ messages in thread

* [PATCH v2 13/13] test: add bthread test
  2021-05-31  6:55 [PATCH v2 00/13] add barebox in-tree testing infrastructure Ahmad Fatoum
                   ` (11 preceding siblings ...)
  2021-05-31  6:55 ` [PATCH v2 12/13] test: self: run selftests as part of the pytest suite Ahmad Fatoum
@ 2021-05-31  6:55 ` Ahmad Fatoum
  12 siblings, 0 replies; 21+ messages in thread
From: Ahmad Fatoum @ 2021-05-31  6:55 UTC (permalink / raw)
  To: barebox; +Cc: Ahmad Fatoum

The bthread -v command already returns an appropriate exit code if
threads can't be scheduled. Have it spawn 16 threads and switch between
them for a second and verify that all of these threads were indeed
terminated.

Signed-off-by: Ahmad Fatoum <a.fatoum@pengutronix.de>
---
 test/kconfig/full.cfg   |  2 ++
 test/py/test_bthread.py | 23 +++++++++++++++++++++++
 2 files changed, 25 insertions(+)
 create mode 100644 test/py/test_bthread.py

diff --git a/test/kconfig/full.cfg b/test/kconfig/full.cfg
index e69de29bb2d1..39275768ea1f 100644
--- a/test/kconfig/full.cfg
+++ b/test/kconfig/full.cfg
@@ -0,0 +1,2 @@
+CONFIG_BTHREAD=y
+CONFIG_CMD_BTHREAD=y
diff --git a/test/py/test_bthread.py b/test/py/test_bthread.py
new file mode 100644
index 000000000000..6e7b4ba5007e
--- /dev/null
+++ b/test/py/test_bthread.py
@@ -0,0 +1,23 @@
+import pytest
+from .helper import *
+
+def stale_spawners(barebox):
+    threads = barebox.run_check("bthread -i")
+    if len(threads) == 0:
+        return False
+    return len([t for t in threads if t.startswith('spawner')]) > 0
+
+def test_bthread(barebox, barebox_config):
+    skip_disabled(barebox_config, "CONFIG_CMD_BTHREAD")
+
+    assert not stale_spawners(barebox)
+
+    _, _, returncode = barebox.run('bthread -vvvv')
+    assert returncode == 0
+
+    assert not stale_spawners(barebox)
+
+    switches = int(barebox.run_check("bthread -c")[0].split()[0])
+    yields   = int(barebox.run_check("bthread -t")[0].split()[0])
+
+    assert yields < switches
-- 
2.29.2


_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [PATCH v2 09/13] test: add labgrid configs for some emulated targets
  2021-05-31  6:55 ` [PATCH v2 09/13] test: add labgrid configs for some emulated targets Ahmad Fatoum
@ 2021-06-02 11:27   ` Rouven Czerwinski
  2021-06-02 11:34     ` Ahmad Fatoum
  0 siblings, 1 reply; 21+ messages in thread
From: Rouven Czerwinski @ 2021-06-02 11:27 UTC (permalink / raw)
  To: Ahmad Fatoum, barebox

On Mon, 2021-05-31 at 08:55 +0200, Ahmad Fatoum wrote:
> There are various ways barebox can run under emulation, some of them are
> described in the documentation and some more in the git history.
> 
> Make running the emulators more convenient to use by collecting some
> known-good emulator invocations in a machine readable format.
> 
> These YAML files can be parsed by labgrid for running tests or executed
> by a new test/emulate.pl runner script that will be added in a follow-up
> commit.
> 
> Using labgrid for this will allow using the same test suite for physical
> targets as well in future.
> 
> Signed-off-by: Ahmad Fatoum <a.fatoum@pengutronix.de>
> ---
>  test/arm/a15@vexpress_defconfig.yaml     | 23 ++++++++++++++++
>  test/arm/a9@vexpress_defconfig.yaml      | 23 ++++++++++++++++
>  test/arm/qemu_virt64_defconfig.yaml      | 27 ++++++++++++++++++
>  test/arm/vexpress_defconfig.yaml         |  1 +
>  test/arm/virt@vexpress_defconfig.yaml    | 25 +++++++++++++++++
>  test/kconfig/virtio-pci.cfg              |  6 ++++
>  test/mips/be@qemu-malta_defconfig.yaml   | 25 +++++++++++++++++
>  test/mips/le@qemu-malta_defconfig.yaml   | 28 +++++++++++++++++++
>  test/mips/qemu-malta_defconfig.yaml      |  1 +
>  test/openrisc/generic_defconfig.yaml     | 23 ++++++++++++++++
>  test/riscv/qemu@virt32_defconfig.yaml    | 30 ++++++++++++++++++++
>  test/riscv/qemu@virt64_defconfig.yaml    | 30 ++++++++++++++++++++
>  test/riscv/sifive_defconfig.yaml         | 28 +++++++++++++++++++
>  test/riscv/tinyemu@virt32_defconfig.yaml | 25 +++++++++++++++++
>  test/riscv/tinyemu@virt64_defconfig.yaml | 25 +++++++++++++++++
>  test/riscv/virt32_defconfig.yaml         |  1 +
>  test/riscv/virt64_defconfig.yaml         |  1 +
>  test/sandbox/sandbox_defconfig.yaml      | 12 ++++++++
>  test/x86/efi_defconfig.yaml              |  1 +
>  test/x86/pc@efi_defconfig.yaml           | 34 +++++++++++++++++++++++
>  test/x86/q35@efi_defconfig.yaml          | 34 +++++++++++++++++++++++
>  test/x86/virtio@efi_defconfig.yaml       | 35 ++++++++++++++++++++++++
>  22 files changed, 438 insertions(+)
>  create mode 100644 test/arm/a15@vexpress_defconfig.yaml
>  create mode 100644 test/arm/a9@vexpress_defconfig.yaml
>  create mode 100644 test/arm/qemu_virt64_defconfig.yaml
>  create mode 120000 test/arm/vexpress_defconfig.yaml
>  create mode 100644 test/arm/virt@vexpress_defconfig.yaml
>  create mode 100644 test/kconfig/virtio-pci.cfg
>  create mode 100644 test/mips/be@qemu-malta_defconfig.yaml
>  create mode 100644 test/mips/le@qemu-malta_defconfig.yaml
>  create mode 120000 test/mips/qemu-malta_defconfig.yaml
>  create mode 100644 test/openrisc/generic_defconfig.yaml
>  create mode 100644 test/riscv/qemu@virt32_defconfig.yaml
>  create mode 100644 test/riscv/qemu@virt64_defconfig.yaml
>  create mode 100644 test/riscv/sifive_defconfig.yaml
>  create mode 100644 test/riscv/tinyemu@virt32_defconfig.yaml
>  create mode 100644 test/riscv/tinyemu@virt64_defconfig.yaml
>  create mode 120000 test/riscv/virt32_defconfig.yaml
>  create mode 120000 test/riscv/virt64_defconfig.yaml
>  create mode 100644 test/sandbox/sandbox_defconfig.yaml
>  create mode 120000 test/x86/efi_defconfig.yaml
>  create mode 100644 test/x86/pc@efi_defconfig.yaml
>  create mode 100644 test/x86/q35@efi_defconfig.yaml
>  create mode 100644 test/x86/virtio@efi_defconfig.yaml
> 
> diff --git a/test/arm/a15@vexpress_defconfig.yaml b/test/arm/a15@vexpress_defconfig.yaml
> new file mode 100644
> index 000000000000..7ed93ba0a19f
> --- /dev/null
> +++ b/test/arm/a15@vexpress_defconfig.yaml
> @@ -0,0 +1,23 @@
> +targets:
> +  main:
> +    drivers:
> +      QEMUDriver:
> +        qemu_bin: qemu
> +        machine: vexpress-a15
> +        cpu: cortex-a15
> +        memory: 1024M
> +        bios: barebox-vexpress-ca15.img
> +        extra_args: ''
> +      BareboxDriver:
> +        prompt: 'barebox@[^:]+:[^ ]+ '
> +        bootstring: 'commandline:'
> +      ShellDriver:
> +        prompt: 'root@\w+:[^ ]+ '
> +        login_prompt: ' login: '
> +        login_timeout: 90
> +        username: 'root'

All of these targets use a Shelldriver, but it is used only to satisfy
the BareboxStrategy. Scrap the ShellDriver and write a short strategy
or scrap the strategy as well or wholesale remove strategy usage
alltogether. Since all thats done is activating the BareboxDriver, this
should be doable without a strategy.

> +      BareboxStrategy: {}
> +images:
> +  barebox-vexpress-ca15.img: $LG_BUILDDIR/images/barebox-vexpress-ca15.img
> +tools:
> +  qemu: /usr/bin/qemu-system-arm
> diff --git a/test/arm/a9@vexpress_defconfig.yaml b/test/arm/a9@vexpress_defconfig.yaml
> new file mode 100644
> index 000000000000..88e09ca052e6
> --- /dev/null
> +++ b/test/arm/a9@vexpress_defconfig.yaml
> @@ -0,0 +1,23 @@
> +targets:
> +  main:
> +    drivers:
> +      QEMUDriver:
> +        qemu_bin: qemu
> +        machine: vexpress-a9
> +        cpu: cortex-a9
> +        memory: 1024M
> +        bios: barebox-vexpress-ca9.img
> +        extra_args: ''
> +      BareboxDriver:
> +        prompt: 'barebox@[^:]+:[^ ]+ '
> +        bootstring: 'commandline:'
> +      ShellDriver:
> +        prompt: 'root@\w+:[^ ]+ '
> +        login_prompt: ' login: '
> +        login_timeout: 90
> +        username: 'root'
> +      BareboxStrategy: {}
> +images:
> +  barebox-vexpress-ca9.img: $LG_BUILDDIR/images/barebox-vexpress-ca9.img
> +tools:
> +  qemu: /usr/bin/qemu-system-arm
> diff --git a/test/arm/qemu_virt64_defconfig.yaml b/test/arm/qemu_virt64_defconfig.yaml
> new file mode 100644
> index 000000000000..2dfaaedef61a
> --- /dev/null
> +++ b/test/arm/qemu_virt64_defconfig.yaml
> @@ -0,0 +1,27 @@
> +targets:
> +  main:
> +    drivers:
> +      QEMUDriver:
> +        qemu_bin: qemu
> +        machine: virt
> +        cpu: cortex-a57
> +        memory: 1024M
> +        kernel: barebox-dt-2nd.img
> +        extra_args: ''
> +      BareboxDriver:
> +        prompt: 'barebox@[^:]+:[^ ]+ '
> +        bootstring: 'commandline:'
> +      ShellDriver:
> +        prompt: 'root@\w+:[^ ]+ '
> +        login_prompt: ' login: '
> +        login_timeout: 90
> +        username: 'root'
> +      BareboxStrategy: {}
> +    features:
> +      - virtio-mmio
> +    runner:
> +      tuxmake_arch: arm64
> +images:
> +  barebox-dt-2nd.img: $LG_BUILDDIR/images/barebox-dt-2nd.img
> +tools:
> +  qemu: /usr/bin/qemu-system-aarch64
> diff --git a/test/arm/vexpress_defconfig.yaml b/test/arm/vexpress_defconfig.yaml
> new file mode 120000
> index 000000000000..732f51b19dc9
> --- /dev/null
> +++ b/test/arm/vexpress_defconfig.yaml
> @@ -0,0 +1 @@
> +a9@vexpress_defconfig.yaml
> \ No newline at end of file
> diff --git a/test/arm/virt@vexpress_defconfig.yaml b/test/arm/virt@vexpress_defconfig.yaml
> new file mode 100644
> index 000000000000..9c2e727c9cb5
> --- /dev/null
> +++ b/test/arm/virt@vexpress_defconfig.yaml
> @@ -0,0 +1,25 @@
> +targets:
> +  main:
> +    drivers:
> +      QEMUDriver:
> +        qemu_bin: qemu
> +        machine: virt
> +        cpu: cortex-a7
> +        memory: 1024M
> +        kernel: barebox-dt-2nd.img
> +        extra_args: ''
> +      BareboxDriver:
> +        prompt: 'barebox@[^:]+:[^ ]+ '
> +        bootstring: 'commandline:'
> +      ShellDriver:
> +        prompt: 'root@\w+:[^ ]+ '
> +        login_prompt: ' login: '
> +        login_timeout: 90
> +        username: 'root'
> +      BareboxStrategy: {}
> +    features:
> +      - virtio-mmio
> +images:
> +  barebox-dt-2nd.img: $LG_BUILDDIR/images/barebox-dt-2nd.img
> +tools:
> +  qemu: /usr/bin/qemu-system-arm
> diff --git a/test/kconfig/virtio-pci.cfg b/test/kconfig/virtio-pci.cfg
> new file mode 100644
> index 000000000000..78237b8fca26
> --- /dev/null
> +++ b/test/kconfig/virtio-pci.cfg
> @@ -0,0 +1,6 @@
> +CONFIG_VIRTIO_MENU=y
> +CONFIG_VIRTIO_PCI=y
> +CONFIG_VIRTIO_CONSOLE=y
> +CONFIG_VIRTIO_BLK=y
> +CONFIG_HWRNG=y
> +CONFIG_HW_RANDOM_VIRTIO=y
> diff --git a/test/mips/be@qemu-malta_defconfig.yaml b/test/mips/be@qemu-malta_defconfig.yaml
> new file mode 100644
> index 000000000000..5217a6ec2a13
> --- /dev/null
> +++ b/test/mips/be@qemu-malta_defconfig.yaml
> @@ -0,0 +1,25 @@
> +targets:
> +  main:
> +    drivers:
> +      QEMUDriver:
> +        qemu_bin: qemu
> +        machine: malta
> +        cpu: M14Kc
> +        memory: 256M
> +        bios: barebox-qemu-malta.img
> +        extra_args: ''
> +      BareboxDriver:
> +        prompt: 'barebox@[^:]+:[^ ]+ '
> +        bootstring: 'commandline:'
> +      ShellDriver:
> +        prompt: 'root@\w+:[^ ]+ '
> +        login_prompt: ' login: '
> +        login_timeout: 90
> +        username: 'root'
> +      BareboxStrategy: {}
> +    features:
> +      - virtio-pci
> +images:
> +  barebox-qemu-malta.img: $LG_BUILDDIR/images/barebox-qemu-malta.img
> +tools:
> +  qemu: /usr/bin/qemu-system-mips
> diff --git a/test/mips/le@qemu-malta_defconfig.yaml b/test/mips/le@qemu-malta_defconfig.yaml
> new file mode 100644
> index 000000000000..3b0f557dde5b
> --- /dev/null
> +++ b/test/mips/le@qemu-malta_defconfig.yaml
> @@ -0,0 +1,28 @@
> +targets:
> +  main:
> +    drivers:
> +      QEMUDriver:
> +        qemu_bin: qemu
> +        machine: malta
> +        cpu: M14Kc
> +        memory: 256M
> +        bios: barebox-qemu-malta.img.swapped
> +        extra_args: ''
> +      BareboxDriver:
> +        prompt: 'barebox@[^:]+:[^ ]+ '
> +        bootstring: 'commandline:'
> +      ShellDriver:
> +        prompt: 'root@\w+:[^ ]+ '
> +        login_prompt: ' login: '
> +        login_timeout: 90
> +        username: 'root'
> +      BareboxStrategy: {}
> +    features:
> +      - virtio-pci
> +    runner:
> +      kconfig_add:
> +        - CONFIG_CPU_LITTLE_ENDIAN=y
> +images:
> +  barebox-qemu-malta.img.swapped: $LG_BUILDDIR/images/barebox-qemu-malta.img.swapped
> +tools:
> +  qemu: /usr/bin/qemu-system-mipsel
> diff --git a/test/mips/qemu-malta_defconfig.yaml b/test/mips/qemu-malta_defconfig.yaml
> new file mode 120000
> index 000000000000..481b6e547782
> --- /dev/null
> +++ b/test/mips/qemu-malta_defconfig.yaml
> @@ -0,0 +1 @@
> +be@qemu-malta_defconfig.yaml
> \ No newline at end of file
> diff --git a/test/openrisc/generic_defconfig.yaml b/test/openrisc/generic_defconfig.yaml
> new file mode 100644
> index 000000000000..ece54da1c191
> --- /dev/null
> +++ b/test/openrisc/generic_defconfig.yaml
> @@ -0,0 +1,23 @@
> +targets:
> +  main:
> +    drivers:
> +      QEMUDriver:
> +        qemu_bin: qemu
> +        machine: or1k-sim
> +        cpu: or1200
> +        memory: 256M
> +        kernel: barebox
> +        extra_args: ''
> +      BareboxDriver:
> +        prompt: 'barebox@[^:]+:[^ ]+ '
> +        bootstring: 'commandline:'
> +      ShellDriver:
> +        prompt: 'root@\w+:[^ ]+ '
> +        login_prompt: ' login: '
> +        login_timeout: 90
> +        username: 'root'
> +      BareboxStrategy: {}
> +images:
> +  barebox: $LG_BUILDDIR/barebox
> +tools:
> +  qemu: /usr/bin/qemu-system-or1k
> diff --git a/test/riscv/qemu@virt32_defconfig.yaml b/test/riscv/qemu@virt32_defconfig.yaml
> new file mode 100644
> index 000000000000..d879ac6880a7
> --- /dev/null
> +++ b/test/riscv/qemu@virt32_defconfig.yaml
> @@ -0,0 +1,30 @@
> +targets:
> +  main:
> +    drivers:
> +      QEMUDriver:
> +        qemu_bin: qemu
> +        machine: virt
> +        cpu: rv32
> +        memory: 256M
> +        kernel: barebox-dt-2nd.img
> +        bios: opensbi-riscv32-generic-fw_dynamic.bin
> +        extra_args: ''
> +      BareboxDriver:
> +        prompt: 'barebox@[^:]+:[^ ]+ '
> +        bootstring: 'commandline:'
> +      ShellDriver:
> +        prompt: 'root@\w+:[^ ]+ '
> +        login_prompt: ' login: '
> +        login_timeout: 90
> +        username: 'root'
> +      BareboxStrategy: {}
> +    features:
> +      - virtio-mmio
> +    runner:
> +      download:
> +        opensbi-riscv32-generic-fw_dynamic.bin: https://github.com/qemu/qemu/blob/v5.2.0/pc-bios/opensbi-riscv32-generic-fw_dynamic.bin?raw=true
> +images:
> +  barebox-dt-2nd.img: $LG_BUILDDIR/images/barebox-dt-2nd.img
> +  opensbi-riscv32-generic-fw_dynamic.bin: $LG_BUILDDIR/opensbi-riscv32-generic-fw_dynamic.bin
> +tools:
> +  qemu: /usr/bin/qemu-system-riscv32
> diff --git a/test/riscv/qemu@virt64_defconfig.yaml b/test/riscv/qemu@virt64_defconfig.yaml
> new file mode 100644
> index 000000000000..5102f9067e1b
> --- /dev/null
> +++ b/test/riscv/qemu@virt64_defconfig.yaml
> @@ -0,0 +1,30 @@
> +targets:
> +  main:
> +    drivers:
> +      QEMUDriver:
> +        qemu_bin: qemu
> +        machine: virt
> +        cpu: rv64
> +        memory: 256M
> +        kernel: barebox-dt-2nd.img
> +        bios: opensbi-riscv64-generic-fw_dynamic.bin
> +        extra_args: ''
> +      BareboxDriver:
> +        prompt: 'barebox@[^:]+:[^ ]+ '
> +        bootstring: 'commandline:'
> +      ShellDriver:
> +        prompt: 'root@\w+:[^ ]+ '
> +        login_prompt: ' login: '
> +        login_timeout: 90
> +        username: 'root'
> +      BareboxStrategy: {}
> +    features:
> +      - virtio-mmio
> +    runner:
> +      download:
> +        opensbi-riscv64-generic-fw_dynamic.bin: https://github.com/qemu/qemu/blob/v5.2.0/pc-bios/opensbi-riscv64-generic-fw_dynamic.bin?raw=true
> +images:
> +  barebox-dt-2nd.img: $LG_BUILDDIR/images/barebox-dt-2nd.img
> +  opensbi-riscv64-generic-fw_dynamic.bin: $LG_BUILDDIR/opensbi-riscv64-generic-fw_dynamic.bin
> +tools:
> +  qemu: /usr/bin/qemu-system-riscv64
> diff --git a/test/riscv/sifive_defconfig.yaml b/test/riscv/sifive_defconfig.yaml
> new file mode 100644
> index 000000000000..534d7b8b044a
> --- /dev/null
> +++ b/test/riscv/sifive_defconfig.yaml
> @@ -0,0 +1,28 @@
> +targets:
> +  main:
> +    drivers:
> +      QEMUDriver:
> +        qemu_bin: qemu
> +        machine: sifive_u
> +        cpu: sifive-u54
> +        memory: 256M
> +        kernel: barebox-hifive-unleashed.img
> +        bios: opensbi-riscv64-generic-fw_dynamic.bin
> +        extra_args: ''
> +      BareboxDriver:
> +        prompt: 'barebox@[^:]+:[^ ]+ '
> +        bootstring: 'commandline:'
> +      ShellDriver:
> +        prompt: 'root@\w+:[^ ]+ '
> +        login_prompt: ' login: '
> +        login_timeout: 90
> +        username: 'root'
> +      BareboxStrategy: {}
> +    runner:
> +      download:
> +        opensbi-riscv64-generic-fw_dynamic.bin: https://github.com/qemu/qemu/blob/v5.2.0/pc-bios/opensbi-riscv64-generic-fw_dynamic.bin?raw=true
> +images:
> +  barebox-hifive-unleashed.img: $LG_BUILDDIR/images/barebox-hifive-unleashed.img
> +  opensbi-riscv64-generic-fw_dynamic.bin: $LG_BUILDDIR/opensbi-riscv64-generic-fw_dynamic.bin
> +tools:
> +  qemu: /usr/bin/qemu-system-riscv64
> diff --git a/test/riscv/tinyemu@virt32_defconfig.yaml b/test/riscv/tinyemu@virt32_defconfig.yaml
> new file mode 100644
> index 000000000000..807a8ce47145
> --- /dev/null
> +++ b/test/riscv/tinyemu@virt32_defconfig.yaml
> @@ -0,0 +1,25 @@
> +targets:
> +  main:
> +    drivers:
> +      TinyEMUDriver:
> +        temu_bin: temu
> +        config: ./Documentation/boards/riscv/barebox-virt32.cfg
> +        image: barebox-dt-2nd.img
> +      BareboxDriver:
> +        prompt: 'barebox@[^:]+:[^ ]+ '
> +        bootstring: 'commandline:'
> +      ShellDriver:
> +        prompt: 'root@\w+:[^ ]+ '
> +        login_prompt: ' login: '
> +        login_timeout: 90
> +        username: 'root'
> +      BareboxStrategy: {}
> +    features:
> +      - virtio-mmio
> +    runner:
> +      download:
> +        bbl32.bin: https://barebox.org/jsbarebox/bbl32.bin
> +images:
> +  barebox-dt-2nd.img: $LG_BUILDDIR/images/barebox-dt-2nd.img
> +tools:
> +  temu: /usr/local/bin/temu
> diff --git a/test/riscv/tinyemu@virt64_defconfig.yaml b/test/riscv/tinyemu@virt64_defconfig.yaml
> new file mode 100644
> index 000000000000..58277d1ce0dc
> --- /dev/null
> +++ b/test/riscv/tinyemu@virt64_defconfig.yaml
> @@ -0,0 +1,25 @@
> +targets:
> +  main:
> +    drivers:
> +      TinyEMUDriver:
> +        temu_bin: temu
> +        config: ./Documentation/boards/riscv/barebox-virt64.cfg
> +        image: barebox-dt-2nd.img
> +      BareboxDriver:
> +        prompt: 'barebox@[^:]+:[^ ]+ '
> +        bootstring: 'commandline:'
> +      ShellDriver:
> +        prompt: 'root@\w+:[^ ]+ '
> +        login_prompt: ' login: '
> +        login_timeout: 90
> +        username: 'root'
> +      BareboxStrategy: {}
> +    features:
> +      - virtio-mmio
> +    runner:
> +      download:
> +        bbl64.bin: https://barebox.org/jsbarebox/bbl64.bin
> +images:
> +  barebox-dt-2nd.img: $LG_BUILDDIR/images/barebox-dt-2nd.img
> +tools:
> +  temu: /usr/local/bin/temu
> diff --git a/test/riscv/virt32_defconfig.yaml b/test/riscv/virt32_defconfig.yaml
> new file mode 120000
> index 000000000000..e9d7237f5339
> --- /dev/null
> +++ b/test/riscv/virt32_defconfig.yaml
> @@ -0,0 +1 @@
> +qemu@virt32_defconfig.yaml
> \ No newline at end of file
> diff --git a/test/riscv/virt64_defconfig.yaml b/test/riscv/virt64_defconfig.yaml
> new file mode 120000
> index 000000000000..ab419d5e7fb4
> --- /dev/null
> +++ b/test/riscv/virt64_defconfig.yaml
> @@ -0,0 +1 @@
> +qemu@virt64_defconfig.yaml
> \ No newline at end of file
> diff --git a/test/sandbox/sandbox_defconfig.yaml b/test/sandbox/sandbox_defconfig.yaml
> new file mode 100644
> index 000000000000..68826e8b5830
> --- /dev/null
> +++ b/test/sandbox/sandbox_defconfig.yaml
> @@ -0,0 +1,12 @@
> +targets:
> +  main:
> +    drivers:
> +      NativeExecutableDriver:
> +        command: ./barebox
> +        image: barebox
> +    runner:
> +      tuxmake_arch: um
> +images:
> +  barebox: $LG_BUILDDIR/barebox
> +tools:
> +  qemu: /usr/local/bin/temu
> diff --git a/test/x86/efi_defconfig.yaml b/test/x86/efi_defconfig.yaml
> new file mode 120000
> index 000000000000..942dc259d56b
> --- /dev/null
> +++ b/test/x86/efi_defconfig.yaml
> @@ -0,0 +1 @@
> +pc@efi_defconfig.yaml
> \ No newline at end of file
> diff --git a/test/x86/pc@efi_defconfig.yaml b/test/x86/pc@efi_defconfig.yaml
> new file mode 100644
> index 000000000000..923b0680fff2
> --- /dev/null
> +++ b/test/x86/pc@efi_defconfig.yaml
> @@ -0,0 +1,34 @@
> +targets:
> +  main:
> +    drivers:
> +      QEMUDriver:
> +        qemu_bin: qemu
> +        machine: pc
> +        cpu: Nehalem
> +        memory: 1024M
> +        kernel: barebox.efi
> +        bios: OVMF.fd
> +        extra_args: ''
> +      BareboxDriver:
> +        prompt: 'barebox@[^:]+:[^ ]+ '
> +        bootstring: 'commandline:'
> +      ShellDriver:
> +        prompt: 'root@\w+:[^ ]+ '
> +        login_prompt: ' login: '
> +        login_timeout: 90
> +        username: 'root'
> +      BareboxStrategy: {}
> +    features:
> +      - pci
> +    runner:
> +      tuxmake_arch: x86_64
> +      kconfig_add:
> +        - CONFIG_DRIVER_SERIAL_NS16550=y
> +        - CONFIG_CONSOLE_ACTIVATE_FIRST=y # avoid duplicate output
> +      download:
> +        OVMF.fd: /usr/share/qemu/OVMF.fd
> +images:
> +  barebox.efi: $LG_BUILDDIR/barebox.efi
> +  OVMF.fd: $LG_BUILDDIR/OVMF.fd
> +tools:
> +  qemu: /usr/bin/qemu-system-x86_64
> diff --git a/test/x86/q35@efi_defconfig.yaml b/test/x86/q35@efi_defconfig.yaml
> new file mode 100644
> index 000000000000..1cfbd9fd88e3
> --- /dev/null
> +++ b/test/x86/q35@efi_defconfig.yaml
> @@ -0,0 +1,34 @@
> +targets:
> +  main:
> +    drivers:
> +      QEMUDriver:
> +        qemu_bin: qemu
> +        machine: q35
> +        cpu: Nehalem
> +        memory: 1024M
> +        kernel: barebox.efi
> +        bios: OVMF.fd
> +        extra_args: -global ICH9-LPC.noreboot=false
> +      BareboxDriver:
> +        prompt: 'barebox@[^:]+:[^ ]+ '
> +        bootstring: 'commandline:'
> +      ShellDriver:
> +        prompt: 'root@\w+:[^ ]+ '
> +        login_prompt: ' login: '
> +        login_timeout: 90
> +        username: 'root'
> +      BareboxStrategy: {}
> +    features:
> +      - pci
> +    runner:
> +        tuxmake_arch: x86_64
> +        kconfig_add:
> +            - CONFIG_DRIVER_SERIAL_NS16550=y
> +            - CONFIG_CONSOLE_ACTIVATE_FIRST=y # avoid duplicate output
> +        download:
> +            OVMF.fd: /usr/share/qemu/OVMF.fd
> +images:
> +  barebox.efi: $LG_BUILDDIR/barebox.efi
> +  OVMF.fd: $LG_BUILDDIR/OVMF.fd
> +tools:
> +  qemu: /usr/bin/qemu-system-x86_64
> diff --git a/test/x86/virtio@efi_defconfig.yaml b/test/x86/virtio@efi_defconfig.yaml
> new file mode 100644
> index 000000000000..3c9a9ae51e23
> --- /dev/null
> +++ b/test/x86/virtio@efi_defconfig.yaml
> @@ -0,0 +1,35 @@
> +targets:
> +  main:
> +    drivers:
> +      QEMUDriver:
> +        qemu_bin: qemu
> +        machine: pc
> +        cpu: Nehalem
> +        memory: 1024M
> +        kernel: barebox.efi
> +        bios: OVMF.fd
> +        extra_args: ''
> +      BareboxDriver:
> +        prompt: 'barebox@[^:]+:[^ ]+ '
> +        bootstring: 'commandline:'
> +      ShellDriver:
> +        prompt: 'root@\w+:[^ ]+ '
> +        login_prompt: ' login: '
> +        login_timeout: 90
> +        username: 'root'
> +      BareboxStrategy: {}
> +    features:
> +      - virtio-pci
> +    runner:
> +      tuxmake_arch: x86_64
> +      kconfig_add:
> +        - test/kconfig/virtio-pci.cfg
> +        - CONFIG_DRIVER_SERIAL_NS16550=y
> +        - CONFIG_CONSOLE_ACTIVATE_FIRST=y # avoid duplicate output
> +      download:
> +        OVMF.fd: /usr/share/qemu/OVMF.fd
> +images:
> +  barebox.efi: $LG_BUILDDIR/barebox.efi
> +  OVMF.fd: $LG_BUILDDIR/OVMF.fd
> +tools:
> +  qemu: /usr/bin/qemu-system-x86_64



_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [PATCH v2 10/13] test: add first sample tests
  2021-05-31  6:55 ` [PATCH v2 10/13] test: add first sample tests Ahmad Fatoum
@ 2021-06-02 11:33   ` Rouven Czerwinski
  2021-06-02 11:35     ` Ahmad Fatoum
  0 siblings, 1 reply; 21+ messages in thread
From: Rouven Czerwinski @ 2021-06-02 11:33 UTC (permalink / raw)
  To: Ahmad Fatoum, barebox

Hi Ahmad,

On Mon, 2021-05-31 at 08:55 +0200, Ahmad Fatoum wrote:
> The test can be run manually with e.g.
> 
>   labgrid-pytest --lg-env test/arm/qemu_virt64_defconfig.yaml test/py
> 
> Signed-off-by: Ahmad Fatoum <a.fatoum@pengutronix.de>
> ---
>  test/.gitignore       |  1 +
>  test/__init__.py      |  0
>  test/conftest.py      | 34 ++++++++++++++++++++++++++++++++++
>  test/py/__init__.py   |  0
>  test/py/helper.py     | 38 ++++++++++++++++++++++++++++++++++++++
>  test/py/test_shell.py | 37 +++++++++++++++++++++++++++++++++++++
>  6 files changed, 110 insertions(+)
>  create mode 100644 test/.gitignore
>  create mode 100644 test/__init__.py
>  create mode 100644 test/conftest.py
>  create mode 100644 test/py/__init__.py
>  create mode 100644 test/py/helper.py
>  create mode 100644 test/py/test_shell.py
> 
> diff --git a/test/.gitignore b/test/.gitignore
> new file mode 100644
> index 000000000000..bee8a64b79a9
> --- /dev/null
> +++ b/test/.gitignore
> @@ -0,0 +1 @@
> +__pycache__
> diff --git a/test/__init__.py b/test/__init__.py
> new file mode 100644
> index 000000000000..e69de29bb2d1
> diff --git a/test/conftest.py b/test/conftest.py
> new file mode 100644
> index 000000000000..5acc1a99e18e
> --- /dev/null
> +++ b/test/conftest.py
> @@ -0,0 +1,34 @@
> +import pytest
> +import os
> +from .py import helper
> +
> +
> +@pytest.fixture(scope='function')
> +def barebox(strategy, target):
> +    strategy.transition('barebox')
> +    return target.get_driver('BareboxDriver')
> +
> +
> +@pytest.fixture(scope='function')
> +def shell(strategy, target):
> +    strategy.transition('shell')
> +    return target.get_driver('ShellDriver')

This fixture is not used anywhere, remove it.

> +
> +
> +@pytest.fixture(scope="session")
> +def barebox_config(strategy, target):
> +    strategy.transition('barebox')
> +    command = target.get_driver("BareboxDriver")
> +    return helper.get_config(command)
> +
> +def pytest_configure(config):
> +    if 'LG_BUILDDIR' not in os.environ:
> +        if 'KBUILD_OUTPUT' in os.environ:
> +            os.environ['LG_BUILDDIR'] = os.environ['KBUILD_OUTPUT']
> +        elif os.path.isdir('build'):
> +            os.environ['LG_BUILDDIR'] = os.path.realpath('build')
> +        else:
> +            os.environ['LG_BUILDDIR'] = os.getcwd()
> +
> +    if os.environ['LG_BUILDDIR'] is not None:
> +        os.environ['LG_BUILDDIR'] = os.path.realpath(os.environ['LG_BUILDDIR'])
> diff --git a/test/py/__init__.py b/test/py/__init__.py
> new file mode 100644
> index 000000000000..e69de29bb2d1
> diff --git a/test/py/helper.py b/test/py/helper.py
> new file mode 100644
> index 000000000000..4a68e83669ba
> --- /dev/null
> +++ b/test/py/helper.py
> @@ -0,0 +1,38 @@
> +from labgrid.driver import BareboxDriver
> +import pytest
> +import os
> +from itertools import filterfalse
> +
> +
> +def get_config(command):
> +    """Returns the enabled config options of barebox, either from
> +    a running instance if supported or by looking into .config
> +    in the build directory.
> +    Args:
> +        command (BareboxDriver): An instance of the BareboxDriver
> +    Returns:
> +        list: list of the enabled config options
> +    """
> +    assert isinstance(command, BareboxDriver)
> +
> +    out, err, returncode = command.run("cat /env/data/config")
> +    if returncode != 0:
> +        try:
> +            with open(os.environ['LG_BUILDDIR'] + "/.config") as f:

Please don't use the LG_ namespace currently used by labgrid, this
variable is introduced by your wrapper script, something like
BB_LG_BUILDDIR indicates that this is only used in the barebox test
scripts.

> +                out = f.read().splitlines()
> +        except OSError:
> +            return set()
> +
> +    options = set()
> +    for line in out:
> +        if line and line.startswith("CONFIG_"):
> +            options.add(line.split('=')[0])
> +    return options
> +
> +
> +def skip_disabled(config, *options):
> +    if bool(config):
> +        undefined = list(filterfalse(config.__contains__, options))
> +
> +        if bool(undefined):
> +            pytest.skip("skipping test due to disabled " + (",".join(undefined)) + " dependency")
> diff --git a/test/py/test_shell.py b/test/py/test_shell.py
> new file mode 100644
> index 000000000000..0d2dfe38c5dd
> --- /dev/null
> +++ b/test/py/test_shell.py
> @@ -0,0 +1,37 @@
> +import pytest
> +from .helper import *
> +
> +
> +def test_barebox_true(barebox, barebox_config):
> +    skip_disabled(barebox_config, "CONFIG_CMD_TRUE")
> +
> +    _, _, returncode = barebox.run('true')
> +    assert returncode == 0
> +
> +
> +def test_barebox_false(barebox, barebox_config):
> +    skip_disabled(barebox_config, "CONFIG_CMD_FALSE")
> +
> +    _, _, returncode = barebox.run('false')
> +    assert returncode == 1
> +
> +def test_barebox_md5sum(barebox, barebox_config):
> +    skip_disabled(barebox_config, "CONFIG_CMD_MD5SUM", "CONFIG_CMD_ECHO")
> +
> +    barebox.run_check("echo -o md5 test")
> +    out = barebox.run_check("md5sum md5")
> +    assert out == ["d8e8fca2dc0f896fd7cb4cb0031ba249  md5"]
> +
> +def test_barebox_version(barebox, barebox_config):
> +    skip_disabled(barebox_config, "CONFIG_CMD_VERSION")
> +
> +    stdout, _, returncode = barebox.run('version')
> +    assert 'barebox' in stdout[1]
> +    assert returncode == 0
> +
> +def test_barebox_no_err(barebox, barebox_config):
> +    skip_disabled(barebox_config, "CONFIG_CMD_DMESG")
> +
> +    # TODO extend by err once all qemu platforms conform
> +    stdout, _, _ = barebox.run('dmesg -l crit,alert,emerg')
> +    assert stdout == []

- rcz



_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [PATCH v2 09/13] test: add labgrid configs for some emulated targets
  2021-06-02 11:27   ` Rouven Czerwinski
@ 2021-06-02 11:34     ` Ahmad Fatoum
  0 siblings, 0 replies; 21+ messages in thread
From: Ahmad Fatoum @ 2021-06-02 11:34 UTC (permalink / raw)
  To: Rouven Czerwinski, barebox

Hi,

On 02.06.21 13:27, Rouven Czerwinski wrote:
> On Mon, 2021-05-31 at 08:55 +0200, Ahmad Fatoum wrote:
>> There are various ways barebox can run under emulation, some of them are
>> described in the documentation and some more in the git history.
>>
>> Make running the emulators more convenient to use by collecting some
>> known-good emulator invocations in a machine readable format.
>>
>> These YAML files can be parsed by labgrid for running tests or executed
>> by a new test/emulate.pl runner script that will be added in a follow-up
>> commit.
>>
>> Using labgrid for this will allow using the same test suite for physical
>> targets as well in future.
>>

> All of these targets use a Shelldriver, but it is used only to satisfy
> the BareboxStrategy. Scrap the ShellDriver and write a short strategy
> or scrap the strategy as well or wholesale remove strategy usage
> alltogether. Since all thats done is activating the BareboxDriver, this
> should be doable without a strategy.

1) I get by with copy-pasting. Can you point me to an example of how that
   strategy or lack thereof should look like?

2) I want to actually test boots later on, mainly barebox image booting
   another, but I coud imagine loading a FIT and booting that as well.
   In that case I would need the ShellDriver, so why do extra work to
   remove it and add it again later?

Cheers,
Ahmad

> 
>> +      BareboxStrategy: {}
>> +images:
>> +  barebox-vexpress-ca15.img: $LG_BUILDDIR/images/barebox-vexpress-ca15.img
>> +tools:
>> +  qemu: /usr/bin/qemu-system-arm
>> diff --git a/test/arm/a9@vexpress_defconfig.yaml b/test/arm/a9@vexpress_defconfig.yaml
>> new file mode 100644
>> index 000000000000..88e09ca052e6
>> --- /dev/null
>> +++ b/test/arm/a9@vexpress_defconfig.yaml
>> @@ -0,0 +1,23 @@
>> +targets:
>> +  main:
>> +    drivers:
>> +      QEMUDriver:
>> +        qemu_bin: qemu
>> +        machine: vexpress-a9
>> +        cpu: cortex-a9
>> +        memory: 1024M
>> +        bios: barebox-vexpress-ca9.img
>> +        extra_args: ''
>> +      BareboxDriver:
>> +        prompt: 'barebox@[^:]+:[^ ]+ '
>> +        bootstring: 'commandline:'
>> +      ShellDriver:
>> +        prompt: 'root@\w+:[^ ]+ '
>> +        login_prompt: ' login: '
>> +        login_timeout: 90
>> +        username: 'root'
>> +      BareboxStrategy: {}
>> +images:
>> +  barebox-vexpress-ca9.img: $LG_BUILDDIR/images/barebox-vexpress-ca9.img
>> +tools:
>> +  qemu: /usr/bin/qemu-system-arm
>> diff --git a/test/arm/qemu_virt64_defconfig.yaml b/test/arm/qemu_virt64_defconfig.yaml
>> new file mode 100644
>> index 000000000000..2dfaaedef61a
>> --- /dev/null
>> +++ b/test/arm/qemu_virt64_defconfig.yaml
>> @@ -0,0 +1,27 @@
>> +targets:
>> +  main:
>> +    drivers:
>> +      QEMUDriver:
>> +        qemu_bin: qemu
>> +        machine: virt
>> +        cpu: cortex-a57
>> +        memory: 1024M
>> +        kernel: barebox-dt-2nd.img
>> +        extra_args: ''
>> +      BareboxDriver:
>> +        prompt: 'barebox@[^:]+:[^ ]+ '
>> +        bootstring: 'commandline:'
>> +      ShellDriver:
>> +        prompt: 'root@\w+:[^ ]+ '
>> +        login_prompt: ' login: '
>> +        login_timeout: 90
>> +        username: 'root'
>> +      BareboxStrategy: {}
>> +    features:
>> +      - virtio-mmio
>> +    runner:
>> +      tuxmake_arch: arm64
>> +images:
>> +  barebox-dt-2nd.img: $LG_BUILDDIR/images/barebox-dt-2nd.img
>> +tools:
>> +  qemu: /usr/bin/qemu-system-aarch64
>> diff --git a/test/arm/vexpress_defconfig.yaml b/test/arm/vexpress_defconfig.yaml
>> new file mode 120000
>> index 000000000000..732f51b19dc9
>> --- /dev/null
>> +++ b/test/arm/vexpress_defconfig.yaml
>> @@ -0,0 +1 @@
>> +a9@vexpress_defconfig.yaml
>> \ No newline at end of file
>> diff --git a/test/arm/virt@vexpress_defconfig.yaml b/test/arm/virt@vexpress_defconfig.yaml
>> new file mode 100644
>> index 000000000000..9c2e727c9cb5
>> --- /dev/null
>> +++ b/test/arm/virt@vexpress_defconfig.yaml
>> @@ -0,0 +1,25 @@
>> +targets:
>> +  main:
>> +    drivers:
>> +      QEMUDriver:
>> +        qemu_bin: qemu
>> +        machine: virt
>> +        cpu: cortex-a7
>> +        memory: 1024M
>> +        kernel: barebox-dt-2nd.img
>> +        extra_args: ''
>> +      BareboxDriver:
>> +        prompt: 'barebox@[^:]+:[^ ]+ '
>> +        bootstring: 'commandline:'
>> +      ShellDriver:
>> +        prompt: 'root@\w+:[^ ]+ '
>> +        login_prompt: ' login: '
>> +        login_timeout: 90
>> +        username: 'root'
>> +      BareboxStrategy: {}
>> +    features:
>> +      - virtio-mmio
>> +images:
>> +  barebox-dt-2nd.img: $LG_BUILDDIR/images/barebox-dt-2nd.img
>> +tools:
>> +  qemu: /usr/bin/qemu-system-arm
>> diff --git a/test/kconfig/virtio-pci.cfg b/test/kconfig/virtio-pci.cfg
>> new file mode 100644
>> index 000000000000..78237b8fca26
>> --- /dev/null
>> +++ b/test/kconfig/virtio-pci.cfg
>> @@ -0,0 +1,6 @@
>> +CONFIG_VIRTIO_MENU=y
>> +CONFIG_VIRTIO_PCI=y
>> +CONFIG_VIRTIO_CONSOLE=y
>> +CONFIG_VIRTIO_BLK=y
>> +CONFIG_HWRNG=y
>> +CONFIG_HW_RANDOM_VIRTIO=y
>> diff --git a/test/mips/be@qemu-malta_defconfig.yaml b/test/mips/be@qemu-malta_defconfig.yaml
>> new file mode 100644
>> index 000000000000..5217a6ec2a13
>> --- /dev/null
>> +++ b/test/mips/be@qemu-malta_defconfig.yaml
>> @@ -0,0 +1,25 @@
>> +targets:
>> +  main:
>> +    drivers:
>> +      QEMUDriver:
>> +        qemu_bin: qemu
>> +        machine: malta
>> +        cpu: M14Kc
>> +        memory: 256M
>> +        bios: barebox-qemu-malta.img
>> +        extra_args: ''
>> +      BareboxDriver:
>> +        prompt: 'barebox@[^:]+:[^ ]+ '
>> +        bootstring: 'commandline:'
>> +      ShellDriver:
>> +        prompt: 'root@\w+:[^ ]+ '
>> +        login_prompt: ' login: '
>> +        login_timeout: 90
>> +        username: 'root'
>> +      BareboxStrategy: {}
>> +    features:
>> +      - virtio-pci
>> +images:
>> +  barebox-qemu-malta.img: $LG_BUILDDIR/images/barebox-qemu-malta.img
>> +tools:
>> +  qemu: /usr/bin/qemu-system-mips
>> diff --git a/test/mips/le@qemu-malta_defconfig.yaml b/test/mips/le@qemu-malta_defconfig.yaml
>> new file mode 100644
>> index 000000000000..3b0f557dde5b
>> --- /dev/null
>> +++ b/test/mips/le@qemu-malta_defconfig.yaml
>> @@ -0,0 +1,28 @@
>> +targets:
>> +  main:
>> +    drivers:
>> +      QEMUDriver:
>> +        qemu_bin: qemu
>> +        machine: malta
>> +        cpu: M14Kc
>> +        memory: 256M
>> +        bios: barebox-qemu-malta.img.swapped
>> +        extra_args: ''
>> +      BareboxDriver:
>> +        prompt: 'barebox@[^:]+:[^ ]+ '
>> +        bootstring: 'commandline:'
>> +      ShellDriver:
>> +        prompt: 'root@\w+:[^ ]+ '
>> +        login_prompt: ' login: '
>> +        login_timeout: 90
>> +        username: 'root'
>> +      BareboxStrategy: {}
>> +    features:
>> +      - virtio-pci
>> +    runner:
>> +      kconfig_add:
>> +        - CONFIG_CPU_LITTLE_ENDIAN=y
>> +images:
>> +  barebox-qemu-malta.img.swapped: $LG_BUILDDIR/images/barebox-qemu-malta.img.swapped
>> +tools:
>> +  qemu: /usr/bin/qemu-system-mipsel
>> diff --git a/test/mips/qemu-malta_defconfig.yaml b/test/mips/qemu-malta_defconfig.yaml
>> new file mode 120000
>> index 000000000000..481b6e547782
>> --- /dev/null
>> +++ b/test/mips/qemu-malta_defconfig.yaml
>> @@ -0,0 +1 @@
>> +be@qemu-malta_defconfig.yaml
>> \ No newline at end of file
>> diff --git a/test/openrisc/generic_defconfig.yaml b/test/openrisc/generic_defconfig.yaml
>> new file mode 100644
>> index 000000000000..ece54da1c191
>> --- /dev/null
>> +++ b/test/openrisc/generic_defconfig.yaml
>> @@ -0,0 +1,23 @@
>> +targets:
>> +  main:
>> +    drivers:
>> +      QEMUDriver:
>> +        qemu_bin: qemu
>> +        machine: or1k-sim
>> +        cpu: or1200
>> +        memory: 256M
>> +        kernel: barebox
>> +        extra_args: ''
>> +      BareboxDriver:
>> +        prompt: 'barebox@[^:]+:[^ ]+ '
>> +        bootstring: 'commandline:'
>> +      ShellDriver:
>> +        prompt: 'root@\w+:[^ ]+ '
>> +        login_prompt: ' login: '
>> +        login_timeout: 90
>> +        username: 'root'
>> +      BareboxStrategy: {}
>> +images:
>> +  barebox: $LG_BUILDDIR/barebox
>> +tools:
>> +  qemu: /usr/bin/qemu-system-or1k
>> diff --git a/test/riscv/qemu@virt32_defconfig.yaml b/test/riscv/qemu@virt32_defconfig.yaml
>> new file mode 100644
>> index 000000000000..d879ac6880a7
>> --- /dev/null
>> +++ b/test/riscv/qemu@virt32_defconfig.yaml
>> @@ -0,0 +1,30 @@
>> +targets:
>> +  main:
>> +    drivers:
>> +      QEMUDriver:
>> +        qemu_bin: qemu
>> +        machine: virt
>> +        cpu: rv32
>> +        memory: 256M
>> +        kernel: barebox-dt-2nd.img
>> +        bios: opensbi-riscv32-generic-fw_dynamic.bin
>> +        extra_args: ''
>> +      BareboxDriver:
>> +        prompt: 'barebox@[^:]+:[^ ]+ '
>> +        bootstring: 'commandline:'
>> +      ShellDriver:
>> +        prompt: 'root@\w+:[^ ]+ '
>> +        login_prompt: ' login: '
>> +        login_timeout: 90
>> +        username: 'root'
>> +      BareboxStrategy: {}
>> +    features:
>> +      - virtio-mmio
>> +    runner:
>> +      download:
>> +        opensbi-riscv32-generic-fw_dynamic.bin: https://github.com/qemu/qemu/blob/v5.2.0/pc-bios/opensbi-riscv32-generic-fw_dynamic.bin?raw=true
>> +images:
>> +  barebox-dt-2nd.img: $LG_BUILDDIR/images/barebox-dt-2nd.img
>> +  opensbi-riscv32-generic-fw_dynamic.bin: $LG_BUILDDIR/opensbi-riscv32-generic-fw_dynamic.bin
>> +tools:
>> +  qemu: /usr/bin/qemu-system-riscv32
>> diff --git a/test/riscv/qemu@virt64_defconfig.yaml b/test/riscv/qemu@virt64_defconfig.yaml
>> new file mode 100644
>> index 000000000000..5102f9067e1b
>> --- /dev/null
>> +++ b/test/riscv/qemu@virt64_defconfig.yaml
>> @@ -0,0 +1,30 @@
>> +targets:
>> +  main:
>> +    drivers:
>> +      QEMUDriver:
>> +        qemu_bin: qemu
>> +        machine: virt
>> +        cpu: rv64
>> +        memory: 256M
>> +        kernel: barebox-dt-2nd.img
>> +        bios: opensbi-riscv64-generic-fw_dynamic.bin
>> +        extra_args: ''
>> +      BareboxDriver:
>> +        prompt: 'barebox@[^:]+:[^ ]+ '
>> +        bootstring: 'commandline:'
>> +      ShellDriver:
>> +        prompt: 'root@\w+:[^ ]+ '
>> +        login_prompt: ' login: '
>> +        login_timeout: 90
>> +        username: 'root'
>> +      BareboxStrategy: {}
>> +    features:
>> +      - virtio-mmio
>> +    runner:
>> +      download:
>> +        opensbi-riscv64-generic-fw_dynamic.bin: https://github.com/qemu/qemu/blob/v5.2.0/pc-bios/opensbi-riscv64-generic-fw_dynamic.bin?raw=true
>> +images:
>> +  barebox-dt-2nd.img: $LG_BUILDDIR/images/barebox-dt-2nd.img
>> +  opensbi-riscv64-generic-fw_dynamic.bin: $LG_BUILDDIR/opensbi-riscv64-generic-fw_dynamic.bin
>> +tools:
>> +  qemu: /usr/bin/qemu-system-riscv64
>> diff --git a/test/riscv/sifive_defconfig.yaml b/test/riscv/sifive_defconfig.yaml
>> new file mode 100644
>> index 000000000000..534d7b8b044a
>> --- /dev/null
>> +++ b/test/riscv/sifive_defconfig.yaml
>> @@ -0,0 +1,28 @@
>> +targets:
>> +  main:
>> +    drivers:
>> +      QEMUDriver:
>> +        qemu_bin: qemu
>> +        machine: sifive_u
>> +        cpu: sifive-u54
>> +        memory: 256M
>> +        kernel: barebox-hifive-unleashed.img
>> +        bios: opensbi-riscv64-generic-fw_dynamic.bin
>> +        extra_args: ''
>> +      BareboxDriver:
>> +        prompt: 'barebox@[^:]+:[^ ]+ '
>> +        bootstring: 'commandline:'
>> +      ShellDriver:
>> +        prompt: 'root@\w+:[^ ]+ '
>> +        login_prompt: ' login: '
>> +        login_timeout: 90
>> +        username: 'root'
>> +      BareboxStrategy: {}
>> +    runner:
>> +      download:
>> +        opensbi-riscv64-generic-fw_dynamic.bin: https://github.com/qemu/qemu/blob/v5.2.0/pc-bios/opensbi-riscv64-generic-fw_dynamic.bin?raw=true
>> +images:
>> +  barebox-hifive-unleashed.img: $LG_BUILDDIR/images/barebox-hifive-unleashed.img
>> +  opensbi-riscv64-generic-fw_dynamic.bin: $LG_BUILDDIR/opensbi-riscv64-generic-fw_dynamic.bin
>> +tools:
>> +  qemu: /usr/bin/qemu-system-riscv64
>> diff --git a/test/riscv/tinyemu@virt32_defconfig.yaml b/test/riscv/tinyemu@virt32_defconfig.yaml
>> new file mode 100644
>> index 000000000000..807a8ce47145
>> --- /dev/null
>> +++ b/test/riscv/tinyemu@virt32_defconfig.yaml
>> @@ -0,0 +1,25 @@
>> +targets:
>> +  main:
>> +    drivers:
>> +      TinyEMUDriver:
>> +        temu_bin: temu
>> +        config: ./Documentation/boards/riscv/barebox-virt32.cfg
>> +        image: barebox-dt-2nd.img
>> +      BareboxDriver:
>> +        prompt: 'barebox@[^:]+:[^ ]+ '
>> +        bootstring: 'commandline:'
>> +      ShellDriver:
>> +        prompt: 'root@\w+:[^ ]+ '
>> +        login_prompt: ' login: '
>> +        login_timeout: 90
>> +        username: 'root'
>> +      BareboxStrategy: {}
>> +    features:
>> +      - virtio-mmio
>> +    runner:
>> +      download:
>> +        bbl32.bin: https://barebox.org/jsbarebox/bbl32.bin
>> +images:
>> +  barebox-dt-2nd.img: $LG_BUILDDIR/images/barebox-dt-2nd.img
>> +tools:
>> +  temu: /usr/local/bin/temu
>> diff --git a/test/riscv/tinyemu@virt64_defconfig.yaml b/test/riscv/tinyemu@virt64_defconfig.yaml
>> new file mode 100644
>> index 000000000000..58277d1ce0dc
>> --- /dev/null
>> +++ b/test/riscv/tinyemu@virt64_defconfig.yaml
>> @@ -0,0 +1,25 @@
>> +targets:
>> +  main:
>> +    drivers:
>> +      TinyEMUDriver:
>> +        temu_bin: temu
>> +        config: ./Documentation/boards/riscv/barebox-virt64.cfg
>> +        image: barebox-dt-2nd.img
>> +      BareboxDriver:
>> +        prompt: 'barebox@[^:]+:[^ ]+ '
>> +        bootstring: 'commandline:'
>> +      ShellDriver:
>> +        prompt: 'root@\w+:[^ ]+ '
>> +        login_prompt: ' login: '
>> +        login_timeout: 90
>> +        username: 'root'
>> +      BareboxStrategy: {}
>> +    features:
>> +      - virtio-mmio
>> +    runner:
>> +      download:
>> +        bbl64.bin: https://barebox.org/jsbarebox/bbl64.bin
>> +images:
>> +  barebox-dt-2nd.img: $LG_BUILDDIR/images/barebox-dt-2nd.img
>> +tools:
>> +  temu: /usr/local/bin/temu
>> diff --git a/test/riscv/virt32_defconfig.yaml b/test/riscv/virt32_defconfig.yaml
>> new file mode 120000
>> index 000000000000..e9d7237f5339
>> --- /dev/null
>> +++ b/test/riscv/virt32_defconfig.yaml
>> @@ -0,0 +1 @@
>> +qemu@virt32_defconfig.yaml
>> \ No newline at end of file
>> diff --git a/test/riscv/virt64_defconfig.yaml b/test/riscv/virt64_defconfig.yaml
>> new file mode 120000
>> index 000000000000..ab419d5e7fb4
>> --- /dev/null
>> +++ b/test/riscv/virt64_defconfig.yaml
>> @@ -0,0 +1 @@
>> +qemu@virt64_defconfig.yaml
>> \ No newline at end of file
>> diff --git a/test/sandbox/sandbox_defconfig.yaml b/test/sandbox/sandbox_defconfig.yaml
>> new file mode 100644
>> index 000000000000..68826e8b5830
>> --- /dev/null
>> +++ b/test/sandbox/sandbox_defconfig.yaml
>> @@ -0,0 +1,12 @@
>> +targets:
>> +  main:
>> +    drivers:
>> +      NativeExecutableDriver:
>> +        command: ./barebox
>> +        image: barebox
>> +    runner:
>> +      tuxmake_arch: um
>> +images:
>> +  barebox: $LG_BUILDDIR/barebox
>> +tools:
>> +  qemu: /usr/local/bin/temu
>> diff --git a/test/x86/efi_defconfig.yaml b/test/x86/efi_defconfig.yaml
>> new file mode 120000
>> index 000000000000..942dc259d56b
>> --- /dev/null
>> +++ b/test/x86/efi_defconfig.yaml
>> @@ -0,0 +1 @@
>> +pc@efi_defconfig.yaml
>> \ No newline at end of file
>> diff --git a/test/x86/pc@efi_defconfig.yaml b/test/x86/pc@efi_defconfig.yaml
>> new file mode 100644
>> index 000000000000..923b0680fff2
>> --- /dev/null
>> +++ b/test/x86/pc@efi_defconfig.yaml
>> @@ -0,0 +1,34 @@
>> +targets:
>> +  main:
>> +    drivers:
>> +      QEMUDriver:
>> +        qemu_bin: qemu
>> +        machine: pc
>> +        cpu: Nehalem
>> +        memory: 1024M
>> +        kernel: barebox.efi
>> +        bios: OVMF.fd
>> +        extra_args: ''
>> +      BareboxDriver:
>> +        prompt: 'barebox@[^:]+:[^ ]+ '
>> +        bootstring: 'commandline:'
>> +      ShellDriver:
>> +        prompt: 'root@\w+:[^ ]+ '
>> +        login_prompt: ' login: '
>> +        login_timeout: 90
>> +        username: 'root'
>> +      BareboxStrategy: {}
>> +    features:
>> +      - pci
>> +    runner:
>> +      tuxmake_arch: x86_64
>> +      kconfig_add:
>> +        - CONFIG_DRIVER_SERIAL_NS16550=y
>> +        - CONFIG_CONSOLE_ACTIVATE_FIRST=y # avoid duplicate output
>> +      download:
>> +        OVMF.fd: /usr/share/qemu/OVMF.fd
>> +images:
>> +  barebox.efi: $LG_BUILDDIR/barebox.efi
>> +  OVMF.fd: $LG_BUILDDIR/OVMF.fd
>> +tools:
>> +  qemu: /usr/bin/qemu-system-x86_64
>> diff --git a/test/x86/q35@efi_defconfig.yaml b/test/x86/q35@efi_defconfig.yaml
>> new file mode 100644
>> index 000000000000..1cfbd9fd88e3
>> --- /dev/null
>> +++ b/test/x86/q35@efi_defconfig.yaml
>> @@ -0,0 +1,34 @@
>> +targets:
>> +  main:
>> +    drivers:
>> +      QEMUDriver:
>> +        qemu_bin: qemu
>> +        machine: q35
>> +        cpu: Nehalem
>> +        memory: 1024M
>> +        kernel: barebox.efi
>> +        bios: OVMF.fd
>> +        extra_args: -global ICH9-LPC.noreboot=false
>> +      BareboxDriver:
>> +        prompt: 'barebox@[^:]+:[^ ]+ '
>> +        bootstring: 'commandline:'
>> +      ShellDriver:
>> +        prompt: 'root@\w+:[^ ]+ '
>> +        login_prompt: ' login: '
>> +        login_timeout: 90
>> +        username: 'root'
>> +      BareboxStrategy: {}
>> +    features:
>> +      - pci
>> +    runner:
>> +        tuxmake_arch: x86_64
>> +        kconfig_add:
>> +            - CONFIG_DRIVER_SERIAL_NS16550=y
>> +            - CONFIG_CONSOLE_ACTIVATE_FIRST=y # avoid duplicate output
>> +        download:
>> +            OVMF.fd: /usr/share/qemu/OVMF.fd
>> +images:
>> +  barebox.efi: $LG_BUILDDIR/barebox.efi
>> +  OVMF.fd: $LG_BUILDDIR/OVMF.fd
>> +tools:
>> +  qemu: /usr/bin/qemu-system-x86_64
>> diff --git a/test/x86/virtio@efi_defconfig.yaml b/test/x86/virtio@efi_defconfig.yaml
>> new file mode 100644
>> index 000000000000..3c9a9ae51e23
>> --- /dev/null
>> +++ b/test/x86/virtio@efi_defconfig.yaml
>> @@ -0,0 +1,35 @@
>> +targets:
>> +  main:
>> +    drivers:
>> +      QEMUDriver:
>> +        qemu_bin: qemu
>> +        machine: pc
>> +        cpu: Nehalem
>> +        memory: 1024M
>> +        kernel: barebox.efi
>> +        bios: OVMF.fd
>> +        extra_args: ''
>> +      BareboxDriver:
>> +        prompt: 'barebox@[^:]+:[^ ]+ '
>> +        bootstring: 'commandline:'
>> +      ShellDriver:
>> +        prompt: 'root@\w+:[^ ]+ '
>> +        login_prompt: ' login: '
>> +        login_timeout: 90
>> +        username: 'root'
>> +      BareboxStrategy: {}
>> +    features:
>> +      - virtio-pci
>> +    runner:
>> +      tuxmake_arch: x86_64
>> +      kconfig_add:
>> +        - test/kconfig/virtio-pci.cfg
>> +        - CONFIG_DRIVER_SERIAL_NS16550=y
>> +        - CONFIG_CONSOLE_ACTIVATE_FIRST=y # avoid duplicate output
>> +      download:
>> +        OVMF.fd: /usr/share/qemu/OVMF.fd
>> +images:
>> +  barebox.efi: $LG_BUILDDIR/barebox.efi
>> +  OVMF.fd: $LG_BUILDDIR/OVMF.fd
>> +tools:
>> +  qemu: /usr/bin/qemu-system-x86_64
> 
> 
> 

-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [PATCH v2 10/13] test: add first sample tests
  2021-06-02 11:33   ` Rouven Czerwinski
@ 2021-06-02 11:35     ` Ahmad Fatoum
  2021-06-03 15:14       ` Jan Lübbe
  0 siblings, 1 reply; 21+ messages in thread
From: Ahmad Fatoum @ 2021-06-02 11:35 UTC (permalink / raw)
  To: Rouven Czerwinski, barebox

Hi,

On 02.06.21 13:33, Rouven Czerwinski wrote:
> Hi Ahmad,
> 
> On Mon, 2021-05-31 at 08:55 +0200, Ahmad Fatoum wrote:
>> The test can be run manually with e.g.
>>
>>   labgrid-pytest --lg-env test/arm/qemu_virt64_defconfig.yaml test/py
>>
>> Signed-off-by: Ahmad Fatoum <a.fatoum@pengutronix.de>
>> ---
>>  test/.gitignore       |  1 +
>>  test/__init__.py      |  0
>>  test/conftest.py      | 34 ++++++++++++++++++++++++++++++++++
>>  test/py/__init__.py   |  0
>>  test/py/helper.py     | 38 ++++++++++++++++++++++++++++++++++++++
>>  test/py/test_shell.py | 37 +++++++++++++++++++++++++++++++++++++
>>  6 files changed, 110 insertions(+)
>>  create mode 100644 test/.gitignore
>>  create mode 100644 test/__init__.py
>>  create mode 100644 test/conftest.py
>>  create mode 100644 test/py/__init__.py
>>  create mode 100644 test/py/helper.py
>>  create mode 100644 test/py/test_shell.py
>>
>> diff --git a/test/.gitignore b/test/.gitignore
>> new file mode 100644
>> index 000000000000..bee8a64b79a9
>> --- /dev/null
>> +++ b/test/.gitignore
>> @@ -0,0 +1 @@
>> +__pycache__
>> diff --git a/test/__init__.py b/test/__init__.py
>> new file mode 100644
>> index 000000000000..e69de29bb2d1
>> diff --git a/test/conftest.py b/test/conftest.py
>> new file mode 100644
>> index 000000000000..5acc1a99e18e
>> --- /dev/null
>> +++ b/test/conftest.py
>> @@ -0,0 +1,34 @@
>> +import pytest
>> +import os
>> +from .py import helper
>> +
>> +
>> +@pytest.fixture(scope='function')
>> +def barebox(strategy, target):
>> +    strategy.transition('barebox')
>> +    return target.get_driver('BareboxDriver')
>> +
>> +
>> +@pytest.fixture(scope='function')
>> +def shell(strategy, target):
>> +    strategy.transition('shell')
>> +    return target.get_driver('ShellDriver')
> 
> This fixture is not used anywhere, remove it.

See previous mail.

>> +
>> +
>> +@pytest.fixture(scope="session")
>> +def barebox_config(strategy, target):
>> +    strategy.transition('barebox')
>> +    command = target.get_driver("BareboxDriver")
>> +    return helper.get_config(command)
>> +
>> +def pytest_configure(config):
>> +    if 'LG_BUILDDIR' not in os.environ:
>> +        if 'KBUILD_OUTPUT' in os.environ:
>> +            os.environ['LG_BUILDDIR'] = os.environ['KBUILD_OUTPUT']
>> +        elif os.path.isdir('build'):
>> +            os.environ['LG_BUILDDIR'] = os.path.realpath('build')
>> +        else:
>> +            os.environ['LG_BUILDDIR'] = os.getcwd()
>> +
>> +    if os.environ['LG_BUILDDIR'] is not None:
>> +        os.environ['LG_BUILDDIR'] = os.path.realpath(os.environ['LG_BUILDDIR'])
>> diff --git a/test/py/__init__.py b/test/py/__init__.py
>> new file mode 100644
>> index 000000000000..e69de29bb2d1
>> diff --git a/test/py/helper.py b/test/py/helper.py
>> new file mode 100644
>> index 000000000000..4a68e83669ba
>> --- /dev/null
>> +++ b/test/py/helper.py
>> @@ -0,0 +1,38 @@
>> +from labgrid.driver import BareboxDriver
>> +import pytest
>> +import os
>> +from itertools import filterfalse
>> +
>> +
>> +def get_config(command):
>> +    """Returns the enabled config options of barebox, either from
>> +    a running instance if supported or by looking into .config
>> +    in the build directory.
>> +    Args:
>> +        command (BareboxDriver): An instance of the BareboxDriver
>> +    Returns:
>> +        list: list of the enabled config options
>> +    """
>> +    assert isinstance(command, BareboxDriver)
>> +
>> +    out, err, returncode = command.run("cat /env/data/config")
>> +    if returncode != 0:
>> +        try:
>> +            with open(os.environ['LG_BUILDDIR'] + "/.config") as f:
> 
> Please don't use the LG_ namespace currently used by labgrid, this
> variable is introduced by your wrapper script, something like
> BB_LG_BUILDDIR indicates that this is only used in the barebox test
> scripts.

Labgrid filters out anything that doesn't start with LG_ AFAIK.

> 
>> +                out = f.read().splitlines()
>> +        except OSError:
>> +            return set()
>> +
>> +    options = set()
>> +    for line in out:
>> +        if line and line.startswith("CONFIG_"):
>> +            options.add(line.split('=')[0])
>> +    return options
>> +
>> +
>> +def skip_disabled(config, *options):
>> +    if bool(config):
>> +        undefined = list(filterfalse(config.__contains__, options))
>> +
>> +        if bool(undefined):
>> +            pytest.skip("skipping test due to disabled " + (",".join(undefined)) + " dependency")
>> diff --git a/test/py/test_shell.py b/test/py/test_shell.py
>> new file mode 100644
>> index 000000000000..0d2dfe38c5dd
>> --- /dev/null
>> +++ b/test/py/test_shell.py
>> @@ -0,0 +1,37 @@
>> +import pytest
>> +from .helper import *
>> +
>> +
>> +def test_barebox_true(barebox, barebox_config):
>> +    skip_disabled(barebox_config, "CONFIG_CMD_TRUE")
>> +
>> +    _, _, returncode = barebox.run('true')
>> +    assert returncode == 0
>> +
>> +
>> +def test_barebox_false(barebox, barebox_config):
>> +    skip_disabled(barebox_config, "CONFIG_CMD_FALSE")
>> +
>> +    _, _, returncode = barebox.run('false')
>> +    assert returncode == 1
>> +
>> +def test_barebox_md5sum(barebox, barebox_config):
>> +    skip_disabled(barebox_config, "CONFIG_CMD_MD5SUM", "CONFIG_CMD_ECHO")
>> +
>> +    barebox.run_check("echo -o md5 test")
>> +    out = barebox.run_check("md5sum md5")
>> +    assert out == ["d8e8fca2dc0f896fd7cb4cb0031ba249  md5"]
>> +
>> +def test_barebox_version(barebox, barebox_config):
>> +    skip_disabled(barebox_config, "CONFIG_CMD_VERSION")
>> +
>> +    stdout, _, returncode = barebox.run('version')
>> +    assert 'barebox' in stdout[1]
>> +    assert returncode == 0
>> +
>> +def test_barebox_no_err(barebox, barebox_config):
>> +    skip_disabled(barebox_config, "CONFIG_CMD_DMESG")
>> +
>> +    # TODO extend by err once all qemu platforms conform
>> +    stdout, _, _ = barebox.run('dmesg -l crit,alert,emerg')
>> +    assert stdout == []
> 
> - rcz
> 
> 
> 

-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [PATCH v2 12/13] test: self: run selftests as part of the pytest suite
  2021-05-31  6:55 ` [PATCH v2 12/13] test: self: run selftests as part of the pytest suite Ahmad Fatoum
@ 2021-06-02 11:36   ` Rouven Czerwinski
  0 siblings, 0 replies; 21+ messages in thread
From: Rouven Czerwinski @ 2021-06-02 11:36 UTC (permalink / raw)
  To: Ahmad Fatoum, barebox

Hi Ahamd,

On Mon, 2021-05-31 at 08:55 +0200, Ahmad Fatoum wrote:
> We don't want to enable self tests in the normal configs as they may,
> in future, bloat size needlessly. Enable it instead in the base.cfg
> fragment and add a test that verifies the selftest command
> runs without errors.
> 
> Selftests can be run on startup for CONFIG_SHELL_NONE systems. This is
> not implemented here. For such systems the test will be skipped
> as CONFIG_CMD_SELFTEST won't be defined. To manually skip with
> emulate.pl, add --no-kconfig-base.
> 
> Signed-off-by: Ahmad Fatoum <a.fatoum@pengutronix.de>
> ---
>  test/kconfig/base.cfg      | 4 ++++
>  test/py/test_bselftests.py | 8 ++++++++
>  2 files changed, 12 insertions(+)
>  create mode 100644 test/py/test_bselftests.py
> 
> diff --git a/test/kconfig/base.cfg b/test/kconfig/base.cfg
> index e69de29bb2d1..6a9f68349816 100644
> --- a/test/kconfig/base.cfg
> +++ b/test/kconfig/base.cfg
> @@ -0,0 +1,4 @@
> +CONFIG_TEST=y
> +CONFIG_SELFTEST=y
> +CONFIG_CMD_SELFTEST=y
> +CONFIG_SELFTEST_ENABLE_ALL=y
> diff --git a/test/py/test_bselftests.py b/test/py/test_bselftests.py
> new file mode 100644
> index 000000000000..48e9d38c12fc
> --- /dev/null
> +++ b/test/py/test_bselftests.py
> @@ -0,0 +1,8 @@
> +import pytest
> +from .helper import *
> +
> +def test_bselftest(barebox, barebox_config):
> +    skip_disabled(barebox_config, "CONFIG_CMD_SELFTEST")
> +
> +    _, _, returncode = barebox.run('selftest')
> +    assert returncode == 0

This will give you the default timeout of 30 seconds for the command
run. As long as you are confident the selftests are going to stay under
this limit this is fine, however you'll probably want to make this
explicit :-)

Also you probably want something akin to this:

    stdout, stderr, returncode = barebox.run('selftest')
    assert returncode == 0, "selftest failed:\n {}\n".format("".join(stdout))

Which will output the selftest stdout if the selftest fails.

- rcz



_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [PATCH v2 10/13] test: add first sample tests
  2021-06-02 11:35     ` Ahmad Fatoum
@ 2021-06-03 15:14       ` Jan Lübbe
  2021-06-03 15:31         ` Ahmad Fatoum
  0 siblings, 1 reply; 21+ messages in thread
From: Jan Lübbe @ 2021-06-03 15:14 UTC (permalink / raw)
  To: Ahmad Fatoum, Rouven Czerwinski, barebox

On Wed, 2021-06-02 at 13:35 +0200, Ahmad Fatoum wrote:
> > > +def get_config(command):
> > > +    """Returns the enabled config options of barebox, either from
> > > +    a running instance if supported or by looking into .config
> > > +    in the build directory.
> > > +    Args:
> > > +        command (BareboxDriver): An instance of the BareboxDriver
> > > +    Returns:
> > > +        list: list of the enabled config options
> > > +    """
> > > +    assert isinstance(command, BareboxDriver)
> > > +
> > > +    out, err, returncode = command.run("cat /env/data/config")
> > > +    if returncode != 0:
> > > +        try:
> > > +            with open(os.environ['LG_BUILDDIR'] + "/.config") as f:
> > 
> > Please don't use the LG_ namespace currently used by labgrid, this
> > variable is introduced by your wrapper script, something like
> > BB_LG_BUILDDIR indicates that this is only used in the barebox test
> > scripts.
> 
> Labgrid filters out anything that doesn't start with LG_ AFAIK.

No. The filter is only for the environment templating. We don't touch the
os.environ dictionary.

Regards,
Jan
-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |


_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [PATCH v2 10/13] test: add first sample tests
  2021-06-03 15:14       ` Jan Lübbe
@ 2021-06-03 15:31         ` Ahmad Fatoum
  0 siblings, 0 replies; 21+ messages in thread
From: Ahmad Fatoum @ 2021-06-03 15:31 UTC (permalink / raw)
  To: Jan Lübbe, Rouven Czerwinski, barebox

Hello Jan,

On 03.06.21 17:14, Jan Lübbe wrote:
> On Wed, 2021-06-02 at 13:35 +0200, Ahmad Fatoum wrote:
>>>> +def get_config(command):
>>>> +    """Returns the enabled config options of barebox, either from
>>>> +    a running instance if supported or by looking into .config
>>>> +    in the build directory.
>>>> +    Args:
>>>> +        command (BareboxDriver): An instance of the BareboxDriver
>>>> +    Returns:
>>>> +        list: list of the enabled config options
>>>> +    """
>>>> +    assert isinstance(command, BareboxDriver)
>>>> +
>>>> +    out, err, returncode = command.run("cat /env/data/config")
>>>> +    if returncode != 0:
>>>> +        try:
>>>> +            with open(os.environ['LG_BUILDDIR'] + "/.config") as f:
>>>
>>> Please don't use the LG_ namespace currently used by labgrid, this
>>> variable is introduced by your wrapper script, something like
>>> BB_LG_BUILDDIR indicates that this is only used in the barebox test
>>> scripts.
>>
>> Labgrid filters out anything that doesn't start with LG_ AFAIK.
> 
> No. The filter is only for the environment templating. We don't touch the
> os.environ dictionary.

I am using LG_BUILDDIR out of the YAML config as well.

> 
> Regards,
> Jan
> 

-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox

^ permalink raw reply	[flat|nested] 21+ messages in thread

end of thread, other threads:[~2021-06-03 15:33 UTC | newest]

Thread overview: 21+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-05-31  6:55 [PATCH v2 00/13] add barebox in-tree testing infrastructure Ahmad Fatoum
2021-05-31  6:55 ` [PATCH v2 01/13] kbuild: add install target Ahmad Fatoum
2021-05-31  6:55 ` [PATCH v2 02/13] kbuild: add ARCH={i386, x86_64} as aliases for x86 Ahmad Fatoum
2021-05-31  6:55 ` [PATCH v2 03/13] kbuild: add ARCH=um alias for sandbox Ahmad Fatoum
2021-05-31  6:55 ` [PATCH v2 04/13] MIPS: qemu-malta: generate swapped image as part of multi-image build Ahmad Fatoum
2021-05-31  6:55 ` [PATCH v2 05/13] openrisc: set default KBUILD_IMAGE Ahmad Fatoum
2021-05-31  6:55 ` [PATCH v2 06/13] Documentation: boards: RISC-V: update TinyEMU support Ahmad Fatoum
2021-05-31  6:55 ` [PATCH v2 07/13] test: add basic barebox self-test infrastructure Ahmad Fatoum
2021-05-31  6:55 ` [PATCH v2 08/13] test: self: port Linux printf kselftest Ahmad Fatoum
2021-05-31  6:55 ` [PATCH v2 09/13] test: add labgrid configs for some emulated targets Ahmad Fatoum
2021-06-02 11:27   ` Rouven Czerwinski
2021-06-02 11:34     ` Ahmad Fatoum
2021-05-31  6:55 ` [PATCH v2 10/13] test: add first sample tests Ahmad Fatoum
2021-06-02 11:33   ` Rouven Czerwinski
2021-06-02 11:35     ` Ahmad Fatoum
2021-06-03 15:14       ` Jan Lübbe
2021-06-03 15:31         ` Ahmad Fatoum
2021-05-31  6:55 ` [PATCH v2 11/13] test: add emulate.pl, a runner for barebox on emulated targets Ahmad Fatoum
2021-05-31  6:55 ` [PATCH v2 12/13] test: self: run selftests as part of the pytest suite Ahmad Fatoum
2021-06-02 11:36   ` Rouven Czerwinski
2021-05-31  6:55 ` [PATCH v2 13/13] test: add bthread test Ahmad Fatoum

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox