mail archive of the barebox mailing list
 help / color / mirror / Atom feed
From: Jules Maselbas <jmaselbas@kalray.eu>
To: Ahmad Fatoum <a.fatoum@pengutronix.de>
Cc: barebox@lists.infradead.org, rcz@pengutronix.de
Subject: Re: [PATCH 09/17] test: add basic barebox self-test infrastructure
Date: Mon, 12 Apr 2021 12:12:45 +0200	[thread overview]
Message-ID: <20210412101245.GB21066@tellis.lin.mbt.kalray.eu> (raw)
In-Reply-To: <20210412071651.8769-10-a.fatoum@pengutronix.de>

Hi Ahmad,

This looks nice, I can't wait to try it out :]

On Mon, Apr 12, 2021 at 09:16:43AM +0200, Ahmad Fatoum wrote:
> Self tests is code written to run within barebox to exercise
> functionality. They offer flexibility to test specific units of barebox
> instead of the program as a whole. Add a very simple infrastructure
> for registering and executing self-tests. THis is based on the Linux
> kselftest modules. We don't utilize modules for this, however, because
> we only have module support on ARM, but we need a generic solution.
> 
> Selftests can be enabled individually and even tested without shell
> support to allow tests to happen for size-restricted barebox images
> as well.
> 
> Signed-off-by: Ahmad Fatoum <a.fatoum@pengutronix.de>
> ---
>  Kconfig             |  1 +
>  Makefile            |  2 +-
>  commands/Makefile   |  1 +
>  commands/selftest.c | 88 +++++++++++++++++++++++++++++++++++++++++++++
>  include/bselftest.h | 70 ++++++++++++++++++++++++++++++++++++
>  test/Kconfig        |  8 +++++
>  test/Makefile       |  1 +
>  test/self/Kconfig   | 33 +++++++++++++++++
>  test/self/Makefile  |  3 ++
>  test/self/core.c    | 39 ++++++++++++++++++++
>  10 files changed, 245 insertions(+), 1 deletion(-)
>  create mode 100644 commands/selftest.c
>  create mode 100644 include/bselftest.h
>  create mode 100644 test/Kconfig
>  create mode 100644 test/Makefile
>  create mode 100644 test/self/Kconfig
>  create mode 100644 test/self/Makefile
>  create mode 100644 test/self/core.c
> 
> diff --git a/Kconfig b/Kconfig
> index 29c32463fb43..7c4cf36881b4 100644
> --- a/Kconfig
> +++ b/Kconfig
> @@ -15,3 +15,4 @@ source "lib/Kconfig"
>  source "crypto/Kconfig"
>  source "firmware/Kconfig"
>  source "scripts/Kconfig"
> +source "test/Kconfig"
> diff --git a/Makefile b/Makefile
> index f6c0c4817046..0a93915cf347 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -581,7 +581,7 @@ endif
>  include $(srctree)/scripts/Makefile.lib
>  
>  # Objects we will link into barebox / subdirs we need to visit
> -common-y		:= common/ drivers/ commands/ lib/ crypto/ net/ fs/ firmware/
> +common-y		:= common/ drivers/ commands/ lib/ crypto/ net/ fs/ firmware/ test/
>  
>  include $(srctree)/arch/$(SRCARCH)/Makefile
>  
> diff --git a/commands/Makefile b/commands/Makefile
> index 447349fd157d..4b45d266fd56 100644
> --- a/commands/Makefile
> +++ b/commands/Makefile
> @@ -130,5 +130,6 @@ obj-$(CONFIG_CMD_SEED)		+= seed.o
>  obj-$(CONFIG_CMD_IP_ROUTE_GET)  += ip-route-get.o
>  obj-$(CONFIG_CMD_BTHREAD)	+= bthread.o
>  obj-$(CONFIG_CMD_UBSAN)		+= ubsan.o
> +obj-$(CONFIG_CMD_SELFTEST)	+= selftest.o
>  
>  UBSAN_SANITIZE_ubsan.o := y
> diff --git a/commands/selftest.c b/commands/selftest.c
> new file mode 100644
> index 000000000000..de5d100de04a
> --- /dev/null
> +++ b/commands/selftest.c
> @@ -0,0 +1,88 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +
> +#define pr_fmt(fmt) "selftest: " fmt
> +
> +#include <common.h>
> +#include <command.h>
> +#include <getopt.h>
> +#include <bselftest.h>
> +#include <complete.h>
> +
> +static int run_selftest(const char *match, bool list)
> +{
> +	struct selftest *test;
> +	int matches = 0;
> +	int err = 0;
> +
> +	list_for_each_entry(test, &selftests, list) {
> +		if (list) {
> +			printf("%s\n", test->name);
> +			matches++;
> +			continue;
> +		}
> +
> +		if (match && strcmp(test->name, match))
> +			continue;
> +
> +		err |= test->func();
> +		matches++;
> +	}
> +
> +	if (!matches) {
> +		if (match)
> +			printf("No tests matching '%s' found.\n", match);
> +		else
> +			printf("No matching tests found.\n");
> +
> +		return -EINVAL;
What if there is no tests at all: is that an error if no tests were ran?
despit not using a match condition ? 

> +	}
> +
> +	return err;
> +}
> +
snip
> diff --git a/test/self/core.c b/test/self/core.c
> new file mode 100644
> index 000000000000..d8c6e5d271ba
> --- /dev/null
> +++ b/test/self/core.c
> @@ -0,0 +1,39 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +
> +#define pr_fmt(fmt) "bselftest: " fmt
> +
> +#include <common.h>
> +#include <bselftest.h>
> +
> +LIST_HEAD(selftests);
> +
> +static int (*old_main)(void);
> +
> +static int run_selftests(void)
> +{
> +	struct selftest *test;
> +	int err = 0;
> +
> +	pr_notice("Configured tests will run now\n");
> +
> +	list_for_each_entry(test, &selftests, list)
> +		err |= test->func();
> +
> +	if (err)
> +		pr_err("Some selftests failed\n");
> +
> +	barebox_main = old_main;
> +	return barebox_main();
> +}
Why not call run_selftests in start_barebox, right before barebox_main ?
Or maybe add a selftest initcall to be run at the very end of initcalls.
So you don't have to save and restore the barebox_main function.

> +
> +static int init_selftests(void)
> +{
> +	if (!IS_ENABLED(CONFIG_SELFTEST_AUTORUN))
> +		return 0;
> +
> +	old_main = barebox_main;
> +	barebox_main = run_selftests;
> +
> +	return 0;
> +}
> +postenvironment_initcall(init_selftests);
> -- 
> 2.29.2
> 
> 
> _______________________________________________
> barebox mailing list
> barebox@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/barebox
> 


_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox


  reply	other threads:[~2021-04-12 10:14 UTC|newest]

Thread overview: 21+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-04-12  7:16 [PATCH 00/17] add barebox in-tree testing infrastructure Ahmad Fatoum
2021-04-12  7:16 ` [PATCH 01/17] kbuild: add install target Ahmad Fatoum
2021-04-12  7:16 ` [PATCH 02/17] kbuild: add ARCH={i386, x86_64} as aliases for x86 Ahmad Fatoum
2021-04-12  7:16 ` [PATCH 03/17] kbuild: add ARCH=um alias for sandbox Ahmad Fatoum
2021-04-12  7:16 ` [PATCH 04/17] MIPS: qemu-malta: generate swapped image as part of multi-image build Ahmad Fatoum
2021-04-12  9:27   ` Jules Maselbas
2021-04-12  9:45     ` Ahmad Fatoum
2021-04-12  7:16 ` [PATCH 05/17] MIPS: qemu-malta: replace board code with syscon-reboot Ahmad Fatoum
2021-04-12  7:16 ` [PATCH 06/17] MIPS: configs: qemu-malta: enable Virt I/O Ahmad Fatoum
2021-04-12  7:16 ` [PATCH 07/17] openrisc: set default KBUILD_IMAGE Ahmad Fatoum
2021-04-12  7:16 ` [PATCH 08/17] Documentation: boards: RISC-V: update TinyEMU support Ahmad Fatoum
2021-04-12  7:16 ` [PATCH 09/17] test: add basic barebox self-test infrastructure Ahmad Fatoum
2021-04-12 10:12   ` Jules Maselbas [this message]
2021-04-12  7:16 ` [PATCH 10/17] test: self: port Linux printf kselftest Ahmad Fatoum
2021-04-12  7:16 ` [PATCH 11/17] test: add labgrid configs for some emulated targets Ahmad Fatoum
2021-04-12  7:16 ` [PATCH 12/17] test: add first sample tests Ahmad Fatoum
2021-04-12  7:16 ` [PATCH 13/17] test: support running labgrid-pytest from out-of-tree build directory Ahmad Fatoum
2021-04-12  7:16 ` [PATCH 14/17] test: add emulate.pl, a runner for barebox on emulated targets Ahmad Fatoum
2021-04-12  7:16 ` [PATCH 15/17] test: self: run selftests as part of the pytest suite Ahmad Fatoum
2021-04-12  7:16 ` [PATCH 16/17] test: add bthread test Ahmad Fatoum
2021-04-12  7:16 ` [PATCH 17/17] [RFC] test: run ./test/emulate.pl in Github action Ahmad Fatoum

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210412101245.GB21066@tellis.lin.mbt.kalray.eu \
    --to=jmaselbas@kalray.eu \
    --cc=a.fatoum@pengutronix.de \
    --cc=barebox@lists.infradead.org \
    --cc=rcz@pengutronix.de \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox