mail archive of the barebox mailing list
 help / color / mirror / Atom feed
From: Ahmad Fatoum <a.fatoum@pengutronix.de>
To: barebox@lists.infradead.org
Cc: Ahmad Fatoum <a.fatoum@pengutronix.de>
Subject: [PATCH 12/14] ci: build: dynamically generate build matrix
Date: Mon,  5 Jun 2023 08:36:21 +0200	[thread overview]
Message-ID: <20230605063623.1254764-12-a.fatoum@pengutronix.de> (raw)
In-Reply-To: <20230605063623.1254764-1-a.fatoum@pengutronix.de>

We have some manual chunking of defconfigs, which makes it more
difficult to see on a glance which defconfig failed.

Let's add a job that generates a JSON of all available defconfigs
for the selected architectures and use that as build matrix for the
build job.

This gives us higher parallelism at the cost of more IO in the form
of container pulls and repository (--depth=1) fetches.

All in all, it seems to save 5 minutes execution time on average, so
let's make use of it.

Signed-off-by: Ahmad Fatoum <a.fatoum@pengutronix.de>
---
 .github/workflows/build-defconfigs.yml | 30 +++++++++++++++-----------
 scripts/list-defconfigs.sh             | 15 +++++++++++++
 2 files changed, 32 insertions(+), 13 deletions(-)
 create mode 100755 scripts/list-defconfigs.sh

diff --git a/.github/workflows/build-defconfigs.yml b/.github/workflows/build-defconfigs.yml
index c5c294300e03..ff9264721d87 100644
--- a/.github/workflows/build-defconfigs.yml
+++ b/.github/workflows/build-defconfigs.yml
@@ -2,8 +2,23 @@ name: build images
 
 on: [push, pull_request]
 
+env:
+  ARCH: arm mips powerpc riscv sandbox x86
+
 jobs:
+  defconfigs:
+    runs-on: ubuntu-latest
+    outputs:
+      matrix: ${{ steps.set-matrix.outputs.matrix }}
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v3
+
+      - name: Collect defconfigs
+        id: set-matrix
+        run: echo "matrix=$(scripts/list-defconfigs.sh $ARCH)" >> $GITHUB_OUTPUT
   build:
+    needs: defconfigs
     runs-on: ubuntu-latest
     container:
       image: ghcr.io/barebox/barebox/barebox-ci:latest
@@ -12,18 +27,7 @@ jobs:
 
     strategy:
       fail-fast: false
-      matrix:
-        arch: [mips, powerpc, riscv, sandbox, x86 ]
-        regex: ["*"]
-        include:
-          - arch: arm
-            regex: "[a-b]*"
-          - arch: arm
-            regex: "[c-m]*"
-          - arch: arm
-            regex: "[n-q]*"
-          - arch: arm
-            regex: "[r-z]*"
+      matrix: ${{ fromJSON(needs.defconfigs.outputs.matrix) }}
 
     steps:
     - name: Checkout code
@@ -36,4 +40,4 @@ jobs:
         ./test/generate-dummy-fw.sh
 
         ./MAKEALL -O build-${{matrix.arch}} -k test/kconfig/disable_size_check.kconf \
-                -k test/kconfig/disable_target_tools.kconf -e '${{matrix.regex}}'
+                -k test/kconfig/disable_target_tools.kconf '${{matrix.config}}'
diff --git a/scripts/list-defconfigs.sh b/scripts/list-defconfigs.sh
new file mode 100755
index 000000000000..eeae9fbfdc30
--- /dev/null
+++ b/scripts/list-defconfigs.sh
@@ -0,0 +1,15 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Output json formatted defconfig list for Github Action consumption
+
+ARCH=${@:-*}
+
+cd arch
+
+archs=$(for arch in $ARCH; do
+	ls -1 $arch/configs | xargs -i printf '{ "arch": "%s", "config": "%s" }\n' \
+		"$arch" "{}" | paste -sd ',' -
+done | paste -sd ',' -)
+
+echo '{ "include" : '" [ $archs ] }"
-- 
2.39.2




  parent reply	other threads:[~2023-06-05  6:37 UTC|newest]

Thread overview: 15+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-06-05  6:36 [PATCH 01/14] gitignore: don't ignore files in .github/ directory Ahmad Fatoum
2023-06-05  6:36 ` [PATCH 02/14] test/Containerfile: reduce duplication in cross toolchain handling Ahmad Fatoum
2023-06-05  6:36 ` [PATCH 03/14] test/Containerfile: install only one RISC-V toolchain Ahmad Fatoum
2023-06-05  6:36 ` [PATCH 04/14] test/Containerfile: drop i386 toolchain Ahmad Fatoum
2023-06-05  6:36 ` [PATCH 05/14] test/Containerfile: update to GCC 13.1.0 Ahmad Fatoum
2023-06-05  6:36 ` [PATCH 06/14] test/Containerfile: drop unneeded dependencies Ahmad Fatoum
2023-06-05  6:36 ` [PATCH 07/14] ci: test-defconfigs.yml: rename to build-defconfigs.yml Ahmad Fatoum
2023-06-05  6:36 ` [PATCH 08/14] ci: shorten name in YAML files Ahmad Fatoum
2023-06-05  6:36 ` [PATCH 09/14] ci: don't fail fast Ahmad Fatoum
2023-06-05  6:36 ` [PATCH 10/14] ci: don't use deprecated Node.js 12 actions Ahmad Fatoum
2023-06-05  6:36 ` [PATCH 11/14] ci: pytest: upload artifacts Ahmad Fatoum
2023-06-05  6:36 ` Ahmad Fatoum [this message]
2023-06-05  6:36 ` [PATCH 13/14] test: labgrid-env: rv64i: use QEmu built-in openSBI Ahmad Fatoum
2023-06-05  6:36 ` [PATCH 14/14] ci: pytest: test RISCV 64-bit as well Ahmad Fatoum
2023-06-06  9:39 ` [PATCH 01/14] gitignore: don't ignore files in .github/ directory Sascha Hauer

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20230605063623.1254764-12-a.fatoum@pengutronix.de \
    --to=a.fatoum@pengutronix.de \
    --cc=barebox@lists.infradead.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox