[PATCH v2 00/11] i.MX8M: speed up the PBL boot path (MMU, SHA-256, eMMC)

[PATCH v2 00/11] i.MX8M: speed up the PBL boot path (MMU, SHA-256, eMMC)

[Johannes Schneider]

This merges two v1 threads that review showed depend on the same first
step -- turning the MMU on in the PBL:

  - "crypto: sha256: PBL multi-block transform via ARMv8 Crypto
    Extensions" [1], where the ask was to keep the dispatch out of
    crypto/sha2.c [2]; and
  - "mci: imx-esdhc: speed up eMMC with ADMA2 + HS400 on i.MX8M" [3],
    where the ask was that PBL DMA needs the memory-synchronization
    (dma_map_*/dma_sync_*) the PBL lacks, and to concentrate on the MMU
    first since it is wanted anyway for a faster SHA-256 [4][5].

Ordered so it can be cut where it suits; each stage builds on the last:

  MMU
   1  enable the MMU + D-cache around the fw-external BL32 verify, via a
      new imx8m_mmu_early_enable() that derives the early-DRAM size from
      the DDR bus width [6]. ~720 KiB verify: ~2 s -> ~300 ms.

  SHA-256
   2  add the crypto-ext transform in a dedicated pbl/sha256.c, gated at
      runtime on ID_AA64ISAR0_EL1.SHA2 with a generic-C fallback.
      verify ~300 ms -> 3-5 ms.

  eMMC / PBL load
   3-7  barebox-proper eMMC speedups: SDHCI error-bail + i.MX ADMA2
        descriptor quirks, HS400/HS400ES, selectable transfer mode.
   8    enable the MMU before the PBL eMMC load so PIO runs warm-cached
        (load_bl33 713 ms -> 241 ms on i.MX8MM).
   *** DMA-free cut point. ***
   9-11 the PBL DMA path, now *with* the missing infrastructure: an opt-in
        CONFIG_PBL_HAS_DMA that gives the PBL real dma_map/dma_sync cache
        maintenance, SDHCI wired to it, and the i.MX8M PBL loader gaining
        SDMA/ADMA2 that select PBL_HAS_DMA and stay coherent with the
        D-cache on (load_bl33 241 ms -> 28 ms on i.MX8MM).

v1's "dma: make dma_mapping_error() NULL-safe" already landed in next and
is dropped. Based on next (261d2f8db0). Measured on i.MX8MM (GS05); both
the PIO+MMU and ADMA2 paths boot correctly (the BL32 SHA-256 verify
passing is the proof the PBL dma_sync is right).

[1] https://lists.infradead.org/pipermail/barebox/2026-June/056744.html
[2] https://lists.infradead.org/pipermail/barebox/2026-July/056890.html
[3] https://lists.infradead.org/pipermail/barebox/2026-June/056837.html
[4] https://lists.infradead.org/pipermail/barebox/2026-June/056842.html
[5] https://lists.infradead.org/pipermail/barebox/2026-July/056891.html
[6] https://lists.infradead.org/pipermail/barebox/2026-July/056889.html

Johannes Schneider (11):
  ARM: i.MX8M: enable MMU in PBL around fw-external BL32 verify
  crypto: sha256: PBL SHA-256 fast path via pbl/sha256.c
  mci: sdhci: bail out on ADMA/transfer errors instead of hanging
  mci: sdhci: honor BROKEN_ADMA_ZEROLEN_DESC / NO_ENDATTR_IN_NOPDESC
    quirks
  mci: imx-esdhc: mark the uSDHC ADMA2 descriptor quirks
  mci: imx-esdhc: support HS400 and HS400ES on i.MX8M
  mci: imx-esdhc: make the transfer mode selectable (PIO/SDMA/ADMA2)
  ARM: i.MX8M: enable the MMU before the PBL eMMC load
  dma: provide the streaming DMA API in the PBL (opt-in via PBL_HAS_DMA)
  mci: sdhci: honour dma_map/dma_sync in the PBL
  mci: imx-esdhc: DMA in the i.MX8M PBL eMMC loader

 arch/arm/cpu/Makefile          |  1 +
 arch/arm/crypto/Makefile       |  3 ++
 arch/arm/mach-imx/atf.c        | 30 +++++++++++
 common/Kconfig                 | 12 +++++
 crypto/Kconfig                 | 12 +++++
 drivers/dma/Makefile           |  1 +
 drivers/mci/Kconfig            | 46 ++++++++++++++--
 drivers/mci/imx-esdhc-common.c | 33 +++++++++---
 drivers/mci/imx-esdhc-pbl.c    | 91 ++++++++++++++++++++++++++++---
 drivers/mci/imx-esdhc.c        | 98 ++++++++++++++++++++++++++++++++--
 drivers/mci/imx-esdhc.h        | 14 +++++
 drivers/mci/sdhci.c            | 51 +++++++++++++-----
 drivers/mci/sdhci.h            |  4 ++
 include/crypto/pbl-sha.h       |  4 ++
 include/dma.h                  |  7 +--
 pbl/Makefile                   |  1 +
 pbl/decomp.c                   |  6 +--
 pbl/sha256.c                   | 85 +++++++++++++++++++++++++++++
 18 files changed, 459 insertions(+), 40 deletions(-)
 create mode 100644 pbl/sha256.c

-- 
2.43.0

[PATCH v2 01/11] ARM: i.MX8M: enable MMU in PBL around fw-external BL32 verify

[Johannes Schneider]

The BL32 fw-external blob is loaded into DRAM by the PBL and then
SHA-256-verified inside get_builtin_firmware_ext(). With the MMU off and
the D-cache cold that verify walks ~720 KiB through uncached DRAM; on a
Cortex-A53 that is ~2 s of pre-BL31 wall-clock every boot.

The verify is the only thing anchoring the BL32 content to the signed PBL
(HABv4 on i.MX8M only signs/loads what fits in OCRAM = the PBL; BL31/BL32
reach DRAM via PBL copies), so skipping the SHA-256 would be a security
regression.

Turn the MMU + D-cache on once DRAM is populated, right before the verify
+ BL31/BL32 memcpy, and drop it again before the BL31 entry (BL31 expects
MMU off). The verify drops from ~2 s to ~300 ms and BL31 early-init also
benefits from the warm cache.

Wrap the enable in imx8m_mmu_early_enable(), which picks the early-DRAM
size from the DDR bus width (16 for i.MX8MN, 32 otherwise), rather than
open-coding mmu_early_enable() at every call site [1]. Mirrors the
Rockchip PBL MMU handling.

[1] https://lists.infradead.org/pipermail/barebox/2026-July/056889.html

Assisted-by: Claude Opus 4.8 (1M context)
Signed-off-by: Johannes Schneider <johannes.schneider@leica-geosystems.com>
---

Notes:
    v2:
    - Wrap the enable in a new imx8m_mmu_early_enable() that derives the
      early-DRAM size from the DDR bus width (16 for i.MX8MN, 32 otherwise),
      instead of open-coding mmu_early_enable() at each call site (Sascha).
    - Folded into this series (was posted as a standalone patch).
    
    - Guard the mmu_early_enable() call with IS_ENABLED(CONFIG_MMU) (it has no
      !MMU stub), matching arch/arm/cpu/uncompress.c (Copilot).

 arch/arm/mach-imx/atf.c | 30 ++++++++++++++++++++++++++++++
 1 file changed, 30 insertions(+)

diff --git a/arch/arm/mach-imx/atf.c b/arch/arm/mach-imx/atf.c
index 948756e26b..0f57a204b7 100644
--- a/arch/arm/mach-imx/atf.c
+++ b/arch/arm/mach-imx/atf.c
@@ -20,6 +20,7 @@
 #include <mach/imx/xload.h>
 #include <mach/imx/snvs.h>
 #include <pbl.h>
+#include <asm/mmu.h>
 
 static void imx_adjust_optee_memory(void **bl32, void **bl32_image, size_t *bl32_size)
 {
@@ -130,6 +131,20 @@ static __noreturn void start_bl31_via_bl_params(void *bl31, void *bl32,
  *     4. Transfers control to BL31
  */
 
+/* Enable the MMU + D-cache over the i.MX8M early-DRAM window. */
+static void imx8m_mmu_early_enable(void)
+{
+	unsigned long memsize;
+
+	if (cpu_is_mx8mn())
+		memsize = imx8m_barebox_earlymem_size(16);
+	else
+		memsize = imx8m_barebox_earlymem_size(32);
+
+	if (IS_ENABLED(CONFIG_MMU))
+		mmu_early_enable(MX8M_DDR_CSD1_BASE_ADDR, memsize);
+}
+
 static __noreturn void
 imx8m_tfa_start_bl31(const void *tfa_bin, size_t tfa_size, void *tfa_dest,
 		     void *tee_bin, size_t tee_size, void *bl33, void *fdt)
@@ -188,6 +203,9 @@ imx8m_tfa_start_bl31(const void *tfa_bin, size_t tfa_size, void *tfa_dest,
 		     "r" (tfa_dest - 16) :
 		     "cc");
 
+	/* BL31 expects MMU off. */
+	mmu_disable();
+
 	/*
 	 * If enabled the bl_params are passed via x0 to the TF-A, except for
 	 * the i.MX8MQ which doesn't support bl_params yet.
@@ -283,6 +301,9 @@ __noreturn void __imx8mm_load_and_start_image_via_tfa(void *fdt, void *bl33)
 	imx8m_setup_snvs();
 	imx8mm_load_bl33(bl33);
 
+	/* Cache DRAM for the BL32 verify + BL31/BL32 memcpy that follow. */
+	imx8m_mmu_early_enable();
+
 	if (IS_ENABLED(CONFIG_FIRMWARE_IMX8MM_OPTEE)) {
 		get_builtin_firmware_ext(imx8mm_bl32_bin, bl33, &bl32);
 		get_builtin_firmware(imx8mm_bl31_bin_optee, &bl31);
@@ -346,6 +367,9 @@ __noreturn void __imx8mp_load_and_start_image_via_tfa(void *fdt, void *bl33)
 	imx8m_setup_snvs();
 	imx8mp_load_bl33(bl33);
 
+	/* Cache DRAM for the BL32 verify + BL31/BL32 memcpy that follow. */
+	imx8m_mmu_early_enable();
+
 	if (IS_ENABLED(CONFIG_FIRMWARE_IMX8MP_OPTEE)) {
 		get_builtin_firmware_ext(imx8mp_bl32_bin, bl33, &bl32);
 		get_builtin_firmware(imx8mp_bl31_bin_optee, &bl31);
@@ -409,6 +433,9 @@ __noreturn void __imx8mn_load_and_start_image_via_tfa(void *fdt, void *bl33)
 	imx8m_setup_snvs();
 	imx8mn_load_bl33(bl33);
 
+	/* Cache DRAM for the BL32 verify + BL31/BL32 memcpy that follow. */
+	imx8m_mmu_early_enable();
+
 	if (IS_ENABLED(CONFIG_FIRMWARE_IMX8MN_OPTEE)) {
 		get_builtin_firmware_ext(imx8mn_bl32_bin, bl33, &bl32);
 		get_builtin_firmware(imx8mn_bl31_bin_optee, &bl31);
@@ -466,6 +493,9 @@ __noreturn void __imx8mq_load_and_start_image_via_tfa(void *fdt, void *bl33)
 	imx8m_setup_snvs();
 	imx8mq_load_bl33(bl33);
 
+	/* Cache DRAM for the BL32 verify + BL31/BL32 memcpy that follow. */
+	imx8m_mmu_early_enable();
+
 	if (IS_ENABLED(CONFIG_FIRMWARE_IMX8MQ_OPTEE)) {
 		get_builtin_firmware_ext(imx8mq_bl32_bin, bl33, &bl32);
 		get_builtin_firmware(imx8mq_bl31_bin_optee, &bl31);
-- 
2.43.0

[PATCH v2 02/11] crypto: sha256: PBL SHA-256 fast path via pbl/sha256.c

[Johannes Schneider]

Hashing MB-scale blobs in the PBL -- e.g. the fw-external SHA-256 verify on
i.MX8M, ~720 KiB of BL32 -- spends hundreds of ms in the generic-C
sha256_transform() even with the D-cache warm.

Add a PBL-local one-shot

    void pbl_sha256(const void *buf, size_t len, u8 out[SHA256_DIGEST_SIZE]);

in a new pbl/sha256.c that picks the best transform available: the generic
digest API by default, or the ARMv8 Crypto Extensions asm core
(arch/arm/crypto/sha2-ce-core.S) under CONFIG_PBL_DIGEST_SHA256_ARM64_CE,
driven through the sha256_base_* helpers (the same batched block/finalize
framing sha2-ce-glue.c uses, without the crypto-API and kernel_neon_begin
shims the PBL has no use for). pbl_barebox_verify() calls it.

The ~720 KiB verify drops from ~300 ms (generic-C) to 3-5 ms (batched
crypto-ext) with a warm D-cache.

Signed-off-by: Johannes Schneider <johannes.schneider@leica-geosystems.com>
Assisted-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---

Notes:
    v2:
    - Move the PBL crypto-ext dispatch out of crypto/sha2.c into a new
      pbl/sha256.c with a one-shot pbl_sha256(); crypto/sha2.c is left
      untouched (Sascha).
    
    - Build pbl/sha256.o unconditionally (pbl-y), like crypto/sha2.o, so the
      ARMv8 CE core always finds its sha256_ce_offsetof_* glue; it no longer
      hangs off CONFIG_HAVE_IMAGE_COMPRESSION.
    
    - Gate the crypto-ext path on ID_AA64ISAR0_EL1.SHA2 and fall back to
      generic C, matching sha2-ce-glue.c; the CE instructions are optional on
      ARMv8 and trap as undefined otherwise (Copilot).

 arch/arm/crypto/Makefile |  3 ++
 crypto/Kconfig           | 12 ++++++
 include/crypto/pbl-sha.h |  4 ++
 pbl/Makefile             |  1 +
 pbl/decomp.c             |  6 +--
 pbl/sha256.c             | 85 ++++++++++++++++++++++++++++++++++++++++
 6 files changed, 106 insertions(+), 5 deletions(-)
 create mode 100644 pbl/sha256.c

diff --git a/arch/arm/crypto/Makefile b/arch/arm/crypto/Makefile
index 55b3ac0538..085c01db09 100644
--- a/arch/arm/crypto/Makefile
+++ b/arch/arm/crypto/Makefile
@@ -15,6 +15,9 @@ sha1-ce-y := sha1-ce-glue.o sha1-ce-core.o
 obj-$(CONFIG_DIGEST_SHA256_ARM64_CE) += sha2-ce.o
 sha2-ce-y := sha2-ce-glue.o sha2-ce-core.o
 
+# Reuse the asm core; the PBL glue lives in pbl/sha256.c.
+pbl-$(CONFIG_PBL_DIGEST_SHA256_ARM64_CE) += sha2-ce-core.o
+
 quiet_cmd_perl = PERL    $@
       cmd_perl = $(PERL) $(<) > $(@)
 
diff --git a/crypto/Kconfig b/crypto/Kconfig
index 528e9a0d22..3dfb316b32 100644
--- a/crypto/Kconfig
+++ b/crypto/Kconfig
@@ -107,6 +107,18 @@ config DIGEST_SHA256_ARM64_CE
 	  Architecture: arm64 using:
 	  - ARMv8 Crypto Extensions
 
+config PBL_DIGEST_SHA256_ARM64_CE
+	bool "SHA-256 in PBL via ARMv8 Crypto Extensions"
+	depends on CPU_V8 && PBL_IMAGE
+	help
+	  Use ARMv8 Crypto Extensions (sha256h/sha256h2/sha256su0/sha256su1)
+	  for the SHA-256 transform inside the PBL. Roughly 100x faster than
+	  the generic-C transform; for callers that hash large blobs (e.g.
+	  fw-external SHA-256 verifies) this is the difference between tens
+	  of ms and hundreds. Requires Cortex-A53 or later with the optional
+	  Crypto Extensions feature.
+
+
 endif
 
 config CRYPTO_PBKDF2
diff --git a/include/crypto/pbl-sha.h b/include/crypto/pbl-sha.h
index 7d323ab479..2508448ab4 100644
--- a/include/crypto/pbl-sha.h
+++ b/include/crypto/pbl-sha.h
@@ -3,6 +3,7 @@
 
 #define __PBL_SHA_H_
 
+#include <crypto/sha.h>
 #include <digest.h>
 #include <types.h>
 
@@ -10,4 +11,7 @@ int sha256_init(struct digest *desc);
 int sha256_update(struct digest *desc, const void *data, unsigned long len);
 int sha256_final(struct digest *desc, u8 *out);
 
+/* One-shot SHA-256 that picks the best transform available in the PBL. */
+void pbl_sha256(const void *buf, size_t len, u8 out[SHA256_DIGEST_SIZE]);
+
 #endif /* __PBL-SHA_H_ */
diff --git a/pbl/Makefile b/pbl/Makefile
index 45cfbf5fba..4506f192fe 100644
--- a/pbl/Makefile
+++ b/pbl/Makefile
@@ -6,6 +6,7 @@
 pbl-y += misc.o
 pbl-y += string.o
 pbl-y += malloc.o
+pbl-y += sha256.o
 pbl-$(CONFIG_HAVE_IMAGE_COMPRESSION) += decomp.o
 pbl-$(CONFIG_LIBFDT) += fdt.o
 pbl-$(CONFIG_PBL_CONSOLE) += console.o
diff --git a/pbl/decomp.c b/pbl/decomp.c
index 1539a6b67e..2b3c35012f 100644
--- a/pbl/decomp.c
+++ b/pbl/decomp.c
@@ -58,8 +58,6 @@ extern unsigned char sha_sum_end[];
 int pbl_barebox_verify(const void *compressed_start, unsigned int len,
 		       const void *hash, unsigned int hash_len)
 {
-	struct sha256_state sha_state = { 0 };
-	struct digest d = { .ctx = &sha_state };
 	char computed_hash[SHA256_DIGEST_SIZE];
 	int i;
 	const char *char_hash = hash;
@@ -67,9 +65,7 @@ int pbl_barebox_verify(const void *compressed_start, unsigned int len,
 	if (hash_len != SHA256_DIGEST_SIZE)
 		return -1;
 
-	sha256_init(&d);
-	sha256_update(&d, compressed_start, len);
-	sha256_final(&d, computed_hash);
+	pbl_sha256(compressed_start, len, computed_hash);
 	if (IS_ENABLED(CONFIG_DEBUG_LL)) {
 		puts_ll("CH ");
 
diff --git a/pbl/sha256.c b/pbl/sha256.c
new file mode 100644
index 0000000000..22af1d6907
--- /dev/null
+++ b/pbl/sha256.c
@@ -0,0 +1,85 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * pbl_sha256() - one-shot SHA-256 for the PBL, picking the best available
+ * transform (ARMv8 Crypto Extensions if present, else generic C).
+ */
+
+#include <common.h>
+#include <crypto/sha.h>
+#include <crypto/pbl-sha.h>
+#include <digest.h>
+
+static void pbl_sha256_generic(const void *buf, size_t len, u8 *out)
+{
+	struct sha256_state state = { };
+	struct digest d = { .ctx = &state, .length = SHA256_DIGEST_SIZE };
+
+	sha256_init(&d);
+	sha256_update(&d, buf, len);
+	sha256_final(&d, out);
+}
+
+#ifdef CONFIG_PBL_DIGEST_SHA256_ARM64_CE
+
+#include <crypto/sha256_base.h>
+#include <linux/linkage.h>
+#include <linux/kernel.h>
+#include <asm/sysreg.h>
+
+/* Layout sha2-ce-core.S expects; it reads count/finalize at the offsets below. */
+struct pbl_sha256_ce_state {
+	struct sha256_state	sst;
+	u32			finalize;
+};
+
+const u32 sha256_ce_offsetof_count    = offsetof(struct pbl_sha256_ce_state, sst.count);
+const u32 sha256_ce_offsetof_finalize = offsetof(struct pbl_sha256_ce_state, finalize);
+
+asmlinkage int sha2_ce_transform(struct pbl_sha256_ce_state *sst,
+				 const u8 *src, int blocks);
+
+static void pbl_sha2_ce_block(struct sha256_state *sst, const u8 *src,
+			      int blocks)
+{
+	struct pbl_sha256_ce_state *s =
+		container_of(sst, struct pbl_sha256_ce_state, sst);
+
+	/* finalize == 0: C does the padding via sha256_base_do_finalize(). */
+	s->finalize = 0;
+	while (blocks) {
+		int rem = sha2_ce_transform(s, src, blocks);
+
+		src += (blocks - rem) * SHA256_BLOCK_SIZE;
+		blocks = rem;
+	}
+}
+
+void pbl_sha256(const void *buf, size_t len, u8 out[SHA256_DIGEST_SIZE])
+{
+	struct pbl_sha256_ce_state s;
+	struct digest d = { .ctx = &s, .length = SHA256_DIGEST_SIZE };
+
+	/*
+	 * The Crypto Extensions are optional on ARMv8 and sha256h & co. trap
+	 * as undefined without them, so gate on ID_AA64ISAR0_EL1.SHA2 like
+	 * sha2-ce-glue.c does at registration and fall back to generic C.
+	 */
+	if (!(read_sysreg(ID_AA64ISAR0_EL1) & ID_AA64ISAR0_EL1_SHA2_MASK)) {
+		pbl_sha256_generic(buf, len, out);
+		return;
+	}
+
+	sha256_base_init(&d);
+	sha256_base_do_update(&d, buf, len, pbl_sha2_ce_block);
+	sha256_base_do_finalize(&d, pbl_sha2_ce_block);
+	sha256_base_finish(&d, out);
+}
+
+#else /* generic C transform via crypto/sha2.c */
+
+void pbl_sha256(const void *buf, size_t len, u8 out[SHA256_DIGEST_SIZE])
+{
+	pbl_sha256_generic(buf, len, out);
+}
+
+#endif
-- 
2.43.0

[PATCH v2 03/11] mci: sdhci: bail out on ADMA/transfer errors instead of hanging

[Johannes Schneider]

sdhci_transfer_data_dma() only watched for the completion and DMA-boundary
interrupts, so a transfer/ADMA error (SDHCI_INT_ERROR, e.g. an ADMA
descriptor fault) left it spinning until the 10s timeout. Check the error
bits and abort the transfer with -EIO, dumping the ADMA error status.

Assisted-by: Claude Opus 4.8 (1M context)
Signed-off-by: Johannes Schneider <johannes.schneider@leica-geosystems.com>
---

Notes:
    v2:
    - Drop the sdhci.h SDHCI_INT_ADMA_ERROR define; upstream now provides it.
      Rebased on next.

 drivers/mci/sdhci.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/drivers/mci/sdhci.c b/drivers/mci/sdhci.c
index 3474ef129b..ccd5d4b83c 100644
--- a/drivers/mci/sdhci.c
+++ b/drivers/mci/sdhci.c
@@ -779,6 +779,14 @@ int sdhci_transfer_data_dma(struct sdhci *sdhci, struct mci_cmd *cmd,
 			goto out;
 		}
 
+		/* e.g. ADMA error: SDHCI_INT_ERROR without a data-error bit */
+		if (irqstat & SDHCI_INT_ERROR) {
+			dev_err(dev, "transfer error: int 0x%08x adma 0x%02x\n",
+				irqstat, sdhci_read8(sdhci, SDHCI_ADMA_ERROR));
+			ret = -EIO;
+			goto out;
+		}
+
 		/*
 		 * We currently don't do anything fancy with DMA
 		 * boundaries, but as we can't disable the feature
-- 
2.43.0

[PATCH v2 04/11] mci: sdhci: honor BROKEN_ADMA_ZEROLEN_DESC / NO_ENDATTR_IN_NOPDESC quirks

[Johannes Schneider]

The ADMA2 table builder always emitted a 64 KiB chunk as a length-0 descriptor
(the 16-bit length field encodes 65536 as 0) and terminated the table with a
separate NOP-END descriptor. Some controllers mishandle these forms; Linux
gates the workarounds behind SDHCI_QUIRK_BROKEN_ADMA_ZEROLEN_DESC (cap the
length below 64 KiB so it is never 0) and SDHCI_QUIRK_NO_ENDATTR_IN_NOPDESC
(mark END on the last transfer descriptor instead of a trailing NOP-END).

Add both quirks to the shared SDHCI layer and honor them in the table builder.
Controllers that set neither keep the previous behaviour exactly, so this is a
no-op for everyone until a host driver opts in.

Assisted-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Assisted-by: GitHub Copilot CLI (gpt-5.5)
Signed-off-by: Johannes Schneider <johannes.schneider@leica-geosystems.com>
---

Notes:
    v2:
    - No functional change; rebased on next.

 drivers/mci/sdhci.c | 35 ++++++++++++++++++++++++-----------
 drivers/mci/sdhci.h |  4 ++++
 2 files changed, 28 insertions(+), 11 deletions(-)

diff --git a/drivers/mci/sdhci.c b/drivers/mci/sdhci.c
index ccd5d4b83c..8ed8a59342 100644
--- a/drivers/mci/sdhci.c
+++ b/drivers/mci/sdhci.c
@@ -646,31 +646,44 @@ static int sdhci_adma_write_desc(struct sdhci *host, void **desc,
 static int sdhci_adma_build_table(struct sdhci *host, dma_addr_t addr,
 				  unsigned int len)
 {
+	unsigned int max_len = SDHCI_ADMA2_MAX_LEN;
 	void *desc = host->adma_table;
 	int ret;
 
+	/*
+	 * Some controllers (e.g. the i.MX uSDHC) cannot handle a length-0
+	 * descriptor, which the 16-bit length field uses to encode the full
+	 * SDHCI_ADMA2_MAX_LEN. Cap the chunk just below that so the length is
+	 * never 0.
+	 */
+	if (host->quirks & SDHCI_QUIRK_BROKEN_ADMA_ZEROLEN_DESC)
+		max_len = SDHCI_ADMA2_MAX_LEN - 4;
+
 	while (len) {
-		unsigned int chunk = min_t(unsigned int, len,
-					   SDHCI_ADMA2_MAX_LEN);
+		unsigned int chunk = min_t(unsigned int, len, max_len);
+		unsigned int attr = ADMA2_TRAN_VALID;
 
+		len -= chunk;
 		/*
-		 * The length field is 16-bit; a length of 0 encodes
-		 * SDHCI_ADMA2_MAX_LEN bytes per the SD Host Controller
-		 * specification.
+		 * Controllers that ignore the END attribute in a trailing NOP
+		 * descriptor want END on the last transfer descriptor instead
+		 * (SDHCI_QUIRK_NO_ENDATTR_IN_NOPDESC).
 		 */
-		ret = sdhci_adma_write_desc(host, &desc, addr, chunk & 0xffff,
-					    ADMA2_TRAN_VALID);
+		if (!len && (host->quirks & SDHCI_QUIRK_NO_ENDATTR_IN_NOPDESC))
+			attr |= ADMA2_END;
+
+		ret = sdhci_adma_write_desc(host, &desc, addr, chunk & 0xffff, attr);
 		if (ret)
 			return ret;
 
 		addr += chunk;
-		len -= chunk;
 	}
 
-	/* Append a terminating descriptor (nop, end, valid). */
-	ret = sdhci_adma_write_desc(host, &desc, 0, 0, ADMA2_NOP_END_VALID);
+	if (host->quirks & SDHCI_QUIRK_NO_ENDATTR_IN_NOPDESC)
+		return 0;
 
-	return ret;
+	/* Append a terminating descriptor (nop, end, valid). */
+	return sdhci_adma_write_desc(host, &desc, 0, 0, ADMA2_NOP_END_VALID);
 }
 
 void sdhci_setup_data_dma(struct sdhci *sdhci, struct mci_data *data,
diff --git a/drivers/mci/sdhci.h b/drivers/mci/sdhci.h
index a3b8e65b55..2a7f9be575 100644
--- a/drivers/mci/sdhci.h
+++ b/drivers/mci/sdhci.h
@@ -341,6 +341,10 @@ struct sdhci {
 	bool v4_mode;		/* Host Version 4 Enable */
 
 	unsigned int quirks;
+/* Controller ignores the END attribute in a NOP ADMA2 descriptor */
+#define SDHCI_QUIRK_NO_ENDATTR_IN_NOPDESC	BIT(6)
+/* Controller cannot handle a length-0 (== 64 KiB) ADMA2 descriptor */
+#define SDHCI_QUIRK_BROKEN_ADMA_ZEROLEN_DESC	BIT(11)
 #define SDHCI_QUIRK_MISSING_CAPS		BIT(27)
 	unsigned int quirks2;
 /* The system physically doesn't support 1.8v, even if the host does */
-- 
2.43.0

[PATCH v2 05/11] mci: imx-esdhc: mark the uSDHC ADMA2 descriptor quirks

[Johannes Schneider]

The i.MX uSDHC ADMA2 engine cannot handle a length-0 (64 KiB) descriptor and
ignores the END attribute in a trailing NOP descriptor; large ADMA transfers
hang otherwise. Set SDHCI_QUIRK_BROKEN_ADMA_ZEROLEN_DESC and
SDHCI_QUIRK_NO_ENDATTR_IN_NOPDESC so the shared table builder emits a safe
descriptor shape, matching Linux sdhci-esdhc-imx. esdhc_populate_sdhci() runs
for both the PBL and the proper driver, so this covers every uSDHC ADMA path.

Assisted-by: Claude Opus 4.8 (1M context)
Signed-off-by: Johannes Schneider <johannes.schneider@leica-geosystems.com>
---

Notes:
    v2:
    - No functional change; rebased on next.

 drivers/mci/imx-esdhc-common.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/drivers/mci/imx-esdhc-common.c b/drivers/mci/imx-esdhc-common.c
index eb6a71915f..d31a2ad779 100644
--- a/drivers/mci/imx-esdhc-common.c
+++ b/drivers/mci/imx-esdhc-common.c
@@ -285,6 +285,14 @@ void esdhc_populate_sdhci(struct fsl_esdhc_host *host)
 		host->sdhci.read16 = esdhc_op_read16_le_tuning;
 		host->sdhci.write16 = esdhc_op_write16_le_tuning;
 	}
+
+	/*
+	 * The uSDHC ADMA2 engine mishandles a length-0 (64 KiB) descriptor and
+	 * the END attribute in a trailing NOP descriptor.
+	 */
+	if (esdhc_is_usdhc(host))
+		host->sdhci.quirks |= SDHCI_QUIRK_BROKEN_ADMA_ZEROLEN_DESC |
+				      SDHCI_QUIRK_NO_ENDATTR_IN_NOPDESC;
 }
 
 static bool esdhc_use_pio_mode(void)
-- 
2.43.0

[PATCH v2 06/11] mci: imx-esdhc: support HS400 and HS400ES on i.MX8M

[Johannes Schneider]

The i.MX8M uSDHC can run HS400/HS400ES, but the barebox driver topped out at
HS200: fsl,imx8mm-usdhc was routed to usdhc_imx6sx_data (HS200 only) and the
HS400 timing/strobe-DLL path was absent.

Add a usdhc_imx8mm_data with ESDHC_FLAG_HS200 | HS400 | HS400_ES and point the
imx8mm/imx8mn/imx8mp compatibles at it. Add the strobe-DLL register block and
esdhc_set_strobe_dll() (gate the card clock, pulse reset, program the read
delay target, poll for REF/SLV lock), an HS400 case in usdhc_set_timing()
(set MIX_CTRL DDREN|HS400_EN, re-apply the clock, run the strobe DLL),
HS400ES handling, and MMC_CAP2_HS400[_ES] advertisement gated on the flags.
The MIXCTRL preserve-mask in __esdhc_send_cmd() is widened from bits 22-25 to
22-27 so the HS400 mode bits survive command writes. Strobe read delay is
configurable via the existing fsl,strobe-dll-delay-target DT binding
(default 0x7).

usdhc_imx8mm_data keeps ESDHC_FLAG_STD_TUNING: barebox already drove these
parts (fsl,imx8mm-usdhc) at HS200 with standard tuning, and U-Boot uses it for
the i.MX8M family too. (Linux uses manual tuning for imx8mm; moving barebox to
manual tuning would be a separate, separately-tested change.)

Register sequence mirrors Linux sdhci-esdhc-imx.c; the barebox idiom follows
the Rockchip dwcmshc HS400 support.

Assisted-by: Claude Opus 4.8 (1M context)
Signed-off-by: Johannes Schneider <johannes.schneider@leica-geosystems.com>
---

Notes:
    v2:
    - No functional change; rebased on next.

 drivers/mci/imx-esdhc-common.c |  4 +-
 drivers/mci/imx-esdhc.c        | 91 ++++++++++++++++++++++++++++++++--
 drivers/mci/imx-esdhc.h        | 14 ++++++
 3 files changed, 103 insertions(+), 6 deletions(-)

diff --git a/drivers/mci/imx-esdhc-common.c b/drivers/mci/imx-esdhc-common.c
index d31a2ad779..a0c2dee32d 100644
--- a/drivers/mci/imx-esdhc-common.c
+++ b/drivers/mci/imx-esdhc-common.c
@@ -360,8 +360,8 @@ int __esdhc_send_cmd(struct fsl_esdhc_host *host, struct mci_cmd *cmd,
 	if (esdhc_is_usdhc(host)) {
 		/* write lower-half of xfertyp to mixctrl */
 		mixctrl = xfertyp;
-		/* Keep the bits 22-25 of the register as is */
-		mixctrl |= (sdhci_read32(&host->sdhci, IMX_SDHCI_MIXCTRL) & (0xF << 22));
+		/* keep tuning bits (22-25) and HS400 mode bits (26-27) as is */
+		mixctrl |= (sdhci_read32(&host->sdhci, IMX_SDHCI_MIXCTRL) & (0x3F << 22));
 		mixctrl |= mci_timing_is_ddr(host->sdhci.timing) ? MIX_CTRL_DDREN : 0;
 		sdhci_write32(&host->sdhci, IMX_SDHCI_MIXCTRL, mixctrl);
 	}
diff --git a/drivers/mci/imx-esdhc.c b/drivers/mci/imx-esdhc.c
index ba33daffc4..e123bd3ede 100644
--- a/drivers/mci/imx-esdhc.c
+++ b/drivers/mci/imx-esdhc.c
@@ -274,12 +274,51 @@ static int esdhc_change_pinstate(struct fsl_esdhc_host *host,
 }
 
 
+/* Ported from Linux esdhc_set_strobe_dll() in drivers/mmc/host/sdhci-esdhc-imx.c */
+static void esdhc_set_strobe_dll(struct fsl_esdhc_host *host)
+{
+	u32 strobe_delay, v;
+	int ret;
+
+	/* disable the card clock before (re)programming the strobe DLL */
+	sdhci_write32(&host->sdhci, ESDHC_VENDOR_SPEC,
+		      sdhci_read32(&host->sdhci, ESDHC_VENDOR_SPEC) &
+		      ~ESDHC_VENDOR_SPEC_FRC_SDCLK_ON);
+	ret = esdhc_poll(host, ESDHC_PRSSTAT, v, (v & ESDHC_CLOCK_GATE_OFF),
+			 100 * USECOND);
+	if (ret)
+		dev_warn(host->dev, "card clock not gated off before strobe DLL\n");
+
+	/* force a reset on the strobe DLL, then clear it before any setting */
+	sdhci_write32(&host->sdhci, ESDHC_STROBE_DLL_CTRL,
+		      ESDHC_STROBE_DLL_CTRL_RESET);
+	sdhci_write32(&host->sdhci, ESDHC_STROBE_DLL_CTRL, 0);
+
+	if (host->boarddata.strobe_dll_delay_target)
+		strobe_delay = host->boarddata.strobe_dll_delay_target;
+	else
+		strobe_delay = ESDHC_STROBE_DLL_CTRL_SLV_DLY_TARGET_DEFAULT;
+
+	v = ESDHC_STROBE_DLL_CTRL_ENABLE |
+	    ESDHC_STROBE_DLL_CTRL_SLV_UPDATE_INT_DEFAULT |
+	    (strobe_delay << ESDHC_STROBE_DLL_CTRL_SLV_DLY_TARGET_SHIFT);
+	sdhci_write32(&host->sdhci, ESDHC_STROBE_DLL_CTRL, v);
+
+	ret = esdhc_poll(host, ESDHC_STROBE_DLL_STATUS, v,
+			 (v & ESDHC_STROBE_DLL_STS_REF_LOCK) &&
+			 (v & ESDHC_STROBE_DLL_STS_SLV_LOCK),
+			 50 * USECOND);
+	if (ret)
+		dev_warn(host->dev,
+			 "HS400 strobe DLL not locked in 50us, status 0x%08x\n", v);
+}
+
 static void usdhc_set_timing(struct fsl_esdhc_host *host, enum mci_timing timing)
 {
 	u32 mixctrl;
 
 	mixctrl = sdhci_read32(&host->sdhci, IMX_SDHCI_MIXCTRL);
-	mixctrl &= ~MIX_CTRL_DDREN;
+	mixctrl &= ~(MIX_CTRL_DDREN | MIX_CTRL_HS400_EN);
 
 	switch (timing) {
 	case MMC_TIMING_UHS_DDR50:
@@ -296,6 +335,14 @@ static void usdhc_set_timing(struct fsl_esdhc_host *host, enum mci_timing timing
 			sdhci_write32(&host->sdhci, IMX_SDHCI_DLL_CTRL, v);
 		}
 		break;
+	case MMC_TIMING_MMC_HS400:
+		/*
+		 * HS400 is a DDR mode; the strobe DLL is set up later from
+		 * esdhc_set_ios() once the HS400 clock is running.
+		 */
+		mixctrl |= MIX_CTRL_DDREN | MIX_CTRL_HS400_EN;
+		sdhci_write32(&host->sdhci, IMX_SDHCI_MIXCTRL, mixctrl);
+		break;
 	case MMC_TIMING_UHS_SDR12:
 	case MMC_TIMING_UHS_SDR25:
 	case MMC_TIMING_UHS_SDR50:
@@ -339,6 +386,19 @@ static void layerscape_set_timing(struct fsl_esdhc_host *host, enum mci_timing t
 	host->sdhci.timing = timing;
 }
 
+static void esdhc_hs400_enhanced_strobe(struct mci_host *mci, struct mci_ios *ios)
+{
+	struct fsl_esdhc_host *host = to_fsl_esdhc(mci);
+	u32 m;
+
+	m = sdhci_read32(&host->sdhci, IMX_SDHCI_MIXCTRL);
+	if (ios->enhanced_strobe)
+		m |= MIX_CTRL_HS400_ES;
+	else
+		m &= ~MIX_CTRL_HS400_ES;
+	sdhci_write32(&host->sdhci, IMX_SDHCI_MIXCTRL, m);
+}
+
 static void esdhc_set_ios(struct mci_host *mci, struct mci_ios *ios)
 {
 	struct fsl_esdhc_host *host = to_fsl_esdhc(mci);
@@ -386,6 +446,13 @@ static void esdhc_set_ios(struct mci_host *mci, struct mci_ios *ios)
 		return;
 	}
 
+	/*
+	 * (Re)program the strobe DLL on every HS400 set_ios(): mci-core selects
+	 * HS400 timing while still at the old clock and only raises the clock on
+	 * a later set_ios(), so the DLL must lock against the final read clock.
+	 */
+	if (ios->timing == MMC_TIMING_MMC_HS400 && ios->clock)
+		esdhc_set_strobe_dll(host);
 }
 
 static int esdhc_card_present(struct mci_host *mci)
@@ -511,6 +578,7 @@ static const struct mci_ops fsl_esdhc_ops = {
 	.set_ios = esdhc_set_ios,
 	.init = esdhc_init,
 	.card_present = esdhc_card_present,
+	.hs400_enhanced_strobe = esdhc_hs400_enhanced_strobe,
 };
 
 static void fsl_esdhc_probe_dt(struct device *dev, struct fsl_esdhc_host *host)
@@ -528,6 +596,9 @@ static void fsl_esdhc_probe_dt(struct device *dev, struct fsl_esdhc_host *host)
 		boarddata->tuning_start_tap = ESDHC_TUNING_START_TAP_DEFAULT;
 	if (of_property_read_u32(np, "fsl,delay-line", &boarddata->delay_line))
 		boarddata->delay_line = 0;
+	if (of_property_read_u32(np, "fsl,strobe-dll-delay-target",
+			     &boarddata->strobe_dll_delay_target))
+		boarddata->strobe_dll_delay_target = 0;
 
 	if (esdhc_is_usdhc(host) && !IS_ERR(host->pinctrl)) {
 		host->pins_100mhz = pinctrl_lookup_state(host->pinctrl,
@@ -567,6 +638,11 @@ static bool usdhc_setup_tuning(struct fsl_esdhc_host *host)
 	mci->ops.execute_tuning = usdhc_execute_tuning;
 	mci->caps2 |= MMC_CAP2_HS200;
 
+	if (host->socdata->flags & ESDHC_FLAG_HS400)
+		mci->caps2 |= MMC_CAP2_HS400_1_8V;
+	if (host->socdata->flags & ESDHC_FLAG_HS400_ES)
+		mci->caps2 |= MMC_CAP2_HS400_ES;
+
 	return true;
 }
 
@@ -684,6 +760,13 @@ static struct esdhc_soc_data usdhc_imx6sx_data = {
 	.clkidx = "per",
 };
 
+static struct esdhc_soc_data usdhc_imx8mm_data = {
+	.flags = ESDHC_FLAG_USDHC | ESDHC_FLAG_STD_TUNING
+	       | ESDHC_FLAG_HAVE_CAP1 | ESDHC_FLAG_HS200
+	       | ESDHC_FLAG_HS400 | ESDHC_FLAG_HS400_ES,
+	.clkidx = "per",
+};
+
 static struct esdhc_soc_data esdhc_ls_be_data = {
 	.flags = ESDHC_FLAG_MULTIBLK_NO_INT | ESDHC_FLAG_BIGENDIAN |
 		 ESDHC_FLAG_LAYERSCAPE,
@@ -702,9 +785,9 @@ static __maybe_unused struct of_device_id fsl_esdhc_compatible[] = {
 	{ .compatible = "fsl,imx6sl-usdhc", .data = &usdhc_imx6sl_data },
 	{ .compatible = "fsl,imx6sx-usdhc", .data = &usdhc_imx6sx_data },
 	{ .compatible = "fsl,imx8mq-usdhc", .data = &usdhc_imx6sx_data },
-	{ .compatible = "fsl,imx8mm-usdhc", .data = &usdhc_imx6sx_data },
-	{ .compatible = "fsl,imx8mn-usdhc", .data = &usdhc_imx6sx_data },
-	{ .compatible = "fsl,imx8mp-usdhc", .data = &usdhc_imx6sx_data },
+	{ .compatible = "fsl,imx8mm-usdhc", .data = &usdhc_imx8mm_data },
+	{ .compatible = "fsl,imx8mn-usdhc", .data = &usdhc_imx8mm_data },
+	{ .compatible = "fsl,imx8mp-usdhc", .data = &usdhc_imx8mm_data },
 	{ .compatible = "fsl,ls1028a-esdhc",.data = &esdhc_ls_le_data  },
 	{ .compatible = "fsl,ls1046a-esdhc",.data = &esdhc_ls_be_data  },
 	{ /* sentinel */ }
diff --git a/drivers/mci/imx-esdhc.h b/drivers/mci/imx-esdhc.h
index df8aca18ad..5562371652 100644
--- a/drivers/mci/imx-esdhc.h
+++ b/drivers/mci/imx-esdhc.h
@@ -66,6 +66,17 @@
 #define  IMX_SDHCI_DLL_CTRL_OVERRIDE_EN_SHIFT	8
 #define IMX_SDHCI_MIX_CTRL_FBCLK_SEL	BIT(25)
 
+/* HS400 data strobe line delay-locked loop (read clock) */
+#define ESDHC_STROBE_DLL_CTRL				0x70
+#define  ESDHC_STROBE_DLL_CTRL_ENABLE			BIT(0)
+#define  ESDHC_STROBE_DLL_CTRL_RESET			BIT(1)
+#define  ESDHC_STROBE_DLL_CTRL_SLV_DLY_TARGET_SHIFT	3
+#define  ESDHC_STROBE_DLL_CTRL_SLV_DLY_TARGET_DEFAULT	0x7
+#define  ESDHC_STROBE_DLL_CTRL_SLV_UPDATE_INT_DEFAULT	(4 << 20)
+#define ESDHC_STROBE_DLL_STATUS				0x74
+#define  ESDHC_STROBE_DLL_STS_REF_LOCK			BIT(1)
+#define  ESDHC_STROBE_DLL_STS_SLV_LOCK			BIT(0)
+
 /* pltfm-specific */
 #define ESDHC_HOST_CONTROL_LE	0x20
 
@@ -163,6 +174,8 @@
 #define ESDHC_FLAG_BIGENDIAN		BIT(10)
 /* Layerscape variant ls1046a, ls1028a, ls1088a, revisit for ls1012a */
 #define ESDHC_FLAG_LAYERSCAPE		BIT(11)
+/* The IP supports HS400ES mode */
+#define ESDHC_FLAG_HS400_ES		BIT(12)
 
 struct esdhc_soc_data {
 	u32 flags;
@@ -177,6 +190,7 @@ struct esdhc_platform_data {
 	unsigned int delay_line;
 	unsigned int tuning_step;       /* The delay cell steps in tuning procedure */
 	unsigned int tuning_start_tap;	/* The start delay cell point in tuning procedure */
+	unsigned int strobe_dll_delay_target;	/* HS400 strobe read-clock delay cell */
 };
 
 struct fsl_esdhc_host {
-- 
2.43.0

[PATCH v2 07/11] mci: imx-esdhc: make the transfer mode selectable (PIO/SDMA/ADMA2)

[Johannes Schneider]

The driver only offered a PIO-or-DMA toggle (CONFIG_MCI_IMX_ESDHC_PIO) and
never called sdhci_setup_adma(), so the controller-resident ADMA2 engine was
unreachable and DMA always meant SDMA.

Replace the single bool with a choice of PIO/SDMA/ADMA2, defaulting to SDMA so
existing configs keep their current behaviour, and call sdhci_setup_adma() in
probe under the ADMA2 option (mirroring the Rockchip dwcmshc driver). It
returns -ENOTSUPP and cleanly falls back to SDMA on controllers without ADMA2
capability, so the new option is safe on all i.MX/Layerscape variants.

Assisted-by: Claude Opus 4.8 (1M context)
Signed-off-by: Johannes Schneider <johannes.schneider@leica-geosystems.com>
---

Notes:
    v2:
    - No functional change; rebased on next.

 drivers/mci/Kconfig     | 19 ++++++++++++++++---
 drivers/mci/imx-esdhc.c |  7 +++++++
 2 files changed, 23 insertions(+), 3 deletions(-)

diff --git a/drivers/mci/Kconfig b/drivers/mci/Kconfig
index 55e63aad92..ebe771216f 100644
--- a/drivers/mci/Kconfig
+++ b/drivers/mci/Kconfig
@@ -177,11 +177,24 @@ config MCI_IMX_ESDHC
 	  Enable this entry to add support to read and write SD cards on a
 	  Freescale i.MX25/35/51 based system.
 
-config MCI_IMX_ESDHC_PIO
-	bool "use PIO mode"
+choice
+	prompt "i.MX esdhc transfer mode"
+	default MCI_IMX_ESDHC_SDMA
 	depends on MCI_IMX_ESDHC
 	help
-	  mostly useful for debugging. Normally you should use DMA.
+	  How the driver moves data words to/from the controller. SDMA and
+	  ADMA2 use the controller's DMA engine; PIO is CPU-driven and slow.
+
+config MCI_IMX_ESDHC_PIO
+	bool "PIO"
+
+config MCI_IMX_ESDHC_SDMA
+	bool "SDMA"
+
+config MCI_IMX_ESDHC_ADMA2
+	bool "ADMA2"
+
+endchoice
 
 config MCI_OMAP_HSMMC
 	bool "OMAP HSMMC"
diff --git a/drivers/mci/imx-esdhc.c b/drivers/mci/imx-esdhc.c
index e123bd3ede..3536ebd6b8 100644
--- a/drivers/mci/imx-esdhc.c
+++ b/drivers/mci/imx-esdhc.c
@@ -698,6 +698,13 @@ static int fsl_esdhc_probe(struct device *dev)
 	if (ret)
 		goto err_clk_disable;
 
+	if (IS_ENABLED(CONFIG_MCI_IMX_ESDHC_ADMA2)) {
+		ret = sdhci_setup_adma(&host->sdhci);
+		if (ret && ret != -ENOTSUPP)
+			dev_warn(dev, "ADMA setup failed (%pe), falling back to SDMA\n",
+				 ERR_PTR(ret));
+	}
+
 	if (esdhc_is_usdhc(host) || esdhc_is_layerscape(host))
 		mci->host_caps |= MMC_CAP_3_3V_DDR | MMC_CAP_1_8V_DDR;
 
-- 
2.43.0

[PATCH v2 08/11] ARM: i.MX8M: enable the MMU before the PBL eMMC load

[Johannes Schneider]

The i.MX8MM/8MP PBL reads the next stage from eMMC with the MMU off, so the
PIO copy and the CPU's writes to DRAM are uncached and slow: on an i.MX8MM
loading the ~1.6 MB next stage took ~713 ms.

Move the imx8m_mmu_early_enable() up to before imx8m{m,p}_load_bl33() so the
copy runs against a warm D-cache. Measured on i.MX8MM: load_bl33 713 ms ->
241 ms. The MMU then stays on for the BL32 verify + BL31/BL32 memcpy that
follow, so this replaces the post-load enable for these two SoCs.

Signed-off-by: Johannes Schneider <johannes.schneider@leica-geosystems.com>
Assisted-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---

Notes:
    v2:
    - New patch. The maintainer-suggested DMA-free speedup: enable the MMU
      before the PBL eMMC load so PIO runs warm-cached, split out so it
      stands on its own (Sascha).

 arch/arm/mach-imx/atf.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/arch/arm/mach-imx/atf.c b/arch/arm/mach-imx/atf.c
index 0f57a204b7..4b6f7ce5e1 100644
--- a/arch/arm/mach-imx/atf.c
+++ b/arch/arm/mach-imx/atf.c
@@ -299,10 +299,10 @@ __noreturn void __imx8mm_load_and_start_image_via_tfa(void *fdt, void *bl33)
 	imx8mm_init_scratch_space();
 	imx8m_save_bootrom_log();
 	imx8m_setup_snvs();
-	imx8mm_load_bl33(bl33);
 
-	/* Cache DRAM for the BL32 verify + BL31/BL32 memcpy that follow. */
+	/* MMU on before the load: PIO runs warm-cached; also serves the verify. */
 	imx8m_mmu_early_enable();
+	imx8mm_load_bl33(bl33);
 
 	if (IS_ENABLED(CONFIG_FIRMWARE_IMX8MM_OPTEE)) {
 		get_builtin_firmware_ext(imx8mm_bl32_bin, bl33, &bl32);
@@ -365,10 +365,10 @@ __noreturn void __imx8mp_load_and_start_image_via_tfa(void *fdt, void *bl33)
 	imx8mp_init_scratch_space();
 	imx8m_save_bootrom_log();
 	imx8m_setup_snvs();
-	imx8mp_load_bl33(bl33);
 
-	/* Cache DRAM for the BL32 verify + BL31/BL32 memcpy that follow. */
+	/* MMU on before the load: PIO runs warm-cached; also serves the verify. */
 	imx8m_mmu_early_enable();
+	imx8mp_load_bl33(bl33);
 
 	if (IS_ENABLED(CONFIG_FIRMWARE_IMX8MP_OPTEE)) {
 		get_builtin_firmware_ext(imx8mp_bl32_bin, bl33, &bl32);
-- 
2.43.0

[PATCH v2 09/11] dma: provide the streaming DMA API in the PBL (opt-in via PBL_HAS_DMA)

[Johannes Schneider]

The PBL has always gotten no-op dma_map_single()/dma_sync_single_for_*()/
dma_unmap_single() stubs, on the assumption that DMA buffers live in
uncached memory because the MMU is only turned on in barebox_arm_entry().
That assumption breaks the moment a PBL DMA user runs with the MMU (and
thus the D-cache) on -- which is the direction the i.MX8M PBL is moving in
for a fast, warm-cache next-stage load.

Add an opt-in CONFIG_PBL_HAS_DMA that routes the PBL through the same real
dma_map/dma_sync implementation (drivers/dma/map.c) and arch cache
maintenance (arch_sync_dma_for_*) that barebox proper uses, rather than the
stubs. It is a plain selectable bool: boards/drivers that do PBL DMA with
the MMU on select it; everyone else is byte-for-byte unchanged and keeps
the stubs.

On ARM this also builds dma_$(S64_32).o (arch_sync_dma_for_device) for the
PBL; arch_sync_dma_for_cpu and the dma_{inv,flush}_range cache ops already
come in via the PBL-linked mmu-common.o / mmu_$(S64_32).o.

No functional change unless PBL_HAS_DMA is selected.

Assisted-by: Claude Opus 4.8 (1M context)
Signed-off-by: Johannes Schneider <johannes.schneider@leica-geosystems.com>

---

Notes:
    v2:
    - New patch. Adds the dma_map/dma_sync cache-maintenance infrastructure
      the PBL was missing, behind an opt-in CONFIG_PBL_HAS_DMA (Sascha).

 arch/arm/cpu/Makefile |  1 +
 common/Kconfig        | 12 ++++++++++++
 drivers/dma/Makefile  |  1 +
 include/dma.h         |  7 ++++---
 4 files changed, 18 insertions(+), 3 deletions(-)

diff --git a/arch/arm/cpu/Makefile b/arch/arm/cpu/Makefile
index 467ef17bfd..d1e94d9a13 100644
--- a/arch/arm/cpu/Makefile
+++ b/arch/arm/cpu/Makefile
@@ -7,6 +7,7 @@ pbl-$(CONFIG_ARM_EXCEPTIONS_PBL) += exceptions_$(S64_32).o interrupts_$(S64_32).
 obj-pbl-$(CONFIG_MMU) += mmu-common.o
 obj-pbl-$(CONFIG_MMU) += mmu_$(S64_32).o
 obj-$(CONFIG_MMU) += dma_$(S64_32).o
+pbl-$(CONFIG_PBL_HAS_DMA) += dma_$(S64_32).o
 obj-pbl-y += lowlevel_$(S64_32).o
 obj-pbl-$(CONFIG_CPU_32v7) += hyp.o
 AFLAGS_hyp.o :=-Wa,-march=armv7-a -Wa,-mcpu=all
diff --git a/common/Kconfig b/common/Kconfig
index 2c75844be9..055f169431 100644
--- a/common/Kconfig
+++ b/common/Kconfig
@@ -87,6 +87,18 @@ config HAS_DMA
 	  Drivers that depend on a DMA implementation can depend on this
 	  config, so that you don't get a compilation error.
 
+config PBL_HAS_DMA
+	bool
+	depends on HAS_DMA && MMU
+	help
+	  Make the streaming DMA API -- dma_map_single(),
+	  dma_sync_single_for_cpu()/_for_device() and dma_unmap_single() --
+	  available in the PBL, backed by the same arch_sync_dma_for_*() cache
+	  maintenance barebox proper uses. Without this the PBL only gets no-op
+	  stubs that assume DMA buffers live in uncached memory, which is only
+	  true while the MMU (and thus the D-cache) is off. A PBL DMA user that
+	  runs with the MMU on must select this so its transfers stay coherent.
+
 config GENERIC_GPIO
 	bool
 
diff --git a/drivers/dma/Makefile b/drivers/dma/Makefile
index 66f9dc2756..c941b69c89 100644
--- a/drivers/dma/Makefile
+++ b/drivers/dma/Makefile
@@ -1,6 +1,7 @@
 # SPDX-License-Identifier: GPL-2.0-only
 obj-$(CONFIG_DMADEVICES)	+= dma-devices.o
 obj-$(CONFIG_HAS_DMA)		+= map.o
+pbl-$(CONFIG_PBL_HAS_DMA)	+= map.o
 obj-$(CONFIG_DMA_API_DEBUG)	+= debug.o
 obj-$(CONFIG_MXS_APBH_DMA)	+= apbh_dma.o
 obj-$(CONFIG_OF_DMA_COHERENCY)	+= of_fixups.o
diff --git a/include/dma.h b/include/dma.h
index 60937a69aa..2a6f3d804a 100644
--- a/include/dma.h
+++ b/include/dma.h
@@ -102,7 +102,7 @@ void arch_sync_dma_for_device(void *vaddr, size_t size,
 			      enum dma_data_direction dir);
 #endif
 
-#if IN_PROPER
+#if IN_PROPER || IS_ENABLED(CONFIG_PBL_HAS_DMA)
 void dma_sync_single_for_cpu(struct device *dev, dma_addr_t address,
 			     size_t size, enum dma_data_direction dir);
 
@@ -116,8 +116,9 @@ void dma_unmap_single(struct device *dev, dma_addr_t dma_addr,
 		      size_t size, enum dma_data_direction dir);
 #else
 /*
- * assumes buffers are in coherent/uncached memory, e.g. because
- * MMU is only enabled in barebox_arm_entry which hasn't run yet.
+ * PBL without CONFIG_PBL_HAS_DMA: assumes buffers are in coherent/uncached
+ * memory, e.g. because the MMU is only enabled in barebox_arm_entry which
+ * hasn't run yet. Select PBL_HAS_DMA to get real cache maintenance instead.
  */
 static inline void dma_sync_single_for_cpu(struct device *dev, dma_addr_t address,
 					   size_t size, enum dma_data_direction dir)
-- 
2.43.0

[PATCH v2 10/11] mci: sdhci: honour dma_map/dma_sync in the PBL

[Johannes Schneider]

The SDHCI DMA path already funnels through dma_map_single() (invalidate
the destination before a read) and dma_unmap_single() (invalidate it
again afterwards), but two things kept that from working in the PBL:

  - sdhci_teardown_data() short-circuited on IN_PBL, so the post-transfer
    invalidate never ran. Drop the IN_PBL guard; with CONFIG_PBL_HAS_DMA
    dma_unmap_single() now does the cache maintenance, and without it the
    call is a no-op stub, so a plain PBL is unaffected.

  - the ADMA2 descriptor table in the PBL is a plain buffer (cacheable
    once the MMU is on) rather than the dma_alloc_coherent() region used
    in barebox proper, so the CPU-written descriptors could sit in the
    D-cache while the engine reads stale DRAM. Clean the table out to
    memory right after building it. Guarded by IN_PBL (proper keeps its
    coherent table) and a no-op without CONFIG_PBL_HAS_DMA.

Together with CONFIG_PBL_HAS_DMA this makes SDMA and ADMA2 transfers
coherent in the PBL with the MMU on. No functional change for barebox
proper or for a PBL built without PBL_HAS_DMA.

Assisted-by: Claude Opus 4.8 (1M context)
Signed-off-by: Johannes Schneider <johannes.schneider@leica-geosystems.com>

---

Notes:
    v2:
    - New patch. Wire the SDHCI PBL transfer path to the new PBL DMA API
      (post-transfer invalidate + ADMA descriptor-table flush).

 drivers/mci/sdhci.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/drivers/mci/sdhci.c b/drivers/mci/sdhci.c
index 8ed8a59342..8108e0b54f 100644
--- a/drivers/mci/sdhci.c
+++ b/drivers/mci/sdhci.c
@@ -732,6 +732,12 @@ void sdhci_setup_data_dma(struct sdhci *sdhci, struct mci_data *data,
 			return;
 		}
 		sdhci_set_adma_addr(sdhci, sdhci->adma_addr);
+
+		/* PBL descriptor table is cacheable; flush it before the engine reads it. */
+		if (IN_PBL)
+			dma_sync_single_for_device(dev, sdhci->adma_addr,
+						   sdhci->adma_table_sz,
+						   DMA_TO_DEVICE);
 	} else {
 		sdhci_set_sdma_addr(sdhci, *dma);
 	}
@@ -743,7 +749,7 @@ void sdhci_teardown_data(struct sdhci *sdhci,
 	struct device *dev = sdhci_dev(sdhci);
 	unsigned nbytes;
 
-	if (IN_PBL || !data || dma_mapping_error(dev, dma))
+	if (!data || dma_mapping_error(dev, dma))
 		return;
 
 	nbytes = data->blocks * data->blocksize;
-- 
2.43.0

[PATCH v2 11/11] mci: imx-esdhc: DMA in the i.MX8M PBL eMMC loader

[Johannes Schneider]

The PBL eMMC loader was hardwired to PIO. Add an opt-in SDMA/ADMA2 path:
a per-loader transfer-mode choice (PIO default / SDMA / ADMA2), route the
imx8m and imx8mp/imx8mn loaders through shared helpers, and retry in PIO if
the DMA attempt fails so a misconfigured board still boots. The PBL has no
mci and runs before BL31, so host.sdhci.version is set by hand and the
ADMA2 descriptor table lives in a static buffer.

On the imx8mp loader the DMA mode is set up before the bootpart EXT_CSD
read so that read also goes through ADMA: a PIO transfer preceding the
first ADMA transfer wedges the uSDHC ADMA engine.

Measured on i.MX8MM (MMU on): load_bl33 241 ms (PIO) -> 28 ms (ADMA2).
SDMA and ADMA2 perform identically (bus-bound); SDMA needs no descriptor
table, so it is the simpler default for boards that just want the speedup.

Assisted-by: Claude Opus 4.8 (1M context)
Signed-off-by: Johannes Schneider <johannes.schneider@leica-geosystems.com>
---

Unlike v1 [1], this runs with the MMU on: the loader already enables the
MMU before the load (warm-cache PIO), and the SDMA/ADMA2 modes select
PBL_HAS_DMA so the transfer gets real dma_map/dma_sync cache maintenance
and stays coherent with the D-cache -- which is the memory-synchronization
the reviewer noted the PBL was missing [2].

[1] https://lists.infradead.org/pipermail/barebox/2026-June/056843.html
[2] https://lists.infradead.org/pipermail/barebox/2026-June/056842.html

Notes:
    v2:
    - Was v1 7/7. Now runs with the MMU on: the SDMA/ADMA2 modes select
      PBL_HAS_DMA so the transfer stays coherent with the D-cache, rather
      than relying on the MMU being off. Folds in the transfer-mode Kconfig.
    - Still retries in PIO if the DMA attempt fails.
    
    - SDMA/ADMA2 now depends on HAS_DMA && MMU so the select PBL_HAS_DMA does
      not bypass its own deps (Copilot).
    - imx8mp loader re-reads the boot partition after falling back to PIO, so
      a failed-DMA EXT_CSD read no longer picks the wrong load offset (Copilot).
    - __esdhc_send_cmd() transfers in PIO when DMA setup left dma==SDHCI_NO_DMA
      (e.g. ADMA table build failed) instead of driving a DMA transfer (Copilot).

 drivers/mci/Kconfig            | 27 ++++++++++
 drivers/mci/imx-esdhc-common.c | 21 ++++++--
 drivers/mci/imx-esdhc-pbl.c    | 91 +++++++++++++++++++++++++++++++---
 3 files changed, 128 insertions(+), 11 deletions(-)

diff --git a/drivers/mci/Kconfig b/drivers/mci/Kconfig
index ebe771216f..d9dc241be9 100644
--- a/drivers/mci/Kconfig
+++ b/drivers/mci/Kconfig
@@ -296,6 +296,33 @@ config MCI_IMX_ESDHC_PBL
 	bool
 	select MCI_SDHCI
 
+choice
+	prompt "i.MX8M PBL eMMC loader transfer mode"
+	default MCI_IMX_ESDHC_PBL_PIO
+	depends on MCI_IMX_ESDHC_PBL
+	help
+	  How the i.MX8M PBL reads the next boot stage from eMMC. The MMU is
+	  enabled for the load either way (a warm D-cache makes even PIO fast);
+	  PIO needs no DMA plumbing and is the safe default. SDMA/ADMA2 are
+	  faster still and select PBL_HAS_DMA so their transfers stay coherent
+	  with the D-cache on; on failure the loader falls back to PIO so the
+	  board still boots.
+
+config MCI_IMX_ESDHC_PBL_PIO
+	bool "PIO"
+
+config MCI_IMX_ESDHC_PBL_SDMA
+	bool "SDMA"
+	depends on HAS_DMA && MMU
+	select PBL_HAS_DMA
+
+config MCI_IMX_ESDHC_PBL_ADMA2
+	bool "ADMA2"
+	depends on HAS_DMA && MMU
+	select PBL_HAS_DMA
+
+endchoice
+
 config MCI_ATMEL_PBL
 	bool
 	select MCI_ATMEL
diff --git a/drivers/mci/imx-esdhc-common.c b/drivers/mci/imx-esdhc-common.c
index a0c2dee32d..76b0ce3bff 100644
--- a/drivers/mci/imx-esdhc-common.c
+++ b/drivers/mci/imx-esdhc-common.c
@@ -295,9 +295,17 @@ void esdhc_populate_sdhci(struct fsl_esdhc_host *host)
 				      SDHCI_QUIRK_NO_ENDATTR_IN_NOPDESC;
 }
 
-static bool esdhc_use_pio_mode(void)
+static bool esdhc_use_pio_mode(struct fsl_esdhc_host *host)
 {
-	return IN_PBL || IS_ENABLED(CONFIG_MCI_IMX_ESDHC_PIO);
+	/*
+	 * In the PBL default to PIO, unless a caller explicitly opted into
+	 * ADMA2 (SDHCI_USE_ADMA, descriptor table set up) or SDMA
+	 * (SDHCI_USE_SDMA).
+	 */
+	if (IN_PBL)
+		return !(host->sdhci.flags & (SDHCI_USE_ADMA | SDHCI_USE_SDMA));
+
+	return IS_ENABLED(CONFIG_MCI_IMX_ESDHC_PIO);
 }
 
 static int esdhc_setup_data(struct fsl_esdhc_host *host, struct mci_data *data,
@@ -317,7 +325,7 @@ static int esdhc_setup_data(struct fsl_esdhc_host *host, struct mci_data *data,
 
 	host->sdhci.sdma_boundary = 0;
 
-	if (esdhc_use_pio_mode())
+	if (esdhc_use_pio_mode(host))
 		sdhci_setup_data_pio(&host->sdhci, data);
 	else
 		sdhci_setup_data_dma(&host->sdhci, data, dma);
@@ -411,7 +419,12 @@ int __esdhc_send_cmd(struct fsl_esdhc_host *host, struct mci_cmd *cmd,
 
 	/* Wait until all of the blocks are transferred */
 	if (data) {
-		if (esdhc_use_pio_mode())
+		/*
+		 * dma == SDHCI_NO_DMA means DMA setup failed (e.g. ADMA table
+		 * build); sdhci_setup_data_dma() has already programmed PIO, and
+		 * the command was issued without the DMA bit, so transfer in PIO.
+		 */
+		if (esdhc_use_pio_mode(host) || dma == SDHCI_NO_DMA)
 			ret = sdhci_transfer_data_pio(&host->sdhci, cmd, data);
 		else
 			ret = sdhci_transfer_data_dma(&host->sdhci, cmd, data, dma);
diff --git a/drivers/mci/imx-esdhc-pbl.c b/drivers/mci/imx-esdhc-pbl.c
index 2402b9aeaf..7ea826fab5 100644
--- a/drivers/mci/imx-esdhc-pbl.c
+++ b/drivers/mci/imx-esdhc-pbl.c
@@ -33,6 +33,12 @@
 
 static u8 ext_csd[512] __aligned(64);
 
+#ifdef CONFIG_MCI_IMX_ESDHC_PBL_ADMA2
+/* ADMA2 descriptor table for the PBL eMMC loader (the PBL has no allocator). */
+static u8 esdhc_adma_table[SDHCI_DEFAULT_ADMA_DESCS * SDHCI_ADMA2_32_DESC_SZ]
+	__aligned(8);
+#endif
+
 static int esdhc_send_ext_csd(struct fsl_esdhc_host *host)
 {
 	struct mci_cmd cmd = {};
@@ -255,6 +261,51 @@ int imx7_esdhc_start_image(int instance)
  * Return: If image successfully loaded, returns 0.
  * A negative error code is returned when this function fails.
  */
+/*
+ * Configure the PBL eMMC loader's DMA mode (ADMA2 or SDMA per Kconfig). The
+ * PBL has no mci, so host->sdhci.version is set by hand because
+ * __sdhci_read_caps() cannot run.
+ */
+static void imx8m_esdhc_pbl_setup_dma(struct fsl_esdhc_host *host)
+{
+	if (IS_ENABLED(CONFIG_MCI_IMX_ESDHC_PBL_SDMA)) {
+		host->sdhci.version = SDHCI_SPEC_300;
+		host->sdhci.flags |= SDHCI_USE_SDMA;
+	}
+
+#ifdef CONFIG_MCI_IMX_ESDHC_PBL_ADMA2
+	host->sdhci.version = SDHCI_SPEC_300;
+	host->sdhci.adma_table = esdhc_adma_table;
+	host->sdhci.adma_addr = virt_to_phys(esdhc_adma_table);
+	host->sdhci.desc_sz = SDHCI_ADMA2_32_DESC_SZ;
+	host->sdhci.adma_table_cnt = SDHCI_DEFAULT_ADMA_DESCS;
+	host->sdhci.adma_table_sz = sizeof(esdhc_adma_table);
+	host->sdhci.flags |= SDHCI_USE_ADMA;
+#endif
+}
+
+static int imx8m_esdhc_load_image_dma(struct fsl_esdhc_host *host,
+				      struct esdhc_soc_data *data, int instance,
+				      void *bl33, u32 offset, u32 ivt_offset)
+{
+	int ret;
+
+	ret = esdhc_load_image(host, MX8M_DDR_CSD1_BASE_ADDR,
+			       (ptrdiff_t)bl33, offset, ivt_offset, false);
+	if (!ret)
+		return 0;
+
+	/* Fall back to PIO so a failed DMA attempt still boots. */
+	if (host->sdhci.flags & (SDHCI_USE_ADMA | SDHCI_USE_SDMA)) {
+		host->sdhci.flags &= ~(SDHCI_USE_ADMA | SDHCI_USE_SDMA);
+		imx8m_esdhc_init(host, data, instance);
+		ret = esdhc_load_image(host, MX8M_DDR_CSD1_BASE_ADDR,
+				       (ptrdiff_t)bl33, offset, ivt_offset, false);
+	}
+
+	return ret;
+}
+
 int imx8m_esdhc_load_image(int instance, void *bl33)
 {
 	struct esdhc_soc_data data;
@@ -265,9 +316,10 @@ int imx8m_esdhc_load_image(int instance, void *bl33)
 	if (ret)
 		return ret;
 
-	return esdhc_load_image(&host, MX8M_DDR_CSD1_BASE_ADDR,
-				(ptrdiff_t)bl33, SZ_32K, SZ_1K,
-				false);
+	imx8m_esdhc_pbl_setup_dma(&host);
+
+	return imx8m_esdhc_load_image_dma(&host, &data, instance, bl33,
+					  SZ_32K, SZ_1K);
 }
 
 /**
@@ -283,21 +335,46 @@ int imx8m_esdhc_load_image(int instance, void *bl33)
  * Return: If image successfully loaded, returns 0.
  * A negative error code is returned when this function fails.
  */
+static int imx8mp_esdhc_try_load(struct fsl_esdhc_host *host, void *bl33)
+{
+	u32 offset = esdhc_bootpart_active(host) ? 0 : SZ_32K;
+
+	return esdhc_load_image(host, MX8M_DDR_CSD1_BASE_ADDR,
+				(ptrdiff_t)bl33, offset, 0, false);
+}
+
 int imx8mp_esdhc_load_image(int instance, void *bl33)
 {
 	struct esdhc_soc_data data;
 	struct fsl_esdhc_host host = { 0 };
-	u32 offset;
 	int ret;
 
 	ret = imx8m_esdhc_init(&host, &data, instance);
 	if (ret)
 		return ret;
 
-	offset = esdhc_bootpart_active(&host)? 0 : SZ_32K;
+	/*
+	 * Set up DMA first so the EXT_CSD read below also goes through ADMA --
+	 * a PIO transfer before the first ADMA transfer wedges the 8MP uSDHC.
+	 */
+	imx8m_esdhc_pbl_setup_dma(&host);
 
-	return esdhc_load_image(&host, MX8M_DDR_CSD1_BASE_ADDR,
-				(ptrdiff_t)bl33, offset, 0, false);
+	ret = imx8mp_esdhc_try_load(&host, bl33);
+	if (!ret)
+		return 0;
+
+	/*
+	 * Fall back to PIO so a failed DMA attempt still boots. Re-read the
+	 * boot partition in PIO too, so the load offset is chosen with the same
+	 * transfer mode that does the load.
+	 */
+	if (host.sdhci.flags & (SDHCI_USE_ADMA | SDHCI_USE_SDMA)) {
+		host.sdhci.flags &= ~(SDHCI_USE_ADMA | SDHCI_USE_SDMA);
+		imx8m_esdhc_init(&host, &data, instance);
+		ret = imx8mp_esdhc_try_load(&host, bl33);
+	}
+
+	return ret;
 }
 
 int imx8mn_esdhc_load_image(int instance, void *bl33)
-- 
2.43.0