RE: [PATCH v2] Porting barebox to a new SoC

[Lior Weintraub <liorw@pliops.com>]

Hi Ahmad,

I also thought that load address was suspicious (0x8000000) but then I saw the following comment on nxp_imx8mq_evk_start:
		/*
		 * On completion the TF-A will jump to MX8MQ_ATF_BL33_BASE_ADDR
		 * in EL2. Copy the image there, but replace the PBL part of
		 * that image with ourselves. On a high assurance boot only the
		 * currently running code is validated and contains the checksum
		 * for the piggy data, so we need to ensure that we are running
		 * the same code in DRAM.
		 */
Our code was based on this function and as a result it copies the barebox image into address 0x8000000 (which is our settings of ATF_BL33_BASE_ADDR).
The fact that Linux image is also copied there seemed reasonable (I might be wrong here) because once loading is done and we jump to Linux there is no need for barebox code anymore.

In any case, at that time when I thought it was wrong I also tried to run the bootm command with "-a 0" but I got exactly the same error:
barebox:/ bootm -o /dev/mtdrom0.oftree -r /dev/mtdrom0.rootfs -a 0 /dev/mtdrom0.kernel
getopt: optindex: 1 nonopts: 0 optind: 1
getopt: optindex: 1 nonopts: 0 optind: 3
getopt: optindex: 1 nonopts: 0 optind: 5
getopt: optindex: 1 nonopts: 0 optind: 7
mtdrom0.kernel: read ofs: 0x00000000 count: 0x00001000

Loading ARM aarch64 Linux image '/dev/mtdrom0.kernel'
header 00000000: 4d 5a 40 fa 3f 1a 23 14 00 00 00 00 00 00 00 00  MZ@.?.#.........
header 00000010: 00 00 b3 00 00 00 00 00 0a 00 00 00 00 00 00 00  ................
header 00000020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
header 00000030: 00 00 00 00 00 00 00 00 41 52 4d 64 40 00 00 00  ........ARMd@...
header 00000040: 50 45 00 00 64 aa 02 00 00 00 00 00 00 00 00 00  PE..d...........
booti: Kernel to be loaded to 0+b30000
__request_region ok: 0x00000000:0x0001ffff flags=0x200
mtdrom0.kernel: read ofs: 0x00000000 count: 0x00020000
__request_region ok: 0x00000000:0x0003ffff flags=0x200
mtdrom0.kernel: read ofs: 0x00020000 count: 0x00020000
__request_region ok: 0x00000000:0x0005ffff flags=0x200
.
.
mtdrom0.kernel: read ofs: 0x00fe0000 count: 0x00018000
__request_region ok: 0x00000000:0x00ff7fff flags=0x200
mtdrom0.rootfs: read ofs: 0x00000000 count: 0x00000800
__request_region: 0x00b30000:0x00b4ffff (image) conflicts with 0x00000000:0x00ff7fff (image)
unable to request SDRAM 0x00b30000-0x00b4ffff
handler failed with: Out of memory

 
The iomem before:
barebox:/ iomem
0x0000000000000000 - 0xffffffffffffffff (size 0x0000000000000000) iomem
  0x0000000000000000 - 0x000000001fffffff (size 0x0000000020000000) ram0
    0x00000000079fef00 - 0x0000000007dfeeff (size 0x0000000000400000) malloc space
    0x0000000007dfef00 - 0x0000000007dfffd1 (size 0x00000000000010d2) board data
    0x0000000007e00000 - 0x0000000007e3d95f (size 0x000000000003d960) barebox
    0x0000000007e3d960 - 0x0000000007e50bb7 (size 0x0000000000013258) barebox data
    0x0000000007e50bb8 - 0x0000000007e53b3f (size 0x0000000000002f88) bss
    0x0000000007ff0000 - 0x0000000007ff7fff (size 0x0000000000008000) stack
  0x0000000020000000 - 0x000000002fffffff (size 0x0000000010000000) 20000000.flash@0.of
  0x000000d000307000 - 0x000000d000307fff (size 0x0000000000001000) amba

After:
barebox:/ iomem
0x0000000000000000 - 0xffffffffffffffff (size 0x0000000000000000) iomem
  0x0000000000000000 - 0x000000001fffffff (size 0x0000000020000000) ram0
    0x00000000079fef00 - 0x0000000007dfeeff (size 0x0000000000400000) malloc space
    0x0000000007dfef00 - 0x0000000007dfffd1 (size 0x00000000000010d2) board data
    0x0000000007e00000 - 0x0000000007e3d95f (size 0x000000000003d960) barebox
    0x0000000007e3d960 - 0x0000000007e50bb7 (size 0x0000000000013258) barebox data
    0x0000000007e50bb8 - 0x0000000007e53b3f (size 0x0000000000002f88) bss
    0x0000000007ff0000 - 0x0000000007ff7fff (size 0x0000000000008000) stack
  0x0000000020000000 - 0x000000002fffffff (size 0x0000000010000000) 20000000.flash@0.of
  0x000000d000307000 - 0x000000d000307fff (size 0x0000000000001000) amba

I did not set global.bootm.image.loadaddr
Speaking of env, the last few lines on the barebox terminal are:
initcall-> load_environment+0x0/0x4c
loading defaultenv
environment load /dev/env0: No such file or directory
Maybe you have to create the partition.
initcalls done

When I run "printenv" I am getting:
barebox:/ printenv
locals:
globals:
bootsource=unknown
bootsource_instance=unknown
PATH=/env/bin

Hope this info can help.
Cheers,
Lior.


> -----Original Message-----
> From: Ahmad Fatoum <a.fatoum@pengutronix.de>
> Sent: Tuesday, June 13, 2023 3:51 PM
> To: Lior Weintraub <liorw@pliops.com>; Ahmad Fatoum <ahmad@a3f.at>;
> barebox@lists.infradead.org
> Subject: Re: [PATCH v2] Porting barebox to a new SoC
> 
> CAUTION: External Sender
> 
> On 13.06.23 14:39, Lior Weintraub wrote:
> > Hi Ahmad,
> >
> > As usual, your comments are spot on :-)
> >
> > After fixing the DT, devinfo command shows correct flash node:
> >    `-- 20000000.flash@0.of
> >       `-- mtdrom0
> >          `-- 0x00000000-0x0fffffff (   256 MiB): /dev/mtdrom0
> >          `-- mtdrom0.kernel
> >             `-- 0x00000000-0x00ff7fff (    16 MiB): /dev/mtdrom0.kernel
> >          `-- mtdrom0.oftree
> >             `-- 0x00000000-0x00007fff (    32 KiB): /dev/mtdrom0.oftree
> >          `-- mtdrom0.rootfs
> >             `-- 0x00000000-0x007fffff (     8 MiB): /dev/mtdrom0.rootfs
> >
> > As can be seen above, we chose to map this "flash" node on DRAM address
> 0x20000000 (offset 512MB).
> > The BL1 code correctly copy the 3 artifacts into those location (verified the
> content via "md -l -s /dev/mtdrom0.kernel").
> > Running "drvinfo" shows:
> > mtdram
> >         20000000.flash@0.of
> >
> > Now when I try to run "bootm -o /dev/mtdrom0.oftree -r
> /dev/mtdrom0.rootfs /dev/mtdrom0.kernel" I am getting this error:
> >
> > barebox:/ bootm -o /dev/mtdrom0.oftree -r /dev/mtdrom0.rootfs
> /dev/mtdrom0.kernel
> > getopt: optindex: 1 nonopts: 0 optind: 1
> > getopt: optindex: 1 nonopts: 0 optind: 3
> > getopt: optindex: 1 nonopts: 0 optind: 5
> > mtdrom0.kernel: read ofs: 0x00000000 count: 0x00001000
> >
> > Loading ARM aarch64 Linux image '/dev/mtdrom0.kernel'
> > header 00000000: 4d 5a 40 fa 3f 1a 23 14 00 00 00 00 00 00 00 00
> MZ@.?.#.........
> > header 00000010: 00 00 b3 00 00 00 00 00 0a 00 00 00 00 00 00 00
> ................
> > header 00000020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> ................
> > header 00000030: 00 00 00 00 00 00 00 00 41 52 4d 64 40 00 00 00
> ........ARMd@...
> > header 00000040: 50 45 00 00 64 aa 02 00 00 00 00 00 00 00 00 00
> PE..d...........
> > booti: Kernel to be loaded to 8000000+b30000
> 
> Kernel image is relocatable and ARM64 header in hexdump says that
> load offset is zero. Yet, barebox wants to load your image to
> 8000000, which is already reserved.
> 
> What's the output of the iomem command before and after the failed
> boot attempt?
> 
> > __request_region ok: 0x08000000:0x0801ffff flags=0x200
> > mtdrom0.kernel: read ofs: 0x00000000 count: 0x00020000
> > __request_region ok: 0x08000000:0x0803ffff flags=0x200
> > mtdrom0.kernel: read ofs: 0x00020000 count: 0x00020000
> > .
> > .
> > mtdrom0.kernel: read ofs: 0x00fe0000 count: 0x00018000
> > __request_region ok: 0x08000000:0x08ff7fff flags=0x200
> > mtdrom0.rootfs: read ofs: 0x00000000 count: 0x00000800
> > __request_region: 0x08b30000:0x08b4ffff (image) conflicts with
> 0x08000000:0x08ff7fff (image)
> > unable to request SDRAM 0x08b30000-0x08b4ffff
> > handler failed with: Out of memory
> 
> Just to be sure: You don't set global.bootm.image.loadaddr, right?
> 
> Cheers,
> Ahmad
> 
> >
> > Appreciate your advice,
> > Cheers,
> > Lior.
> >
> >
> >> -----Original Message-----
> >> From: Ahmad Fatoum <a.fatoum@pengutronix.de>
> >> Sent: Monday, June 12, 2023 6:08 PM
> >> To: Lior Weintraub <liorw@pliops.com>; Ahmad Fatoum <ahmad@a3f.at>;
> >> barebox@lists.infradead.org
> >> Subject: Re: [PATCH v2] Porting barebox to a new SoC
> >>
> >> CAUTION: External Sender
> >>
> >> Hi,
> >>
> >> On 12.06.23 16:59, Lior Weintraub wrote:
> >>> Hello Ahmad,
> >>>
> >>> Just for simple simulations we make the ROM size 192MB so it can include
> all
> >> needed artifacts.
> >>
> >> I am not convinced that this is much of a simplification over having
> >> your Qemu machine provide cfi-flash.
> >>
> >>> When we get this simple system to work we will move the relevant parts
> to
> >> BL2.
> >>
> >> Ok.
> >>
> >>> DRAM is also simulated now as SRAM so we are not worried about
> >> initializations.
> >>>
> >>> So if I understand correctly, we can decide that address 0x10000000 will
> be
> >> reserved for "flash" and add the following node:
> >>> flash@0 {
> >>
> >> flash@10000000, but that's just for informational purposes.
> >>
> >>>     compatible = "mtd-rom";
> >>>     probe-type = "map_rom";
> >>>     reg = <0x10000000 0x10000000>;
> >>>     bank-width = <4>;
> >>>     device-width = <1>;
> >>>
> >>>     #address-cells = <1>;
> >>>     #size-cells = <1>;
> >>>     kernel@0 {
> >>>         label = "kernel";
> >>>         reg = <0x0 0x01000000>;
> >>>     };
> >>>     rootfs@1000000 {
> >>>         label = "rootfs";
> >>>         reg = <0x01000000 0x00800000>;
> >>>     };
> >>> };
> >>>
> >>> When I use this node on our DT I see the following devinfo:
> >>> barebox:/ devinfo
> >>> `-- global
> >>> `-- nv
> >>> `-- platform
> >>>    `-- machine
> >>>    `-- psci.of
> >>>    `-- 1000000010000000.flash@0.of
> >>
> >> As you can see the address is wrong. That's because you have
> >> #address-cells = <2>;
> >> #size-cells = <2>;
> >>
> >> Yet, you wrote reg as if it was <1>/<1>. Change it to
> >>
> >>   reg = <0x0 0x10000000 0x0 0x10000000>;
> >>
> >> Remember all device tree integers are 32-bit big endian.
> >>
> >>>    `-- soc.of
> >>>       `-- c000000000.sram@c000000000.of
> >>>       `-- soc:pmu.of
> >>>       `-- soc:timer.of
> >>>       `-- e000000000.interrupt-controller@E000000000.of
> >>>    `-- mem0
> >>>       `-- 0x00000000-0x0fffffff (   256 MiB): /dev/ram0
> >>>    `-- mem1
> >>>       `-- 0x00000000-0xffffffffffffffff (   0 Bytes): /dev/mem
> >>> `-- amba
> >>>    `-- d000307000.serial@d000307000.of
> >>>       `-- cs0
> >>>          `-- 0x00000000-0xffffffffffffffff (   0 Bytes): /dev/cs0
> >>> `-- spi
> >>> `-- fs
> >>>    `-- ramfs0
> >>>    `-- devfs0
> >>>    `-- pstore0
> >>>
> >>> Not sure how to proceed from here...
> >>
> >> Type drvinfo command to see what drivers are bound to what devices.
> >> If you see no driver bound to your flash device, then maybe you need
> >> to enable the correct driver, that would be CONFIG_MTD_MTDRAM (which
> >> handles both read-only and read-write memory mapped MTD).
> >>
> >> Cheers,
> >> Ahmad
> >>
> >>> Cheers,
> >>> Lior.
> >>>
> >>>> -----Original Message-----
> >>>> From: Ahmad Fatoum <a.fatoum@pengutronix.de>
> >>>> Sent: Monday, June 12, 2023 3:29 PM
> >>>> To: Lior Weintraub <liorw@pliops.com>; Ahmad Fatoum
> <ahmad@a3f.at>;
> >>>> barebox@lists.infradead.org
> >>>> Subject: Re: [PATCH v2] Porting barebox to a new SoC
> >>>>
> >>>> CAUTION: External Sender
> >>>>
> >>>> Hello Lior,
> >>>>
> >>>> On 12.06.23 11:27, Lior Weintraub wrote:
> >>>>> Hi Ahmad,
> >>>>>
> >>>>> Regarding the rootfs and Linux run question:
> >>>>> Our board doesn't include eMMC\SD card.
> >>>>> We only have our NOR Flash to store the binaries and our BL1 code will
> >> make
> >>>> sure to copy those blobs into DRAM.
> >>>>
> >>>> How could BL1 copy artifacts in DRAM when BL2 first needs to set up
> >> DRAM?
> >>>>
> >>>>> The use of QEMU needs to be as simple as possible so no hidden virtio
> >>>> drivers and such because we would like to simulate the real HW flow.
> >>>>
> >>>> cfi-flash is just memory-mapped flash, which is the next simplest thing
> >>>> after BL1 copies stuff into (limited-to-4M) on-chip SRAM.
> >>>>
> >>>>> Our DTS file now includes the GIC, timer and arm,psci-1.0.
> >>>>> We have an initial build of Linux kernel Image (using buildroot) and a
> >>>> rootfs.cpio.xz.
> >>>>> I assume we can somehow reserve a portion of the DRAM to store
> those
> >>>> binaries and let barebox boot this Linux.
> >>>>
> >>>> You can make this work, but this is not how your actual system will
> >>>> look like and trying to make this work is harder than it needs to be.
> >>>>
> >>>> Just add a cfi-flash that corresponds to the Linux/rootfs flash in
> >>>> your actual system, then boot with bootm (type help bootm for info
> >>>> on how to use it). You don't need to hardcode the load addresses,
> >>>> barebox will determine them automatically.
> >>>>
> >>>>> Can you please advise how to make it work?
> >>>>
> >>>> For completion's sake, if you have 64M of RAM that's preloaded with
> >>>> boot images:
> >>>>
> >>>>   - Remove the 64M from the barebox /memory node. You can use a
> >> different
> >>>>     DT for kernel, but if you have memory that barebox shouldn't override,
> >>>>     you need to tell it that.
> >>>>
> >>>>   - Add a mtd-rom node that describes these 64M that you have. You can
> >>>>     add partitions for the region used for kernel and oftree
> >>>>
> >>>>   - boot with bootm -o /dev/mtdrom0.oftree -r /dev/mtdrom0.initrd \
> >>>>     /dev/mtdrom0.kernel
> >>>>
> >>>> That's admittedly cumbersome, but necessary, so barebox knows what
> >>>> memory
> >>>> it may use for itself and what memory may be used for boot.
> >>>>
> >>>> Correct solution is to use cfi-flash or similar.
> >>>>
> >>>> Cheers,
> >>>> Ahmad
> >>>> --
> >>>> Pengutronix e.K.                           |                             |
> >>>> Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
> >>>> 31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
> >>>> Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555
> |
> >>>
> >>
> >> --
> >> Pengutronix e.K.                           |                             |
> >> Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
> >> 31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
> >> Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |
> >
> 
> --
> Pengutronix e.K.                           |                             |
> Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
> 31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
> Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |