Barebox PBL with uncompressed barebox proper

Barebox PBL with uncompressed barebox proper

[Lior Weintraub]

Hi,

Is there as option to build a barebox PBL with uncompressed barebox proper?
The incentive for that is to accelerate RTL simulations.

Cheers,
Lior.  

Re: Barebox PBL with uncompressed barebox proper

[Sascha Hauer]

On Wed, Aug 16, 2023 at 10:36:16AM +0000, Lior Weintraub wrote:
> Hi,
> 
> Is there as option to build a barebox PBL with uncompressed barebox proper?
> The incentive for that is to accelerate RTL simulations.

Yes, just enable CONFIG_IMAGE_COMPRESSION_NONE.

Sascha

-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

RE: Barebox PBL with uncompressed barebox proper

[Lior Weintraub]

Thanks Sascha!

Before applying the recommended change the trace showed:
uncompress.c: memory at 0xc000000000, size 0x00300000
uncompress.c: uncompressing barebox binary at 0x000000c000002b60 (size 0x00030def) to 0xc000100000 (uncompressed size: 0x0005a9a0)
uncompress.c: jumping to uncompressed image at 0x000000c000100000

After applying this configuration, the .img file was increased (as expected) and the trace shows:
uncompress.c: memory at 0xc000000000, size 0x00300000
uncompress.c: uncompressing barebox binary at 0x000000c000002480 (size 0x0005a9a4) to 0xc000100000 (uncompressed size: 0x0005a9a0)
uncompress.c: jumping to uncompressed image at 0x000000c000100000

Indeed is seems link an uncompressed image because the sizes of the "compressed" match to the uncompress (well except 4 bytes which probably indicate the image size or the compression type (just a guess)).

I assume that the decompress function detects the header and know that it is an uncompressed image and then just copy it to another location (in my case 0xc000100000).

Can we avoid this step?
Since the image was loaded into SRAM we wish to run locally without the extra relocation (which also takes simulation time).

Cheers,
Lior.



> -----Original Message-----
> From: Sascha Hauer <s.hauer@pengutronix.de>
> Sent: Wednesday, August 16, 2023 1:44 PM
> To: Lior Weintraub <liorw@pliops.com>
> Cc: barebox@lists.infradead.org; Ahmad Fatoum
> <a.fatoum@pengutronix.de>
> Subject: Re: Barebox PBL with uncompressed barebox proper
> 
> CAUTION: External Sender
> 
> On Wed, Aug 16, 2023 at 10:36:16AM +0000, Lior Weintraub wrote:
> > Hi,
> >
> > Is there as option to build a barebox PBL with uncompressed barebox
> proper?
> > The incentive for that is to accelerate RTL simulations.
> 
> Yes, just enable CONFIG_IMAGE_COMPRESSION_NONE.
> 
> Sascha
> 
> --
> Pengutronix e.K.                           |                             |
> Steuerwalder Str. 21                       | http://secure-
> web.cisco.com/1UnbNIgGun7ZeK6m1EHXDowsycba0F_lMTYur4_RVJsp4rB1
> ereLHOgvggoEbLoWjLGwrPExhSzzQ8_G3V0h3ExWVvBuKIKqI0Os4LQS49bsp
> VjE-
> CO4T3S2ZjTMOr6Zn4i26fhtU5aEQyC8IVbO5bS9GaPkn7N7HCoMkRdp9iJEN
> 1yyziAlcWL6E3-HooXhzCvJNDgJVGS6Cwtu0RJS7Q9HRM0wWC5maq1_-
> Th0JtrOdGEfrEhRM0eunMSxkkz28iCJx83LpDEt_uOnC9j--
> S7DjsTGkGApcwPIRy1Dml3Ebg1BkbYQ0UQZqxwJcIPKK/http%3A%2F%2Fww
> w.pengutronix.de%2F  |
> 31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
> Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

Re: Barebox PBL with uncompressed barebox proper

[Sascha Hauer]

On Wed, Aug 16, 2023 at 11:23:26AM +0000, Lior Weintraub wrote:
> Thanks Sascha!
> 
> Before applying the recommended change the trace showed:
> uncompress.c: memory at 0xc000000000, size 0x00300000
> uncompress.c: uncompressing barebox binary at 0x000000c000002b60 (size 0x00030def) to 0xc000100000 (uncompressed size: 0x0005a9a0)
> uncompress.c: jumping to uncompressed image at 0x000000c000100000
> 
> After applying this configuration, the .img file was increased (as expected) and the trace shows:
> uncompress.c: memory at 0xc000000000, size 0x00300000
> uncompress.c: uncompressing barebox binary at 0x000000c000002480 (size 0x0005a9a4) to 0xc000100000 (uncompressed size: 0x0005a9a0)
> uncompress.c: jumping to uncompressed image at 0x000000c000100000
> 
> Indeed is seems link an uncompressed image because the sizes of the
> "compressed" match to the uncompress (well except 4 bytes which
> probably indicate the image size or the compression type (just a
> guess)).
> 
> I assume that the decompress function detects the header and know that
> it is an uncompressed image and then just copy it to another location
> (in my case 0xc000100000).
> 
> Can we avoid this step?
> Since the image was loaded into SRAM we wish to run locally without
> the extra relocation (which also takes simulation time).

I don't think this is easily possible, at least not in an upstreamable
way. Normally barebox puts itself at the end of available RAM and puts
the malloc space directly beneath it.

What you describe here seems to be a very special purpose barebox. What
you could do is to disable PBL support and only build a barebox proper.
Then add your own entry point and jump to start_barebox() from there.
You'll need to copy/adjust the useful things from
barebox_non_pbl_start() as well.

I am not sure what you are trying to archieve here, because copying the
binary usually takes time in the order of milliseconds and that is
normally not a problem.

Sascha


-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

RE: Barebox PBL with uncompressed barebox proper

[Lior Weintraub]

Thanks Sascha,

The thing I am trying to achieve is to run RTL simulations in a reasonable time frame on our bootloader and barebox loading.
On RTL simulation environment it takes about 5 min for each 40 us which is about 2 hours for 1 ms of CPU time.
When the original code with the relocation and decompression was tested it was still in the middle of decompression after 2 hole days of simulation.
I managed to patch the code to avoid the decompression (which is actually a memcpy in this case) and it seems to work fine (tested on QEMU).
The patch I used:

diff --git a/arch/arm/cpu/uncompress.c b/arch/arm/cpu/uncompress.c
index a481c4634d..cde03dc0c1 100644
--- a/arch/arm/cpu/uncompress.c
+++ b/arch/arm/cpu/uncompress.c
@@ -71,9 +71,14 @@ void __noreturn barebox_pbl_start(unsigned long membase, unsigned long memsize,
 	pg_len = pg_end - pg_start;
 	uncompressed_len = get_unaligned((const u32 *)(pg_start + pg_len - 4));
 
-	if (IS_ENABLED(CONFIG_RELOCATABLE))
+	if (IS_ENABLED(CONFIG_RELOCATABLE)) {
+#ifdef CONFIG_IMAGE_COMPRESSION_NONE
+        barebox_base = (void *)pg_start;
+#else
 		barebox_base = arm_mem_barebox_image(membase, endmem,
 						     uncompressed_len + MAX_BSS_SIZE);
+#endif
+    }
 	else
 		barebox_base = TEXT_BASE;
 
@@ -87,12 +92,14 @@ void __noreturn barebox_pbl_start(unsigned long membase, unsigned long memsize,
 	free_mem_ptr = arm_mem_early_malloc(endmem);
 	free_mem_end_ptr = arm_mem_early_malloc_end(endmem);
 
+#ifndef CONFIG_IMAGE_COMPRESSION_NONE
 	pr_debug("uncompressing barebox binary at 0x%p (size 0x%08x) to 0x%08lx (uncompressed size: 0x%08x)\n",
 			pg_start, pg_len, barebox_base, uncompressed_len);
 
 	pbl_barebox_uncompress((void*)barebox_base, pg_start, pg_len);
 
 	sync_caches_for_execution();
+#endif
 
 	if (IS_ENABLED(CONFIG_THUMB2_BAREBOX))
 		barebox = (void *)(barebox_base + 1);


Your call if that makes sense to include in the upstream or not :-)
Thanks again for your kind support,
Cheers,
Lior.

> -----Original Message-----
> From: Sascha Hauer <s.hauer@pengutronix.de>
> Sent: Wednesday, August 16, 2023 5:28 PM
> To: Lior Weintraub <liorw@pliops.com>
> Cc: barebox@lists.infradead.org; Ahmad Fatoum
> <a.fatoum@pengutronix.de>
> Subject: Re: Barebox PBL with uncompressed barebox proper
> 
> CAUTION: External Sender
> 
> On Wed, Aug 16, 2023 at 11:23:26AM +0000, Lior Weintraub wrote:
> > Thanks Sascha!
> >
> > Before applying the recommended change the trace showed:
> > uncompress.c: memory at 0xc000000000, size 0x00300000
> > uncompress.c: uncompressing barebox binary at 0x000000c000002b60
> (size 0x00030def) to 0xc000100000 (uncompressed size: 0x0005a9a0)
> > uncompress.c: jumping to uncompressed image at 0x000000c000100000
> >
> > After applying this configuration, the .img file was increased (as expected)
> and the trace shows:
> > uncompress.c: memory at 0xc000000000, size 0x00300000
> > uncompress.c: uncompressing barebox binary at 0x000000c000002480
> (size 0x0005a9a4) to 0xc000100000 (uncompressed size: 0x0005a9a0)
> > uncompress.c: jumping to uncompressed image at 0x000000c000100000
> >
> > Indeed is seems link an uncompressed image because the sizes of the
> > "compressed" match to the uncompress (well except 4 bytes which
> > probably indicate the image size or the compression type (just a
> > guess)).
> >
> > I assume that the decompress function detects the header and know that
> > it is an uncompressed image and then just copy it to another location
> > (in my case 0xc000100000).
> >
> > Can we avoid this step?
> > Since the image was loaded into SRAM we wish to run locally without
> > the extra relocation (which also takes simulation time).
> 
> I don't think this is easily possible, at least not in an upstreamable
> way. Normally barebox puts itself at the end of available RAM and puts
> the malloc space directly beneath it.
> 
> What you describe here seems to be a very special purpose barebox. What
> you could do is to disable PBL support and only build a barebox proper.
> Then add your own entry point and jump to start_barebox() from there.
> You'll need to copy/adjust the useful things from
> barebox_non_pbl_start() as well.
> 
> I am not sure what you are trying to archieve here, because copying the
> binary usually takes time in the order of milliseconds and that is
> normally not a problem.
> 
> Sascha
> 
> 
> --
> Pengutronix e.K.                           |                             |
> Steuerwalder Str. 21                       | http://secure-
> web.cisco.com/1XpYzNc191JjtGaWh_8mPulQQYcyNjNxrljKOy4EXP2rJ6BI69y
> LQZalS_CIiCjQ0DKAyT3wD0pqV_fDARkSkZJR-
> QINkixMViIHKcS0xPNjMFyt4edli64apIWuOxcO_28UOr2R8MQOuS7ZeOhXjm
> fpSK85jwv7AhOTHiCnokRyVhHNWfbb1o05jLvnoR6zBmZ8U5MiQ5cwXYnEq
> w4fYe6wBKoBmerRBThjYiwC7_6_5LUEtO3BFAF-
> brnGnHBd9opXwB2Lq4kXhbQ4HLNnL1ZiAuKMKm7EYhvbN3y3BML9T-
> mz5ZvxUqx6VzqE8FAGv/http%3A%2F%2Fwww.pengutronix.de%2F  |
> 31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
> Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

RE: Barebox PBL with uncompressed barebox proper

[Lior Weintraub]

Sorry. Used wrong casing on barebox_base assignment.
This is the correct patch:

diff --git a/arch/arm/cpu/uncompress.c b/arch/arm/cpu/uncompress.c
index a481c4634d..c554143acc 100644
--- a/arch/arm/cpu/uncompress.c
+++ b/arch/arm/cpu/uncompress.c
@@ -71,9 +71,14 @@ void __noreturn barebox_pbl_start(unsigned long membase, unsigned long memsize,
 	pg_len = pg_end - pg_start;
 	uncompressed_len = get_unaligned((const u32 *)(pg_start + pg_len - 4));
 
-	if (IS_ENABLED(CONFIG_RELOCATABLE))
+	if (IS_ENABLED(CONFIG_RELOCATABLE)) {
+#ifdef CONFIG_IMAGE_COMPRESSION_NONE
+        barebox_base = (unsigned long)pg_start;
+#else
 		barebox_base = arm_mem_barebox_image(membase, endmem,
 						     uncompressed_len + MAX_BSS_SIZE);
+#endif
+    }
 	else
 		barebox_base = TEXT_BASE;
 
@@ -87,12 +92,14 @@ void __noreturn barebox_pbl_start(unsigned long membase, unsigned long memsize,
 	free_mem_ptr = arm_mem_early_malloc(endmem);
 	free_mem_end_ptr = arm_mem_early_malloc_end(endmem);
 
+#ifndef CONFIG_IMAGE_COMPRESSION_NONE
 	pr_debug("uncompressing barebox binary at 0x%p (size 0x%08x) to 0x%08lx (uncompressed size: 0x%08x)\n",
 			pg_start, pg_len, barebox_base, uncompressed_len);
 
 	pbl_barebox_uncompress((void*)barebox_base, pg_start, pg_len);
 
 	sync_caches_for_execution();
+#endif
 
 	if (IS_ENABLED(CONFIG_THUMB2_BAREBOX))
 		barebox = (void *)(barebox_base + 1);



> -----Original Message-----
> From: Lior Weintraub
> Sent: Wednesday, August 16, 2023 7:49 PM
> To: Sascha Hauer <s.hauer@pengutronix.de>
> Cc: barebox@lists.infradead.org; Ahmad Fatoum
> <a.fatoum@pengutronix.de>
> Subject: RE: Barebox PBL with uncompressed barebox proper
> 
> Thanks Sascha,
> 
> The thing I am trying to achieve is to run RTL simulations in a reasonable time
> frame on our bootloader and barebox loading.
> On RTL simulation environment it takes about 5 min for each 40 us which is
> about 2 hours for 1 ms of CPU time.
> When the original code with the relocation and decompression was tested it
> was still in the middle of decompression after 2 hole days of simulation.
> I managed to patch the code to avoid the decompression (which is actually a
> memcpy in this case) and it seems to work fine (tested on QEMU).
> The patch I used:
> 
> diff --git a/arch/arm/cpu/uncompress.c b/arch/arm/cpu/uncompress.c
> index a481c4634d..cde03dc0c1 100644
> --- a/arch/arm/cpu/uncompress.c
> +++ b/arch/arm/cpu/uncompress.c
> @@ -71,9 +71,14 @@ void __noreturn barebox_pbl_start(unsigned long
> membase, unsigned long memsize,
>  	pg_len = pg_end - pg_start;
>  	uncompressed_len = get_unaligned((const u32 *)(pg_start + pg_len -
> 4));
> 
> -	if (IS_ENABLED(CONFIG_RELOCATABLE))
> +	if (IS_ENABLED(CONFIG_RELOCATABLE)) {
> +#ifdef CONFIG_IMAGE_COMPRESSION_NONE
> +        barebox_base = (void *)pg_start;
> +#else
>  		barebox_base = arm_mem_barebox_image(membase,
> endmem,
>  						     uncompressed_len +
> MAX_BSS_SIZE);
> +#endif
> +    }
>  	else
>  		barebox_base = TEXT_BASE;
> 
> @@ -87,12 +92,14 @@ void __noreturn barebox_pbl_start(unsigned long
> membase, unsigned long memsize,
>  	free_mem_ptr = arm_mem_early_malloc(endmem);
>  	free_mem_end_ptr = arm_mem_early_malloc_end(endmem);
> 
> +#ifndef CONFIG_IMAGE_COMPRESSION_NONE
>  	pr_debug("uncompressing barebox binary at 0x%p (size 0x%08x) to
> 0x%08lx (uncompressed size: 0x%08x)\n",
>  			pg_start, pg_len, barebox_base, uncompressed_len);
> 
>  	pbl_barebox_uncompress((void*)barebox_base, pg_start, pg_len);
> 
>  	sync_caches_for_execution();
> +#endif
> 
>  	if (IS_ENABLED(CONFIG_THUMB2_BAREBOX))
>  		barebox = (void *)(barebox_base + 1);
> 
> 
> Your call if that makes sense to include in the upstream or not :-)
> Thanks again for your kind support,
> Cheers,
> Lior.
> 
> > -----Original Message-----
> > From: Sascha Hauer <s.hauer@pengutronix.de>
> > Sent: Wednesday, August 16, 2023 5:28 PM
> > To: Lior Weintraub <liorw@pliops.com>
> > Cc: barebox@lists.infradead.org; Ahmad Fatoum
> > <a.fatoum@pengutronix.de>
> > Subject: Re: Barebox PBL with uncompressed barebox proper
> >
> > CAUTION: External Sender
> >
> > On Wed, Aug 16, 2023 at 11:23:26AM +0000, Lior Weintraub wrote:
> > > Thanks Sascha!
> > >
> > > Before applying the recommended change the trace showed:
> > > uncompress.c: memory at 0xc000000000, size 0x00300000
> > > uncompress.c: uncompressing barebox binary at 0x000000c000002b60
> > (size 0x00030def) to 0xc000100000 (uncompressed size: 0x0005a9a0)
> > > uncompress.c: jumping to uncompressed image at 0x000000c000100000
> > >
> > > After applying this configuration, the .img file was increased (as expected)
> > and the trace shows:
> > > uncompress.c: memory at 0xc000000000, size 0x00300000
> > > uncompress.c: uncompressing barebox binary at 0x000000c000002480
> > (size 0x0005a9a4) to 0xc000100000 (uncompressed size: 0x0005a9a0)
> > > uncompress.c: jumping to uncompressed image at 0x000000c000100000
> > >
> > > Indeed is seems link an uncompressed image because the sizes of the
> > > "compressed" match to the uncompress (well except 4 bytes which
> > > probably indicate the image size or the compression type (just a
> > > guess)).
> > >
> > > I assume that the decompress function detects the header and know that
> > > it is an uncompressed image and then just copy it to another location
> > > (in my case 0xc000100000).
> > >
> > > Can we avoid this step?
> > > Since the image was loaded into SRAM we wish to run locally without
> > > the extra relocation (which also takes simulation time).
> >
> > I don't think this is easily possible, at least not in an upstreamable
> > way. Normally barebox puts itself at the end of available RAM and puts
> > the malloc space directly beneath it.
> >
> > What you describe here seems to be a very special purpose barebox. What
> > you could do is to disable PBL support and only build a barebox proper.
> > Then add your own entry point and jump to start_barebox() from there.
> > You'll need to copy/adjust the useful things from
> > barebox_non_pbl_start() as well.
> >
> > I am not sure what you are trying to archieve here, because copying the
> > binary usually takes time in the order of milliseconds and that is
> > normally not a problem.
> >
> > Sascha
> >
> >
> > --
> > Pengutronix e.K.                           |                             |
> > Steuerwalder Str. 21                       | http://secure-
> >
> web.cisco.com/1XpYzNc191JjtGaWh_8mPulQQYcyNjNxrljKOy4EXP2rJ6BI69y
> > LQZalS_CIiCjQ0DKAyT3wD0pqV_fDARkSkZJR-
> >
> QINkixMViIHKcS0xPNjMFyt4edli64apIWuOxcO_28UOr2R8MQOuS7ZeOhXjm
> >
> fpSK85jwv7AhOTHiCnokRyVhHNWfbb1o05jLvnoR6zBmZ8U5MiQ5cwXYnEq
> > w4fYe6wBKoBmerRBThjYiwC7_6_5LUEtO3BFAF-
> > brnGnHBd9opXwB2Lq4kXhbQ4HLNnL1ZiAuKMKm7EYhvbN3y3BML9T-
> > mz5ZvxUqx6VzqE8FAGv/http%3A%2F%2Fwww.pengutronix.de%2F  |
> > 31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
> > Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

Re: Barebox PBL with uncompressed barebox proper

[Sascha Hauer]

On Wed, Aug 16, 2023 at 04:48:37PM +0000, Lior Weintraub wrote:
> Thanks Sascha,
> 
> The thing I am trying to achieve is to run RTL simulations in a
> reasonable time frame on our bootloader and barebox loading.
> On RTL simulation environment it takes about 5 min for each 40 us
> which is about 2 hours for 1 ms of CPU time.

Ah, I see, now it makes sense to me ;)

In that case I would really just skip PBL support and jump into barebox
proper right away like I described.

It doesn't make sense to upstream such hacks though, it will be
bitrotted the next time somebody tries to use it.

Sascha

-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |