barebox state w/ qemu

barebox state w/ qemu

[Wes Chow]

I am trying to better understand barebox and playing with it via
Buildroot with Qemu. One thing I don't understand is how the state
backend is configured. I see this:

https://github.com/barebox/barebox/blob/d74c84582591ac1f93b203d733831fcf18e6b033/common/boards/qemu-virt/qemu-virt-flash.dtso#L4

..however I don't know how to read dts overlay files. Do I need to set
up a disk with a partition scheme that matches lines 17-30? Or is it
sufficient to just have a partition with the "barebox-state" label?

Also possibly of relevance, I'm starting qemu like so:

qemu-system-aarch64 -m 2048M -cpu cortex-a57 -machine virt -display
none -serial mon:stdio -kernel output/images/barebox-dt-2nd.img
-device virtio-blk-pci,drive=hd0,disable-legacy=on -drive
file=output/images/rootfs.ext2,if=none,format=raw,id=hd0

The rootfs.ext2 disk becomes available to barebox as /dev/virtioblk0.
How does barebox know which disk to search for the backend?

In the target image, I installed dt-utils and so have access to the
barebox-state command. How does the barebox-state executable know
where to look for the state backend? Is this passed down through the
device tree?

Thanks,
Wes

Re: barebox state w/ qemu

[Ahmad Fatoum]

Hello Wes,

On 08.07.24 04:22, Wes Chow wrote:
> I am trying to better understand barebox and playing with it via
> Buildroot with Qemu. One thing I don't understand is how the state
> backend is configured. I see this:
> 
> https://github.com/barebox/barebox/blob/d74c84582591ac1f93b203d733831fcf18e6b033/common/boards/qemu-virt/qemu-virt-flash.dtso#L4
> 
> ..however I don't know how to read dts overlay files.

The device trees and device tree overlays you see in the barebox source
tree are normally compiled _into_ barebox. Board code then references
them. For Qemu's state overlay, this is being done here:

  https://github.com/barebox/barebox/blob/d74c84582591ac1f93b203d733831fcf18e6b033/common/boards/qemu-virt/board.c#L76

Normally, the barebox device tree already describes the state and
environment, but Qemu is a special case, because the device tree is
not compiled into barebox, but supplied by the virtual machine.

Thus this overlay to add some barebox specific information. As this
should happen very early, so that the environment can take effect early
on, the overlays aren't applied dynamically in this case, but compiled-in.

> Do I need to set
> up a disk with a partition scheme that matches lines 17-30? Or is it
> sufficient to just have a partition with the "barebox-state" label?

"fixed-partitions" are partitions that have a fixed offset that isn't
of any on-disk (MBR, GPT, ...etc.) partitioning. So you don't need
to do anything special with your disk, just ensure you don't place
any data you don't want clobbered into the environment/state partition.

> Also possibly of relevance, I'm starting qemu like so:
> 
> qemu-system-aarch64 -m 2048M -cpu cortex-a57 -machine virt -display
> none -serial mon:stdio -kernel output/images/barebox-dt-2nd.img
> -device virtio-blk-pci,drive=hd0,disable-legacy=on -drive
> file=output/images/rootfs.ext2,if=none,format=raw,id=hd0
> 
> The rootfs.ext2 disk becomes available to barebox as /dev/virtioblk0.
> How does barebox know which disk to search for the backend?

This is described in the state device tree node. In the case of the Qemu
overlay, it's /state/backend-type = <&backend_state_flash>, which is
partition@3e00000 on the cfi-flash.

Because that flash is a fixed part of the virt machine, you should
already have functional state:

  barebox@ARM QEMU virt64:/ state
  registered state instances:
  state                (backend: raw, path: /dev/nor0.barebox-state)

Is this not the case and if not, what errors did you get?

If you got no errors at all, which defconfig did you use?
multi_v8_defconfig should have everything needed for this to just
work out of the box.

Placing the state on the Virt I/O disk is possible too in theory, but more
complicated, because there is no device tree nodes you can easily reference
in the DT. I'd suggest that you use the flash for state, even if your
rootfs is on a Virt I/O block device.

> In the target image, I installed dt-utils and so have access to the
> barebox-state command. How does the barebox-state executable know
> where to look for the state backend? Is this passed down through the
> device tree?

Yes. When the barebox state driver successfully probes, it takes care to
fix up into the kernel device tree the state description node, so dt-utils
can find it. If you are interested to see what fixups barebox is going to
apply, you can list them with (+ means take the previous arguments, but
fix it up):

  of_diff /mnt/virtioblk0/boot/my-devicetree.dtb +

Cheers,
Ahmad

> 
> Thanks,
> Wes
> 
> 

-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

Re: barebox state w/ qemu

[Wes Chow]

Ahmad,

> Normally, the barebox device tree already describes the state and
> environment, but Qemu is a special case, because the device tree is
> not compiled into barebox, but supplied by the virtual machine.
>
> Thus this overlay to add some barebox specific information. As this
> should happen very early, so that the environment can take effect early
> on, the overlays aren't applied dynamically in this case, but compiled-in.

Thanks, that's very helpful. I'm starting to understand but I'm not
quite there yet...

> Because that flash is a fixed part of the virt machine, you should
> already have functional state:
>
>   barebox@ARM QEMU virt64:/ state
>   registered state instances:
>   state                (backend: raw, path: /dev/nor0.barebox-state)
>
> Is this not the case and if not, what errors did you get?

I do have the same thing. If I'm reading the docs correctly though I
should be able to modify state with the nv command, right? This
doesn't seem to persist the foo variable:

 barebox@ARM QEMU virt64:/ state
 registered state instances:
 state                (backend: raw, path: /dev/nor0.barebox-state)
 barebox@ARM QEMU virt64:/ nv foo=bar
 nv variable modified, will save nv variables on shutdown
 barebox@ARM QEMU virt64:/ nv
   allow_color: true
   autoboot_timeout: 3
   foo: bar
   user: none
 barebox@ARM QEMU virt64:/ poweroff

When I restart Qemu the "foo" variable goes away. And actually, every
time I start Qemu I get this message:

 state state.of: Fresh state detected, continuing with defaults

Do I need to specify a flash device when starting Qemu? I'm running
this (taken from the docs):

 qemu-system-aarch64 -m 2048M -cpu cortex-a57 -machine virt -display
none -serial stdio -kernel ./images/barebox-dt-2nd.img

... but I'm guessing I'm missing a pflash argument. I've been trying
various incantations but haven't been able to get Qemu to boot
successfully if I supply a `-drive if=pflash,...`.

Thanks,
Wes

Re: barebox state w/ qemu

[Ahmad Fatoum]

Hello Wes,

On 09.07.24 04:13, Wes Chow wrote:
> Ahmad,
> 
>> Normally, the barebox device tree already describes the state and
>> environment, but Qemu is a special case, because the device tree is
>> not compiled into barebox, but supplied by the virtual machine.
>>
>> Thus this overlay to add some barebox specific information. As this
>> should happen very early, so that the environment can take effect early
>> on, the overlays aren't applied dynamically in this case, but compiled-in.
> 
> Thanks, that's very helpful. I'm starting to understand but I'm not
> quite there yet...

No problem. Once we have figured this out, maybe we should reword the parts
of the docs that were confusing.

>> Because that flash is a fixed part of the virt machine, you should
>> already have functional state:
>>
>>   barebox@ARM QEMU virt64:/ state
>>   registered state instances:
>>   state                (backend: raw, path: /dev/nor0.barebox-state)
>>
>> Is this not the case and if not, what errors did you get?
> 
> I do have the same thing. If I'm reading the docs correctly though I
> should be able to modify state with the nv command, right?

Not quite. As you see in the overlay, there are two partitions created
for barebox use: The environment partition and state.

The environment partition can contain anything: scripts, default values
for variables (nv), boot splashes, binaries ...etc.

The state partition however only contains the variables that you explicitly
define in your device tree state description. Unlike the environment,
these variables are laid out redundantly, so they are accessible in a
power-fail safe manner.

Systems with verified boot can thus elect to disable the mutable
environment completely and only keep the barebox built-in environment
while still being able to do A/B boot and such as that uses the
barebox state as backend, which is separate.

> This doesn't seem to persist the foo variable:
> 
>  barebox@ARM QEMU virt64:/ state
>  registered state instances:
>  state                (backend: raw, path: /dev/nor0.barebox-state)
>  barebox@ARM QEMU virt64:/ nv foo=bar
>  nv variable modified, will save nv variables on shutdown
>  barebox@ARM QEMU virt64:/ nv
>    allow_color: true
>    autoboot_timeout: 3
>    foo: bar
>    user: none
>  barebox@ARM QEMU virt64:/ poweroff

state variables can be set via device parameters, devinfo state
will tell you what variables you have and then you can set them like
this:

  state.bootstate.system0.priority=30

The variables have tab completion, so you can just state.<Tab> to list
them.

State is automatically saved when barebox exits, so you don't
need to call state -s manually to write your changes back.

> When I restart Qemu the "foo" variable goes away. And actually, every
> time I start Qemu I get this message:
> 
>  state state.of: Fresh state detected, continuing with defaults
> 
> Do I need to specify a flash device when starting Qemu? I'm running
> this (taken from the docs):
> 
>  qemu-system-aarch64 -m 2048M -cpu cortex-a57 -machine virt -display
> none -serial stdio -kernel ./images/barebox-dt-2nd.img

The assumption (that's probably not written down anywhere) is that the
user would use reset, not poweroff if they want state to persist.

Of course, on real boards, both are expected to work, but it's IMO an acceptable
tradeoff for emulation.

> ... but I'm guessing I'm missing a pflash argument. I've been trying
> various incantations but haven't been able to get Qemu to boot
> successfully if I supply a `-drive if=pflash,...`.

I think the problem is that QEMU expects you to place a "BIOS" (first stage
bootloader) into the pflash if you specify it. Unlike e.g. vexpress, barebox
for Qemu Virt64 only generates an image for use with -kernel.

You can use the second flash though (-drive if=pflash,unit=1) and change
the device tree overlay to point at.

There's surely neater ways to go about it. What are you trying to do?

Cheers,
Ahmad

> 
> Thanks,
> Wes
> 

-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

Re: barebox state w/ qemu

[Wes Chow]

>
> Not quite. As you see in the overlay, there are two partitions created
> for barebox use: The environment partition and state.

Ah, so I misunderstood the relationship between environment and state.
I had thought that state was the storage mechanism for persisting
things in the environment.


> State is automatically saved when barebox exits, so you don't
> need to call state -s manually to write your changes back.

Got it, yes I can see it working now.

>
> The assumption (that's probably not written down anywhere) is that the
> user would use reset, not poweroff if they want state to persist.

Ah that explains it -- no, I didn't see this in documentation
anywhere, but I'm a noob. I may write up a blog post or something on
what I've been learning.

> Of course, on real boards, both are expected to work, but it's IMO an acceptable
> tradeoff for emulation.

Got it.. that should work for my purposes. I was able to see that the
nv variables persisted after a reboot so that's great.

> > ... but I'm guessing I'm missing a pflash argument. I've been trying
> > various incantations but haven't been able to get Qemu to boot
> > successfully if I supply a `-drive if=pflash,...`.
>
> I think the problem is that QEMU expects you to place a "BIOS" (first stage
> bootloader) into the pflash if you specify it. Unlike e.g. vexpress, barebox
> for Qemu Virt64 only generates an image for use with -kernel.
>
> You can use the second flash though (-drive if=pflash,unit=1) and change
> the device tree overlay to point at.
>
> There's surely neater ways to go about it. What are you trying to do?

Thanks, that's good to know.. I don't think I need to go down this
route though. My high level goal is that I would like to have a
hands-on understanding of setting up an A/B boot strategy, and I
figured I'd learn via Qemu emulation before attempting on real
hardware. In a past defunct project we used Qemu emulation for a lot
of development and testing and then deployed onto Raspberry Pi CM3s,
so I just naturally reached for Qemu again for this personal project.
Ultimately I want to do this with real hardware, likely Raspberry Pis
or something similar.

After I set state:

 barebox@ARM QEMU virt64:/ state.bootstate.system0.priority=30

How do I see that that's been reflected in the state? Is there a way
for me to dump out all state values?

In the target, I'm getting an error with barebox-state:

 # barebox-state -d
 state: state failed to parse path to backend: No such device
 unable to initialize state: No such device

Do I need to configure barebox-state somehow? Perhaps the Linux kernel
can't see the flash?

Thanks,
Wes

Re: barebox state w/ qemu

[Wes Chow]

> In the target, I'm getting an error with barebox-state:
>
>  # barebox-state -d
>  state: state failed to parse path to backend: No such device
>  unable to initialize state: No such device
>
> Do I need to configure barebox-state somehow? Perhaps the Linux kernel
> can't see the flash?

I discovered the problem.. I'm using Buildroot with the
qemu_aarch64_defconfig and I needed to add these Linux kernel config
options:

CONFIG_MTD=y
CONFIG_MTD_CFI=y
CONFIG_MTD_OF_PARTS=y
CONFIG_MTD_CFI=y
CONFIG_MTD_CFI_INTELEXT=y
CONFIG_MTD_CFI_UTIL=y
CONFIG_MTD_PHYSMAP=y
CONFIG_MTD_PHYSMAP_OF=y

I can successfully run barebox-state on the target, dump, and set
state variables. I'll submit a patch to Buildroot.

Wes