Devicetree Maintenance in barebox

Devicetree Maintenance in barebox

[Sascha Hauer]

Hi All,

It's becoming more obvious that devicetree maintenance is painful
because we have to sync them to the kernel regularly. My hope was that
this would get simpler once the devicetrees get their own repository
outside the kernel, but it seems that won't happen anytime soon.

So my current idea to continue with barebox devicetrees is:

- Maintain a kernel branch which has all devicetree changes we need in
  barebox in a clean step-by-step series
- rebase this branch regularly on the newer kernel
- Copy the resulting devicetrees to barebox

The upside is that we have up to date devicetrees in barebox without
having to resync them by hand on a per SoC basis.  Of course this also
means that we lose the devicetree history and breakage may be introduced
with some huge commits saying "Update devicetrees to Linux-3.x".

Any better ideas? I think we have to do something.

Sascha

-- 
Pengutronix e.K.                           |                             |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox

Re: Devicetree Maintenance in barebox

[Alexander Shiyan]

Hello.

Пятница,  7 февраля 2014, 8:13 +01:00 от Sascha Hauer <s.hauer@pengutronix.de>:
> Hi All,
> 
> It's becoming more obvious that devicetree maintenance is painful
> because we have to sync them to the kernel regularly. My hope was that
> this would get simpler once the devicetrees get their own repository
> outside the kernel, but it seems that won't happen anytime soon.
> 
> So my current idea to continue with barebox devicetrees is:
> 
> - Maintain a kernel branch which has all devicetree changes we need in
> barebox in a clean step-by-step series
> - rebase this branch regularly on the newer kernel
> - Copy the resulting devicetrees to barebox
> 
> The upside is that we have up to date devicetrees in barebox without
> having to resync them by hand on a per SoC basis.  Of course this also
> means that we lose the devicetree history and breakage may be introduced
> with some huge commits saying "Update devicetrees to Linux-3.x".
> 
> Any better ideas? I think we have to do something.

As far as I know, in the community have any thoughts on the transfer of the DT data
in a separate project, but most likely it will not be soon ...

At the moment, I want to suggest to use only non-modified DT files, i.e. barebox
should use to build their own files, which include the original data from the kernel.

Just for example:
# ls
bb_<board.dts>
<board.dts>

# cat bb_<board.dts>

#include "<board.dts>"
/* All overrides here */
chosen {
  ...
};

---
_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox

Re: Devicetree Maintenance in barebox

[Jean-Christophe PLAGNIOL-VILLARD]

On 08:13 Fri 07 Feb     , Sascha Hauer wrote:
> Hi All,
> 
> It's becoming more obvious that devicetree maintenance is painful
> because we have to sync them to the kernel regularly. My hope was that
> this would get simpler once the devicetrees get their own repository
> outside the kernel, but it seems that won't happen anytime soon.
> 
> So my current idea to continue with barebox devicetrees is:
> 
> - Maintain a kernel branch which has all devicetree changes we need in
>   barebox in a clean step-by-step series
> - rebase this branch regularly on the newer kernel
> - Copy the resulting devicetrees to barebox
> 
> The upside is that we have up to date devicetrees in barebox without
> having to resync them by hand on a per SoC basis.  Of course this also
> means that we lose the devicetree history and breakage may be introduced
> with some huge commits saying "Update devicetrees to Linux-3.x".
> 
> Any better ideas? I think we have to do something.

push the kernel to split the tree as a sub module at least

so other project can import it

Best Regards,
J.
> 
> Sascha
> 
> -- 
> Pengutronix e.K.                           |                             |
> Industrial Linux Solutions                 | http://www.pengutronix.de/  |
> Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
> Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |
> 
> _______________________________________________
> barebox mailing list
> barebox@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/barebox

_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox

Re: Devicetree Maintenance in barebox

[Sascha Hauer]

On Fri, Feb 07, 2014 at 11:39:33AM +0400, Alexander Shiyan wrote:
> Hello.
> 
> Пятница,  7 февраля 2014, 8:13 +01:00 от Sascha Hauer <s.hauer@pengutronix.de>:
> > Hi All,
> > 
> > It's becoming more obvious that devicetree maintenance is painful
> > because we have to sync them to the kernel regularly. My hope was that
> > this would get simpler once the devicetrees get their own repository
> > outside the kernel, but it seems that won't happen anytime soon.
> > 
> > So my current idea to continue with barebox devicetrees is:
> > 
> > - Maintain a kernel branch which has all devicetree changes we need in
> > barebox in a clean step-by-step series
> > - rebase this branch regularly on the newer kernel
> > - Copy the resulting devicetrees to barebox
> > 
> > The upside is that we have up to date devicetrees in barebox without
> > having to resync them by hand on a per SoC basis.  Of course this also
> > means that we lose the devicetree history and breakage may be introduced
> > with some huge commits saying "Update devicetrees to Linux-3.x".
> > 
> > Any better ideas? I think we have to do something.
> 
> As far as I know, in the community have any thoughts on the transfer of the DT data
> in a separate project, but most likely it will not be soon ...

Yeah, I also don't think this will happen soon. Even if it does I think
this won't solve all problems. We will still need barebox specific
adjustments.

> 
> At the moment, I want to suggest to use only non-modified DT files, i.e. barebox
> should use to build their own files, which include the original data from the kernel.
> 
> Just for example:
> # ls
> bb_<board.dts>
> <board.dts>
> 
> # cat bb_<board.dts>
> 
> #include "<board.dts>"
> /* All overrides here */
> chosen {
>   ...
> };

I like the idea. One disadvantage may be that the structure in the dts
directory gets more unclear, but I guess we can't have everything.

Sascha

-- 
Pengutronix e.K.                           |                             |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox

Re: Devicetree Maintenance in barebox

[Alexander Shiyan]

Пятница,  7 февраля 2014, 9:47 +01:00 от Sascha Hauer <s.hauer@pengutronix.de>:
> On Fri, Feb 07, 2014 at 11:39:33AM +0400, Alexander Shiyan wrote:
> > Hello.
> > 
> > Пятница,  7 февраля 2014, 8:13 +01:00 от Sascha Hauer
> <s.hauer@pengutronix.de>:
> > > Hi All,
> > > 
> > > It's becoming more obvious that devicetree maintenance is painful
> > > because we have to sync them to the kernel regularly. My hope was that
> > > this would get simpler once the devicetrees get their own repository
> > > outside the kernel, but it seems that won't happen anytime soon.
> > > 
> > > So my current idea to continue with barebox devicetrees is:
> > > 
> > > - Maintain a kernel branch which has all devicetree changes we need in
> > > barebox in a clean step-by-step series
> > > - rebase this branch regularly on the newer kernel
> > > - Copy the resulting devicetrees to barebox
> > > 
> > > The upside is that we have up to date devicetrees in barebox without
> > > having to resync them by hand on a per SoC basis.  Of course this also
> > > means that we lose the devicetree history and breakage may be introduced
> > > with some huge commits saying "Update devicetrees to Linux-3.x".
> > > 
> > > Any better ideas? I think we have to do something.
> > 
> > As far as I know, in the community have any thoughts on the transfer of the
> DT data
> > in a separate project, but most likely it will not be soon ...
> 
> Yeah, I also don't think this will happen soon. Even if it does I think
> this won't solve all problems. We will still need barebox specific
> adjustments.
> 
> > 
> > At the moment, I want to suggest to use only non-modified DT files, i.e.
> barebox
> > should use to build their own files, which include the original data from
> the kernel.
> > 
> > Just for example:
> > # ls
> > bb_<board.dts>
> > <board.dts>
> > 
> > # cat bb_<board.dts>
> > 
> > #include "<board.dts>"
> > /* All overrides here */
> > chosen {
> >   ...
> > };
> 
> I like the idea. One disadvantage may be that the structure in the dts
> directory gets more unclear, but I guess we can't have everything.

We can store our (bb-only) DTS content in a separate place
and keep kernel dts directory clean.

---
_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox

Re: Devicetree Maintenance in barebox

[Jason Cooper]

Hi Sascha,

+ Grant Likely, Ian Campbell, devicetree ML

This discussion started on the barebox bootloader mailinglist

On Fri, Feb 07, 2014 at 08:13:32AM +0100, Sascha Hauer wrote:
> It's becoming more obvious that devicetree maintenance is painful
> because we have to sync them to the kernel regularly. My hope was that
> this would get simpler once the devicetrees get their own repository
> outside the kernel, but it seems that won't happen anytime soon.

hmm.  Ian Campbell has a tree he is working on:

  git://xenbits.xen.org/people/ianc/device-tree-rebasing.git

Also, In the DT meeting earlier this week, Grant Likely said he has the
request in to create a separate mailinglist for collaboration between
the different devicetree users (BSD, Linux, etc).

> So my current idea to continue with barebox devicetrees is:
> 
> - Maintain a kernel branch which has all devicetree changes we need in
>   barebox in a clean step-by-step series
> - rebase this branch regularly on the newer kernel
> - Copy the resulting devicetrees to barebox
> 
> The upside is that we have up to date devicetrees in barebox without
> having to resync them by hand on a per SoC basis.  Of course this also
> means that we lose the devicetree history and breakage may be introduced
> with some huge commits saying "Update devicetrees to Linux-3.x".
> 
> Any better ideas? I think we have to do something.

I think the proper solution will percolate out of the first
cross-project discussions on the new ML.

imho, the goal is to not have any project tied to a specific version of
the devicetree.  iow, we don't break backwards compatibility in the
devicetrees, and projects should revert to default behavior if new dt
parameters are missing.  This means Linux and BSD shouldn't need to keep
a current copy of the devicetree in their trees.  However, building the
bootloader is a different animal.  It needs to provide the dt blob...

Definitely fodder for the new ML.

Grant, can you please add Sascha to the list of folks to notify when the
new ML is ready?

thx,

Jason.

_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox

Re: Devicetree Maintenance in barebox

[Jean-Christophe PLAGNIOL-VILLARD]

On 09:10 Fri 07 Feb     , Jason Cooper wrote:
> Hi Sascha,
> 
> + Grant Likely, Ian Campbell, devicetree ML
> 
> This discussion started on the barebox bootloader mailinglist
> 
> On Fri, Feb 07, 2014 at 08:13:32AM +0100, Sascha Hauer wrote:
> > It's becoming more obvious that devicetree maintenance is painful
> > because we have to sync them to the kernel regularly. My hope was that
> > this would get simpler once the devicetrees get their own repository
> > outside the kernel, but it seems that won't happen anytime soon.
> 
> hmm.  Ian Campbell has a tree he is working on:
> 
>   git://xenbits.xen.org/people/ianc/device-tree-rebasing.git
> 
> Also, In the DT meeting earlier this week, Grant Likely said he has the
> request in to create a separate mailinglist for collaboration between
> the different devicetree users (BSD, Linux, etc).
> 
> > So my current idea to continue with barebox devicetrees is:
> > 
> > - Maintain a kernel branch which has all devicetree changes we need in
> >   barebox in a clean step-by-step series
> > - rebase this branch regularly on the newer kernel
> > - Copy the resulting devicetrees to barebox
> > 
> > The upside is that we have up to date devicetrees in barebox without
> > having to resync them by hand on a per SoC basis.  Of course this also
> > means that we lose the devicetree history and breakage may be introduced
> > with some huge commits saying "Update devicetrees to Linux-3.x".
> > 
> > Any better ideas? I think we have to do something.
> 
> I think the proper solution will percolate out of the first
> cross-project discussions on the new ML.
> 
> imho, the goal is to not have any project tied to a specific version of
> the devicetree.  iow, we don't break backwards compatibility in the
> devicetrees, and projects should revert to default behavior if new dt
> parameters are missing.  This means Linux and BSD shouldn't need to keep
> a current copy of the devicetree in their trees.  However, building the
> bootloader is a different animal.  It needs to provide the dt blob...
> 
> Definitely fodder for the new ML.
> 
> Grant, can you please add Sascha to the list of folks to notify when the
> new ML is ready?

Yes we do need to split the DT ASAP

Best Regards,
J.
> 
> thx,
> 
> Jason.
> 
> _______________________________________________
> barebox mailing list
> barebox@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/barebox

_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox

Re: Devicetree Maintenance in barebox

[Jon Loeliger]

> Hi Sascha,
> 
> + Grant Likely, Ian Campbell, devicetree ML
> 
> Also, In the DT meeting earlier this week, Grant Likely said he has the
> request in to create a separate mailinglist for collaboration between
> the different devicetree users (BSD, Linux, etc).

...

> I think the proper solution will percolate out of the first
> cross-project discussions on the new ML.

...

> Definitely fodder for the new ML.
> 
> Grant, can you please add Sascha to the list of folks to notify when
> the new ML is ready?

I don't think there needs to be a different mailing list
in order to combine or discuss other OS's use of the device
tree compiler.  The  DTC is OS and Use-agnostic.  Discussions
of DTC needs for FreeBSD can happen right here as the orginal
purpose of this list was DTC discussion.

Are you, and Grant(?), suggesting that a separate list
should be created for FreeBSD use of DTS-file contents?
Or that DTS-file-content related discussions should be
separated from DTC discussions?


> imho, the goal is to not have any project tied to a specific version
> of the devicetree.
>
> iow, we don't break backwards compatibility in the
> devicetrees, and projects should revert to default behavior if new dt
> parameters are missing.  This means Linux and BSD shouldn't need to keep
> a current copy of the devicetree in their trees.  However, building the
> bootloader is a different animal.  It needs to provide the dt blob...

The devicetree source file format hasn't changed in years.
Yes, it is enhanced, but compatibly.  Or do you mean the
contents of the DTB for some specific platform?

Thanks,
jdl

_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox

Re: Devicetree Maintenance in barebox

[Ian Campbell]

On Sun, 2014-02-09 at 11:58 -0600, Jon Loeliger wrote:
> > Hi Sascha,
> > 
> > + Grant Likely, Ian Campbell, devicetree ML
> > 
> > Also, In the DT meeting earlier this week, Grant Likely said he has the
> > request in to create a separate mailinglist for collaboration between
> > the different devicetree users (BSD, Linux, etc).
> 
> ...
> 
> > I think the proper solution will percolate out of the first
> > cross-project discussions on the new ML.
> 
> ...
> 
> > Definitely fodder for the new ML.
> > 
> > Grant, can you please add Sascha to the list of folks to notify when
> > the new ML is ready?
> 
> I don't think there needs to be a different mailing list
> in order to combine or discuss other OS's use of the device
> tree compiler.  The  DTC is OS and Use-agnostic.  Discussions
> of DTC needs for FreeBSD can happen right here as the orginal
> purpose of this list was DTC discussion.
> 
> Are you, and Grant(?), suggesting that a separate list
> should be created for FreeBSD use of DTS-file contents?
> Or that DTS-file-content related discussions should be
> separated from DTC discussions?

The latter. See http://www.spinics.net/lists/devicetree/msg19209.html .
The issue is the enormous volume of Linux specific stuff which is too
much of a firehose to sensibly suggest non-Linux people to subscribe
too.

> 
> > imho, the goal is to not have any project tied to a specific version
> > of the devicetree.
> >
> > iow, we don't break backwards compatibility in the
> > devicetrees, and projects should revert to default behavior if new dt
> > parameters are missing.  This means Linux and BSD shouldn't need to keep
> > a current copy of the devicetree in their trees.  However, building the
> > bootloader is a different animal.  It needs to provide the dt blob...
> 
> The devicetree source file format hasn't changed in years.
> Yes, it is enhanced, but compatibly.  Or do you mean the
> contents of the DTB for some specific platform?
> 
> Thanks,
> jdl
> 



_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox

Re: Devicetree Maintenance in barebox

[Ian Campbell]

On Fri, 2014-02-07 at 09:10 -0500, Jason Cooper wrote:
> Hi Sascha,
> 
> + Grant Likely, Ian Campbell, devicetree ML
> 
> This discussion started on the barebox bootloader mailinglist
> 
> On Fri, Feb 07, 2014 at 08:13:32AM +0100, Sascha Hauer wrote:
> > It's becoming more obvious that devicetree maintenance is painful
> > because we have to sync them to the kernel regularly. My hope was that
> > this would get simpler once the devicetrees get their own repository
> > outside the kernel, but it seems that won't happen anytime soon.
> 
> hmm.  Ian Campbell has a tree he is working on:
> 
>   git://xenbits.xen.org/people/ianc/device-tree-rebasing.git

This is automatically extracted out of the linux git tree (using
git-filter-branch), maintaining all the history etc. I intend to keep
this running until such a time as the DTS files are migrated out of
Linux.

For the time being though patches to this tree need to go through the
Linux tree in the normal way.

(also, as the name states it is potentially rebasing, although there
hasn't been an actual need since the early days when I was still getting
things working)

Ian.


_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox

Re: Devicetree Maintenance in barebox

[Jason Cooper]

On Sun, Feb 09, 2014 at 11:58:06AM -0600, Jon Loeliger wrote:
> > Hi Sascha,
> > 
> > + Grant Likely, Ian Campbell, devicetree ML
> > 
> > Also, In the DT meeting earlier this week, Grant Likely said he has the
> > request in to create a separate mailinglist for collaboration between
> > the different devicetree users (BSD, Linux, etc).
> 
> ...
> 
> > I think the proper solution will percolate out of the first
> > cross-project discussions on the new ML.
> 
> ...
> 
> > Definitely fodder for the new ML.
> > 
> > Grant, can you please add Sascha to the list of folks to notify when
> > the new ML is ready?
> 
> I don't think there needs to be a different mailing list
> in order to combine or discuss other OS's use of the device
> tree compiler.  The  DTC is OS and Use-agnostic.  Discussions
> of DTC needs for FreeBSD can happen right here as the orginal
> purpose of this list was DTC discussion.

Sorry, I wasn't clear.  I was referring to the devicetree bindings
currently being created in the linux tree, and the dts files for the
boards Linux supports.

And by 'here', I presume you me devicetree@vger.kernel.org...

> Are you, and Grant(?), suggesting that a separate list
> should be created for FreeBSD use of DTS-file contents?
> Or that DTS-file-content related discussions should be
> separated from DTC discussions?

As Ian mentioned, the separate list is to engage other consumers of
devicetree bindings/dts files/dtc use without the firehose of Linux
patches.

> > imho, the goal is to not have any project tied to a specific version
> > of the devicetree.
> >
> > iow, we don't break backwards compatibility in the
> > devicetrees, and projects should revert to default behavior if new dt
> > parameters are missing.  This means Linux and BSD shouldn't need to keep
> > a current copy of the devicetree in their trees.  However, building the
> > bootloader is a different animal.  It needs to provide the dt blob...
> 
> The devicetree source file format hasn't changed in years.
> Yes, it is enhanced, but compatibly.  Or do you mean the
> contents of the DTB for some specific platform?

I was referring specifically to dt bindings for new IP blocks, and new
versions of the same.

One problem we occasionally run into with kirkwood/dove/mvebu is that
it is very convenient having the dts tree in the linux tree.  It's
tempting to keep tying the dtb to the linux kernel version.  Which makes
it a lot harder to maintain backwards compatibility.

That gets a lot easier once the bindings and the dts files have their
own tree with their own release cycle.  Then, patch submitters would be
forced to consider how changing a binding affects the driver and vice
versa.  The concept of 'deployed dtbs' without the new whizbang bindings
would have to be considered and properly handled.

Don't get me wrong, all of us are trying very hard to do this now.  But
I think it's more like running it in qemu vice running on real hardware.
We're simulating in our heads what we think the problems will be, as
opposed to experiencing them and avoiding them.

imho, the sooner we have a separate tree for dts/bindings, the better.

thx,

Jason.

_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox

Re: Devicetree Maintenance in barebox

[Jason Cooper]

On Fri, Feb 07, 2014 at 08:13:32AM +0100, Sascha Hauer wrote:
> Hi All,
> 
> It's becoming more obvious that devicetree maintenance is painful
> because we have to sync them to the kernel regularly. My hope was that
> this would get simpler once the devicetrees get their own repository
> outside the kernel, but it seems that won't happen anytime soon.

As one of my todo items from last week's devicetree irc meeting, I've
now proposed just that:

  http://article.gmane.org/gmane.comp.devicetree.compiler/14

Also, cross-posted to devicetree and devicetree-spec.

thx,

Jason.

_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox