barebox picotcp integration (2015.06.14)

barebox picotcp integration (2015.06.14)

[Antony Pavlov]

Hi!

I have just published latest picotcp-enabled barebox.
Please see my 20150614.picotcp branch in my github barebox repo
(https://github.com/frantony/barebox/tree/20150614.picotcp).

This version is based on latest barebox-next and latest picotcp-development.
Here is barebox picotcp network features:

       * network interfaces and the 'ifconfig' command to rule them;
       * the 'route' command to see current picotcp route table;
       * the 'picoping' command to use picotcp ping support (also now barebox responds
         to ping requests);
       * the 'dhclient' command to use picotcp dhcp client;
       * picotcp-based tftp filesystem support.

Changes since 2015.03.31:

       * split patches (ifconfig, route, picoping and dhclient commands
         are introduced in separate patches now);
       * rework picotcp Kconfig stuff;
       * fix formatting.

Here is note/todo list:

  1. tftp write operation is not supported;

  2. with picotcp we can't easely use 'tftp' command with old syntax
     (no $<current ethernet>.serverip anymore);

  3. just now picotcp-based tftp file transfer is 5 times slower than
     old realisation (partialy it is my bad, e.g. extra memory copy operation
     is used for code simplification);

  4. picotcp has dns support, but just now it is unused in barebox;
 
  5. no nfs support in picotcp (should we can keep current barebox nfs support on
     top of barebox UDP support?).

-- 
Best regards,
  Antony Pavlov

_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox

Re: barebox picotcp integration (2015.06.14)

[Sascha Hauer]

On Mon, Jun 15, 2015 at 01:13:43AM +0300, Antony Pavlov wrote:
> Hi!
> 
> I have just published latest picotcp-enabled barebox.
> Please see my 20150614.picotcp branch in my github barebox repo
> (https://github.com/frantony/barebox/tree/20150614.picotcp).
> 
> This version is based on latest barebox-next and latest picotcp-development.
> Here is barebox picotcp network features:
> 
>        * network interfaces and the 'ifconfig' command to rule them;
>        * the 'route' command to see current picotcp route table;
>        * the 'picoping' command to use picotcp ping support (also now barebox responds
>          to ping requests);
>        * the 'dhclient' command to use picotcp dhcp client;
>        * picotcp-based tftp filesystem support.
> 
> Changes since 2015.03.31:
> 
>        * split patches (ifconfig, route, picoping and dhclient commands
>          are introduced in separate patches now);
>        * rework picotcp Kconfig stuff;
>        * fix formatting.
> 
> Here is note/todo list:
> 
>   1. tftp write operation is not supported;

Why are you using the picotcp tftp implementation? picotcp surely
supports sending/receiving udp packets, right? Wouldn't it be a good
first step to replace the barebox udp API with the one picotcp provides?
I mean I would expect that you replace only the network stack, not the
network stack including the applications. If at some point we decide
that the tftp implementation in picotcp is better than the one in
barebox that would be the time to switch it.

> 
>   2. with picotcp we can't easely use 'tftp' command with old syntax
>      (no $<current ethernet>.serverip anymore);
> 
>   3. just now picotcp-based tftp file transfer is 5 times slower than
>      old realisation (partialy it is my bad, e.g. extra memory copy operation
>      is used for code simplification);

An additional memcpy shouldn't make tftp 5 times slower. There must be
something else.

Sascha

-- 
Pengutronix e.K.                           |                             |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox

Re: barebox picotcp integration (2015.06.14)

[Antony Pavlov]

On Wed, 17 Jun 2015 11:28:57 +0200
Sascha Hauer <s.hauer@pengutronix.de> wrote:

> On Mon, Jun 15, 2015 at 01:13:43AM +0300, Antony Pavlov wrote:
> > Hi!
> > 
> > I have just published latest picotcp-enabled barebox.
> > Please see my 20150614.picotcp branch in my github barebox repo
> > (https://github.com/frantony/barebox/tree/20150614.picotcp).
> > 
...
> 
> Why are you using the picotcp tftp implementation? picotcp surely
> supports sending/receiving udp packets, right? Wouldn't it be a good
> first step to replace the barebox udp API with the one picotcp provides?
> I mean I would expect that you replace only the network stack, not the
> network stack including the applications. If at some point we decide
> that the tftp implementation in picotcp is better than the one in
> barebox that would be the time to switch it.
> 

I have reworked tftp support: now barebox tftp implementation is used on top
of picotcp udp/ip stack and works 2 times slower than original u-boot stack-based
implementation (I have tested it with sandbox arch).

Please see draft tftp commit here: https://github.com/frantony/barebox/tree/20150620.picotcp

I'll try to switch barebox nfs support to picotcp udp/ip in several days.

-- 
Best regards,
  Antony Pavlov

_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox

Re: barebox picotcp integration (2015.06.14)

[Sascha Hauer]

On Sat, Jun 20, 2015 at 02:09:12PM +0300, Antony Pavlov wrote:
> On Wed, 17 Jun 2015 11:28:57 +0200
> Sascha Hauer <s.hauer@pengutronix.de> wrote:
> 
> > On Mon, Jun 15, 2015 at 01:13:43AM +0300, Antony Pavlov wrote:
> > > Hi!
> > > 
> > > I have just published latest picotcp-enabled barebox.
> > > Please see my 20150614.picotcp branch in my github barebox repo
> > > (https://github.com/frantony/barebox/tree/20150614.picotcp).
> > > 
> ...
> > 
> > Why are you using the picotcp tftp implementation? picotcp surely
> > supports sending/receiving udp packets, right? Wouldn't it be a good
> > first step to replace the barebox udp API with the one picotcp provides?
> > I mean I would expect that you replace only the network stack, not the
> > network stack including the applications. If at some point we decide
> > that the tftp implementation in picotcp is better than the one in
> > barebox that would be the time to switch it.
> > 
> 
> I have reworked tftp support: now barebox tftp implementation is used on top
> of picotcp udp/ip stack and works 2 times slower than original u-boot stack-based
> implementation (I have tested it with sandbox arch).

Do you have an idea why this is slower? I had a quick look into it and
found nothing obvious.

Sascha

-- 
Pengutronix e.K.                           |                             |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox

Re: barebox picotcp integration (2015.06.14)

[Antony Pavlov]

On Mon, 22 Jun 2015 08:00:22 +0200
Sascha Hauer <s.hauer@pengutronix.de> wrote:

> On Sat, Jun 20, 2015 at 02:09:12PM +0300, Antony Pavlov wrote:
> > On Wed, 17 Jun 2015 11:28:57 +0200
> > Sascha Hauer <s.hauer@pengutronix.de> wrote:
> > 
> > > On Mon, Jun 15, 2015 at 01:13:43AM +0300, Antony Pavlov wrote:
> > > > Hi!
> > > > 
> > > > I have just published latest picotcp-enabled barebox.
> > > > Please see my 20150614.picotcp branch in my github barebox repo
> > > > (https://github.com/frantony/barebox/tree/20150614.picotcp).
> > > > 
> > ...
> > > 
> > > Why are you using the picotcp tftp implementation? picotcp surely
> > > supports sending/receiving udp packets, right? Wouldn't it be a good
> > > first step to replace the barebox udp API with the one picotcp provides?
> > > I mean I would expect that you replace only the network stack, not the
> > > network stack including the applications. If at some point we decide
> > > that the tftp implementation in picotcp is better than the one in
> > > barebox that would be the time to switch it.
> > > 
> > 
> > I have reworked tftp support: now barebox tftp implementation is used on top
> > of picotcp udp/ip stack and works 2 times slower than original u-boot stack-based
> > implementation (I have tested it with sandbox arch).
> 
> Do you have an idea why this is slower? I had a quick look into it and
> found nothing obvious.
> 

I have added nfs-over-picotcp support, please see https://github.com/frantony/barebox/tree/20150624.picotcp

I use very simple benchmarks.
I have sandbox barebox (192.168.1.2) and nfs- and tftp- servers on the same host (192.168.1.1).
I copy 16MiB (16777216 bytes) file from server to sandbox barebox over network several times.
I use time command inside sandox barebox for computing transfer time.

barebox@barebox sandbox:/ eth0.ipaddr=192.168.1.2
barebox@barebox sandbox:/ mount -t tftp 192.168.1.1 /mnt
...
barebox@barebox sandbox:/ time cp /mnt/16M .
time: 546ms

Here is typical transfer time values:

+--------+--------------+--------------+
|        |   U-boot     |   picotcp    |
|        |    stack     |    stack     |
+--------+--------------+--------------+
|        |              |              |
|  tftp  |    546ms     |    859ms     |
|        |              |              |
+--------+--------------+--------------+
|        |              |              |
|  nfs   |   1037ms     |   1393ms     |
|        |              |              |
+--------+--------------+--------------+

There are some rather simple possibilities to slighly improve picotcp-barebox performance:

  * drop extra memory copy;
  * drop extra pico_tick() calls (is_timeout() calls poller_call(), so there is no need
                                  to call poller_call() explicitly in nfs/tftp code);
  * there is an issue with sustained tftp small file copy. E.g.

         barebox@barebox sandbox:/ ifconfig eth 192.168.1.2 255.255.255.0
         Assigned ipv4 192.168.1.2 to device eth
         barebox@barebox sandbox:/ mount -t tftp 192.168.1.1 /mnt
         barebox@barebox sandbox:/ while true; do time cp /mnt/1M . ; done
         ...
         Warning: I have 122 timers
         Warning: I have 121 timers
         time: 48ms
         Warning: I have 122 timers
         Warning: I have 121 timers
         time: 53ms
         Warning: I have 122 timers
         Warning: I have 121 timers
         time: 54ms

Just now picotcp-enabled barebox uses

   * ping from picotcp;
   * dhcp from picotcp;
   * no DNS support.

Sascha! Do you think I should use barebox' own dhcp and ping realisation or keep one from picotcp?
Should I use DNS from barebox (over picotcp) or picotcp DNS realization?

--
Best regards,
  Antony Pavlov

_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox

Re: barebox picotcp integration (2015.06.14)

[Sascha Hauer]

On Wed, Jun 24, 2015 at 09:11:47AM +0300, Antony Pavlov wrote:
> On Mon, 22 Jun 2015 08:00:22 +0200
> Sascha Hauer <s.hauer@pengutronix.de> wrote:
> 
> > On Sat, Jun 20, 2015 at 02:09:12PM +0300, Antony Pavlov wrote:
> > > On Wed, 17 Jun 2015 11:28:57 +0200
> > > Sascha Hauer <s.hauer@pengutronix.de> wrote:
> > > 
> > > > On Mon, Jun 15, 2015 at 01:13:43AM +0300, Antony Pavlov wrote:
> > > > > Hi!
> > > > > 
> > > > > I have just published latest picotcp-enabled barebox.
> > > > > Please see my 20150614.picotcp branch in my github barebox repo
> > > > > (https://github.com/frantony/barebox/tree/20150614.picotcp).
> > > > > 
> > > ...
> > > > 
> > > > Why are you using the picotcp tftp implementation? picotcp surely
> > > > supports sending/receiving udp packets, right? Wouldn't it be a good
> > > > first step to replace the barebox udp API with the one picotcp provides?
> > > > I mean I would expect that you replace only the network stack, not the
> > > > network stack including the applications. If at some point we decide
> > > > that the tftp implementation in picotcp is better than the one in
> > > > barebox that would be the time to switch it.
> > > > 
> > > 
> > > I have reworked tftp support: now barebox tftp implementation is used on top
> > > of picotcp udp/ip stack and works 2 times slower than original u-boot stack-based
> > > implementation (I have tested it with sandbox arch).
> > 
> > Do you have an idea why this is slower? I had a quick look into it and
> > found nothing obvious.
> > 
> 
> I have added nfs-over-picotcp support, please see https://github.com/frantony/barebox/tree/20150624.picotcp
> 
> I use very simple benchmarks.
> I have sandbox barebox (192.168.1.2) and nfs- and tftp- servers on the same host (192.168.1.1).
> I copy 16MiB (16777216 bytes) file from server to sandbox barebox over network several times.
> I use time command inside sandox barebox for computing transfer time.
> 
> barebox@barebox sandbox:/ eth0.ipaddr=192.168.1.2
> barebox@barebox sandbox:/ mount -t tftp 192.168.1.1 /mnt
> ...
> barebox@barebox sandbox:/ time cp /mnt/16M .
> time: 546ms
> 
> Here is typical transfer time values:
> 
> +--------+--------------+--------------+
> |        |   U-boot     |   picotcp    |
> |        |    stack     |    stack     |
> +--------+--------------+--------------+
> |        |              |              |
> |  tftp  |    546ms     |    859ms     |
> |        |              |              |
> +--------+--------------+--------------+
> |        |              |              |
> |  nfs   |   1037ms     |   1393ms     |
> |        |              |              |
> +--------+--------------+--------------+
> 
> There are some rather simple possibilities to slighly improve picotcp-barebox performance:
> 
>   * drop extra memory copy;
>   * drop extra pico_tick() calls (is_timeout() calls poller_call(), so there is no need
>                                   to call poller_call() explicitly in nfs/tftp code);
>   * there is an issue with sustained tftp small file copy. E.g.
> 
>          barebox@barebox sandbox:/ ifconfig eth 192.168.1.2 255.255.255.0
>          Assigned ipv4 192.168.1.2 to device eth
>          barebox@barebox sandbox:/ mount -t tftp 192.168.1.1 /mnt
>          barebox@barebox sandbox:/ while true; do time cp /mnt/1M . ; done
>          ...
>          Warning: I have 122 timers
>          Warning: I have 121 timers
>          time: 48ms
>          Warning: I have 122 timers
>          Warning: I have 121 timers
>          time: 53ms
>          Warning: I have 122 timers
>          Warning: I have 121 timers
>          time: 54ms
> 
> Just now picotcp-enabled barebox uses
> 
>    * ping from picotcp;
>    * dhcp from picotcp;
>    * no DNS support.
> 
> Sascha! Do you think I should use barebox' own dhcp and ping realisation or keep one from picotcp?
> Should I use DNS from barebox (over picotcp) or picotcp DNS realization?

I assume that the barebox network stack and picotcp will have to
co-exist at least for some time. The barebox network stack will be
smaller than picotcp whereas picotcp offers more features. This means we
have to have a compatible API between both. This means for TFTP, NFS and
DNS it should not matter which network stack is in use, it should work
with both transparently. Since DHCP uses UDP the current barebox
implementation comes for free, so why not keep it for now? We can still
add the picotcp implementation later / let the user select / switch to
picotcp DHCP entirely later. The same with DNS: the barebox
implementation comes for free, so we can keep it around. For ping I
don't really care. Ping uses icmp and also is the only icmp user, so it
might not be worth it to make the barebox/picotcp APIs compatible,
instead we can just use two different implementations of the ping
command based on the network stack in use.

The network stack is a crucial part of barebox. We have to make sure
that we can use both stacks at least until we realize that the picotcp
support is so mature (by mature I mean its integration in barebox, not
the stack itself which for sure is better than what barebox ever had)
and that the size overhead of picotcp is negligible. At that point we
can drop the current barebox network stack.

Sascha

-- 
Pengutronix e.K.                           |                             |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox