mail archive of the barebox mailing list
 help / color / mirror / Atom feed
From: Enrico Scholz <enrico.scholz@sigma-chemnitz.de>
To: barebox@lists.infradead.org
Cc: Enrico Scholz <enrico.scholz@sigma-chemnitz.de>
Subject: [PATCH v3 00/18] add "windowsize" (RFC 7440) support for tftp
Date: Mon, 15 Aug 2022 10:42:04 +0200	[thread overview]
Message-ID: <cover.1660552646.git.enrico.scholz@sigma-chemnitz.de> (raw)

The tftp "windowsize" greatly improves the performance of tftp
transfers.  This patchset adds support for it.

The first two patches are a little bit unrelated and enhance the 'cp
-v' output by giving information about the transfer speed.  They can
be dropped if they are unwanted.

I tested the function with an iMX8MP platform in three environments:

  - at home over OpenVPN on an ADSL 50 line  -->  27x speedup
  - 1 Gb/s connection --> 9x speedup
  - connection over 100 Mb/s switch  -->  4x speedup

In the test, I downloaded variable sized files which were filled from
/dev/urandom.  E.g.

| :/ global tftp.windowsize=128
| :/ cp -v /mnt/tftp/data-100MiB /tmp/data && sha1sum /tmp/data
|         [################################################################] 104857600 bytes, 98550375 bytes/s

For slow connection speeds, smaller files (1MiB, 4 MiB + 20 MiB) were
used.


The numbers (bytes/s) are

 | windowsize | VPN       | 1 Gb/s     | 100 Mb/s   |
 |------------|-----------|------------|------------|
 | 128        | 3.869.284 | 98.643.085 | 11.434.852 |
 |  64        | 3.863.581 | 98.550.375 | 11.434.852 |
 |  48        | 3.431.580 | 94.211.680 | 11.275.010 |
 |  32        | 2.835.129 | 85.250.081 | 10.985.605 |
 |  24        | 2.344.858 | 77.787.537 | 10.765.667 |
 |  16        | 1.734.186 | 67.519.381 | 10.210.087 |
 |  12        | 1.403.340 | 61.972.576 |  9.915.612 |
 |   8        | 1.002.462 | 50.852.376 |  9.016.130 |
 |   6        |   775.573 | 42.781.558 |  8.422.297 |
 |   4        |   547.845 | 32.066.544 |  6.835.567 |
 |   3        |   412.987 | 26.526.081 |  6.322.435 |
 |   2        |   280.987 | 19.120.641 |  5.494.241 |
 |   1        |   141.699 | 10.431.516 |  2.967.224 |
 |------------|-----------|------------|------------|
 | unpatched  |   140.587 | 10.553.301 |  2.978.063 |


The window size related parts of the patchset (with deactivated
selftest) increase the barebox binary size by

| add/remove: 4/0 grow/shrink: 8/2 up/down: 1269/-32 (1237)
| Function                                     old     new   delta
| tftp_handler                                 756    1200    +444
| tftp_put_data                                  -     184    +184
| tftp_do_open                                 428     608    +180
| tftp_window_cache_remove                       -     124    +124
| tftp_window_cache_get_pos                      -     120    +120
| tftp_send                                    296     392     +96
| tftp_do_close                                260     312     +52
| tftp_init                                     16      60     +44
| __FUNCTION__                                 610     623     +13
| tftp_open                                     64      68      +4
| tftp_lookup                                  136     140      +4
| g_tftp_window_size                             -       4      +4
| tftp_read                                    180     164     -16
| tftp_poll                                    180     164     -16
| Total: Before=626114, After=627351, chg +0.20%

Turning of the datagram cache (CONFIG_FS_TFTP_REORDER_CACHE_SIZE=0)
reduces the overhead to

| add/remove: 1/0 grow/shrink: 7/2 up/down: 537/-32 (505)
| Function                                     old     new   delta
| tftp_handler                                 756     992    +236
| tftp_do_open                                 428     564    +136
| tftp_send                                    296     392     +96
| tftp_init                                     16      60     +44
| __FUNCTION__                                 610     623     +13
| tftp_open                                     64      68      +4
| tftp_lookup                                  136     140      +4
| g_tftp_window_size                             -       4      +4
| tftp_read                                    180     164     -16
| tftp_poll                                    180     164     -16
| Total: Before=626114, After=626619, chg +0.08%

Restoring the old behaviour by CONFIG_FS_TFTP_MAX_WINDOW_SIZE=1 shows
an overhead of

| add/remove: 1/0 grow/shrink: 7/2 up/down: 449/-32 (417)
| Function                                     old     new   delta
| tftp_handler                                 756     988    +232
| tftp_do_open                                 428     564    +136
| tftp_init                                     16      60     +44
| __FUNCTION__                                 610     623     +13
| tftp_send                                    296     308     +12
| tftp_open                                     64      68      +4
| tftp_lookup                                  136     140      +4
| g_tftp_window_size                             -       4      +4
| tftp_read                                    180     164     -16
| tftp_poll                                    180     164     -16
| Total: Before=626114, After=626531, chg +0.07%


---
v2 -> v3
  - use "port=XX" mount options instead of global 'tftp.port' variable
  - allocate fifo and send buffer dynamically based on block- and
    window size of the transfer.  Do not use fixed constants anymore
  - rewritten cache code; use bitmap based functions with O(1)
    complexity instead of iterating over (small) arrays
  - unittest for cache functions
  - add information about binary sizes

v1 -> v2
  - fixes for non rfc7440 servers
---
Enrico Scholz (18):
  progress: add close_progress() to display some statistics
  libfile:copy_file: show statistics in verbose mode
  tftp: add some 'const' annotations
  tftp: allow to change tftp port
  cmd:tftp: add '-P' option to set tftp server port number
  tftp: minor refactoring of RRQ/WRQ packet generation code
  tftp: replace hardcoded blksize by global constant
  tftp: allocate buffers and fifo dynamically
  tftp: add sanity check for OACK response
  tftp: record whether tftp file is opened for lookup operation only
  tftp: reduce block size on lookup requests
  tftp: refactor data processing
  tftp: detect out-of-memory situations
  tftp: implement 'windowsize' (RFC 7440) support
  tftp: do not use 'priv->block' for RRQ
  tftp: add debug_assert() macro
  tftp: reorder tftp packets
  tftp: add selftest

 commands/tftp.c     |  22 +-
 fs/Kconfig          |  36 +++
 fs/tftp-selftest.h  |  56 ++++
 fs/tftp.c           | 640 +++++++++++++++++++++++++++++++++++++++-----
 include/progress.h  |   1 +
 lib/libfile.c       |   3 +
 lib/show_progress.c |  25 ++
 test/self/Kconfig   |   7 +
 8 files changed, 717 insertions(+), 73 deletions(-)
 create mode 100644 fs/tftp-selftest.h

-- 
2.37.1




             reply	other threads:[~2022-08-15  8:50 UTC|newest]

Thread overview: 25+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-08-15  8:42 Enrico Scholz [this message]
2022-08-15  8:42 ` [PATCH v3 01/18] progress: add close_progress() to display some statistics Enrico Scholz
2022-08-19  7:21   ` Sascha Hauer
2022-08-19  7:53     ` Enrico Scholz
2022-08-15  8:42 ` [PATCH v3 02/18] libfile:copy_file: show statistics in verbose mode Enrico Scholz
2022-08-15  8:42 ` [PATCH v3 03/18] tftp: add some 'const' annotations Enrico Scholz
2022-08-15  8:42 ` [PATCH v3 04/18] tftp: allow to change tftp port Enrico Scholz
2022-08-15  8:42 ` [PATCH v3 05/18] cmd:tftp: add '-P' option to set tftp server port number Enrico Scholz
2022-08-15  8:42 ` [PATCH v3 06/18] tftp: minor refactoring of RRQ/WRQ packet generation code Enrico Scholz
2022-08-15  8:42 ` [PATCH v3 07/18] tftp: replace hardcoded blksize by global constant Enrico Scholz
2022-08-15  8:42 ` [PATCH v3 08/18] tftp: allocate buffers and fifo dynamically Enrico Scholz
2022-08-15  8:42 ` [PATCH v3 09/18] tftp: add sanity check for OACK response Enrico Scholz
2022-08-15  8:42 ` [PATCH v3 10/18] tftp: record whether tftp file is opened for lookup operation only Enrico Scholz
2022-08-15  8:42 ` [PATCH v3 11/18] tftp: reduce block size on lookup requests Enrico Scholz
2022-08-15  8:42 ` [PATCH v3 12/18] tftp: refactor data processing Enrico Scholz
2022-08-15  8:42 ` [PATCH v3 13/18] tftp: detect out-of-memory situations Enrico Scholz
2022-08-15  8:42 ` [PATCH v3 14/18] tftp: implement 'windowsize' (RFC 7440) support Enrico Scholz
2022-08-15  8:42 ` [PATCH v3 15/18] tftp: do not use 'priv->block' for RRQ Enrico Scholz
2022-08-15  8:42 ` [PATCH v3 16/18] tftp: add debug_assert() macro Enrico Scholz
2022-08-15  8:42 ` [PATCH v3 17/18] tftp: reorder tftp packets Enrico Scholz
2022-08-15  8:42 ` [PATCH v3 18/18] tftp: add selftest Enrico Scholz
2022-08-16  9:19 ` [PATCH v3 00/18] add "windowsize" (RFC 7440) support for tftp Sascha Hauer
2022-08-16  9:40   ` Enrico Scholz
2022-08-19  7:10     ` Sascha Hauer
2022-08-19  7:22 ` Sascha Hauer

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=cover.1660552646.git.enrico.scholz@sigma-chemnitz.de \
    --to=enrico.scholz@sigma-chemnitz.de \
    --cc=barebox@lists.infradead.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox