short entropy delay causes RNG hardware error

short entropy delay causes RNG hardware error

[Stefano Manni]

Hello,

on a custom board based on imx6ul once linux starts I see lot of errors
like this:

  caam_jr 2141000.jr: 2000005b: CCB: desc idx 0: RNG: Hardware error

approximately every 10 seconds. And if I try to read from device the
same error occurs:

  ~# dd if=/dev/hwrng of=/dev/null bs=1 count=1
  caam_jr 2141000.jr: 2000005b: CCB: desc idx 0: RNG: Hardware error
  dd: /dev/hwrng: Invalid argument

No errors are fired in barebox where the RNG self-test is run
succesfully:

  rng_self_test: RNG software self-test passed
  caam 2140000.crypto@2140000.of: Instantiated RNG4 SH0
  caam 2140000.crypto@2140000.of: Instantiated RNG4 SH1
  caam 2140000.crypto@2140000.of: registering rng-caam

Here [1] I found that enlarging the entroy delay may help so I tried to
do the same in barebox by changing RTSDCTL_ENT_DLY_MIN from 3200 to
4800 and it succeded, I no longer see those errors in linux and trying
to read from the device works perfectly.

The strange thing is that on another board based on imx6sx I never see
those errors.

Is it something new to you? 
May we change the delay? On linux-imx 6.6.3 3200 is still used.

Best,
Stefano

[1] 
https://lore.kernel.org/all/CAOMZO5D0m1xCfgFifKz1H+oYQSfxsfuZp4U39rPMACmzv1fvjQ@mail.gmail.com/#t

Re: short entropy delay causes RNG hardware error

[Ahmad Fatoum]

Hi Stefano,

On 05.08.24 18:01, Stefano Manni wrote:
> Hello,
> 
> on a custom board based on imx6ul once linux starts I see lot of errors
> like this:
> 
>   caam_jr 2141000.jr: 2000005b: CCB: desc idx 0: RNG: Hardware error
> 
> approximately every 10 seconds.

Do you have rngd enabled? You don't need this anymore with a recent
enough Linux (>= v6.1 or perhaps earlier).

This doesn't fix the underlying issue of course, but it will save you
some CPU time.

> And if I try to read from device the
> same error occurs:
> 
>   ~# dd if=/dev/hwrng of=/dev/null bs=1 count=1
>   caam_jr 2141000.jr: 2000005b: CCB: desc idx 0: RNG: Hardware error
>   dd: /dev/hwrng: Invalid argument
> 
> No errors are fired in barebox where the RNG self-test is run
> succesfully:
> 
>   rng_self_test: RNG software self-test passed
>   caam 2140000.crypto@2140000.of: Instantiated RNG4 SH0
>   caam 2140000.crypto@2140000.of: Instantiated RNG4 SH1
>   caam 2140000.crypto@2140000.of: registering rng-caam
> 
> Here [1] I found that enlarging the entroy delay may help so I tried to
> do the same in barebox by changing RTSDCTL_ENT_DLY_MIN from 3200 to
> 4800 and it succeded, I no longer see those errors in linux and trying
> to read from the device works perfectly.

That's the correct resolution, yes.

> The strange thing is that on another board based on imx6sx I never see
> those errors.

There can be quite some variance in the time needed to collect enough
entropy, even for SoCs cut from the same wafer, so this is not unusual.

> Is it something new to you? 

This issue has been there for a long time. Optimally, NXP would
store a suitable delay value into the OTP in the factory or at least
public suitable initial delay values for their SoCs, but unfortunately
they don't.

> May we change the delay? On linux-imx 6.6.3 3200 is still used.

The driver starts at 3200 and increases the delay in steps of 40 until
it reaches a suitable value. There's a maximum (12800), but it's not
used as that would slow down the RNG. Thus the search at probe time for
the lowest possible delay value that still allows instantiating the RNG.

Unfortunately, a successful instantiated RNG doesn't necessarily mean that
actual runtime usage of the CAAM won't trigger errors.

Increasing RTSDCTL_ENT_DLY_MIN to a value appropriate for your board is
thus the way to go. I think it would be appropriate to add a device tree
property, so the minimum delay value can be set on a per-board basis.

Cheers,
Ahmad


> 
> Best,
> Stefano
> 
> [1] 
> https://lore.kernel.org/all/CAOMZO5D0m1xCfgFifKz1H+oYQSfxsfuZp4U39rPMACmzv1fvjQ@mail.gmail.com/#t
> 
> 
> 
> 

-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

Re: short entropy delay causes RNG hardware error

[Stefano Manni]

Hello Ahmad,

On Mon, 2024-08-05 at 18:56 +0200, Ahmad Fatoum wrote:
> 
> Do you have rngd enabled? You don't need this anymore with a recent
> enough Linux (>= v6.1 or perhaps earlier).
> 
> This doesn't fix the underlying issue of course, but it will save you
> some CPU time.

Your speculation that rngd is running comes from the error pops every
10seconds? I cannot see any systemd service related to rng running on
my machine. How can I disable it?

> 
> > And if I try to read from device the
> > same error occurs:
> > 
> >   ~# dd if=/dev/hwrng of=/dev/null bs=1 count=1
> >   caam_jr 2141000.jr: 2000005b: CCB: desc idx 0: RNG: Hardware
> > error
> >   dd: /dev/hwrng: Invalid argument
> > 
> > No errors are fired in barebox where the RNG self-test is run
> > succesfully:
> > 
> >   rng_self_test: RNG software self-test passed
> >   caam 2140000.crypto@2140000.of: Instantiated RNG4 SH0
> >   caam 2140000.crypto@2140000.of: Instantiated RNG4 SH1
> >   caam 2140000.crypto@2140000.of: registering rng-caam
> > 
> > Here [1] I found that enlarging the entroy delay may help so I
> > tried to
> > do the same in barebox by changing RTSDCTL_ENT_DLY_MIN from 3200 to
> > 4800 and it succeded, I no longer see those errors in linux and
> > trying
> > to read from the device works perfectly.
> 
> That's the correct resolution, yes.
> 
> > The strange thing is that on another board based on imx6sx I never
> > see
> > those errors.
> 
> There can be quite some variance in the time needed to collect enough
> entropy, even for SoCs cut from the same wafer, so this is not
> unusual.
> 
> > Is it something new to you? 
> 
> This issue has been there for a long time. Optimally, NXP would
> store a suitable delay value into the OTP in the factory or at least
> public suitable initial delay values for their SoCs, but
> unfortunately
> they don't.
> 
> > May we change the delay? On linux-imx 6.6.3 3200 is still used.
> 
> The driver starts at 3200 and increases the delay in steps of 40
> until
> it reaches a suitable value. There's a maximum (12800), but it's not
> used as that would slow down the RNG. Thus the search at probe time
> for
> the lowest possible delay value that still allows instantiating the
> RNG.
> 
> Unfortunately, a successful instantiated RNG doesn't necessarily mean
> that
> actual runtime usage of the CAAM won't trigger errors.
> 
> Increasing RTSDCTL_ENT_DLY_MIN to a value appropriate for your board
> is
> thus the way to go. I think it would be appropriate to add a device
> tree
> property, so the minimum delay value can be set on a per-board basis.
> 

Got it.

Thank you

Re: short entropy delay causes RNG hardware error

[Ahmad Fatoum]

Hello Stefano,

On 06.08.24 10:33, Stefano Manni wrote:
> Hello Ahmad,
> 
> On Mon, 2024-08-05 at 18:56 +0200, Ahmad Fatoum wrote:
>>
>> Do you have rngd enabled? You don't need this anymore with a recent
>> enough Linux (>= v6.1 or perhaps earlier).
>>
>> This doesn't fix the underlying issue of course, but it will save you
>> some CPU time.
> 
> Your speculation that rngd is running comes from the error pops every
> 10seconds? I cannot see any systemd service related to rng running on
> my machine. How can I disable it?

Maybe you have haveged running? You could use inotify to see who
is opening the HWRNG's device file.

>> Increasing RTSDCTL_ENT_DLY_MIN to a value appropriate for your board
>> is
>> thus the way to go. I think it would be appropriate to add a device
>> tree
>> property, so the minimum delay value can be set on a per-board basis.
>>
> 
> Got it.

The property would also be appropriate to have in the upstream device
tree bindings, but it's ok to add it to barebox only for now. In that case
use a barebox, prefix though.

Cheers,
Ahmad


> 
> Thank you
> 
> 

-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |