Re: imx7d dual core boot

[Ahmad Fatoum <a.fatoum@pengutronix.de>]

Hello Giorgio,

On 3/30/20 4:33 PM, Giorgio wrote:
> Hi Ahmad,
> 
> I think I've found the problem with my imx7d:
> in your commit id ae0a375ba71ba9b9a70cb7eda177445fcfbf586d:
> 
> ARM: cache-armv7: remove duplicate domain initialization
> ...
> 
> you removed the two asm lines:
> 
> 		movne	r1, #-1
> 		mcrne	p15, 0, r1, c3, c0, 0	@ load domain access control
> 
> from arch/arm/cpu/cache-armv7.S

Thanks for bisecting this.
The patch you ended up at is stopping barebox from runnning in manager domain.
This is necessary to avoid speculative execution on memory mapped IO regions.
With manager domain, you can access/execute everything.
Without, you will have to clear the eXecute Never bit for regions you need to execute.
It's cleared by default for the SDRAM banks and for the barebox text section.
SRAM or BootROM that's jumped into after MMU setup need to be explicitly marked
as executable.

That said, I am not sure why it breaks..

I've got myself an i.MX7D and will take a look the coming days
where it goes wrong. If you have new info/patches to share in the
meanwhile, please do. :)

> If I add them back then the imx7d does not hang anymore and
> the kernel is able to boot and power up the second core as
> expected.
> 
> I have just one last question: what is the reason for the
> 'mov r12, lr' at the beginning of v7_mmu_cache_on ?

It looks like a left-over.
837795895801e6c3 ("ARM __mmu_cache_*: Do not clobber registers")
made it obsolete.

Cheers
Ahmad

> 
> giorgio
> 
> On 3/27/20 11:01 AM, Ahmad Fatoum wrote:
>> Hello,
>>
>> On 3/27/20 9:27 AM, Giorgio wrote:
>>> Hi,
>>>
>>> maybe 'crash' was not the best word to describe the effect I see;
>>>
>>> the bootloader just stops running, never returns from the call
>>> to __mmu_cache_on(), there's no debug messages or stack dumps;
>>> in this case the kernel image is not started.
>>>
>>> I think on my arch / soc __mmu_cache_on() calls some asm coded function
>>> defined in arch/arm/cpu/cache-armv7.S (v7_mmu_cache_on) and one
>>> of the opcodes there crashes the cpu directly. Maybe I should try to
>>> dump the core state (registers, ...) on the console just before jumping
>>> to v7_mmu_cache_on.
>>
>> AFAIU, non-secure barebox keeps reusing the secure monitor barebox till
>> it boots the kernel. The secure monitor already had translation tables
>> set up and active and non-secure barebox reuses those.
>>
>> Given that this should've worked once, could you run a git-bisect and
>> see when it broke?
>>
>> If you make bnet the default target, only board interaction you need
>> for bisecting should be the power cycle when you do a git bisect bad.
>>
>> This is most likely the fastest way to find what broke it.
>>
>> Cheers
>> Ahmad
>>
>>>
>>> giorgio
>>>
>>> On 3/27/20 6:56 AM, Ahmad Fatoum wrote:
>>>> Hello,
>>>>
>>>> On 3/26/20 11:21 AM, Giorgio wrote:
>>>>> Hi,
>>>>>
>>>>> I'm having problems booting a linux kernel image on an imx7d.
>>>>>
>>>>> What I want is to boot in nonsecure mode and that the kernel enables the
>>>>> second core with the PSCI.
>>>>>
>>>>> What I see is that barebox crashes within the call to __mmu_cache_on()
>>>>> in the file arch/arm/cpu/sm.c:
>>>>
>>>> The kind of crash may be useful info. Can you copy the whole boot log?
>>>>
>>>>>
>>>>> int armv7_secure_monitor_install(void)
>>>>> {
>>>>> ...
>>>>> 	if (mmuon) {
>>>>> 		/*
>>>>> 		 * If the MMU was already turned on in secure mode, enable it in
>>>>> 		 * non-secure mode aswell
>>>>> 		 */
>>>>> 		__mmu_cache_on();
>>>>> 	}
>>>>> ...
>>>>>
>>>>>
>>>>> As a workaround, if I comment the call to __mmu_cache_on(), the system
>>>>> boots properly with 2 cores:
>>>>>
>>>>> Loading ARM Linux zImage '/mnt/boot/kernel.img'
>>>>> Loading devicetree from '/mnt/boot/devtree.dtb'
>>>>> commandline: console=ttymxc5,115200n8 ip=::::gdm:eth0:off root=PARTUUID=37199a57-ec72-41f9-ab27-445329d1bd77 rootwait
>>>>> Starting kernel in nonsecure mode
>>>>> [    0.000000] 000: Booting Linux on physical CPU 0x0
>>>>> [    0.000000] 000: Linux version 5.4.24-rt15-dirty (giorgio@BVblfs) (gcc version 9.2.1 20200110 (OSELAS.Toolchain 9.2.1)) #1 SMP PREEMPT_RT Fri Mar 13 17:25:51 CET 2020
>>>>> [    0.000000] 000: CPU: ARMv7 Processor [410fc075] revision 5 (ARMv7), cr=10c5387d
>>>>> [    0.000000] 000: CPU: div instructions available: patching division code
>>>>> [    0.000000] 000: CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache
>>>>> t[    0.000000] 000: Memory policy: Data cache writealloc
>>>>> [    0.000000] 000: cma: Reserved 64 MiB at 0xfa000000
>>>>> [    0.000000] 000: psci: probing for conduit method from DT.
>>>>> [    0.000000] 000: psci: PSCIv1.0 detected in firmware.
>>>>> [    0.000000] 000: psci: Using standard PSCI v0.2 function IDs
>>>>> [    0.000000] 000: psci: MIGRATE_INFO_TYPE not supported.
>>>>> [    0.000000] 000: psci: SMC Calling Convention v1.0
>>>>> [    0.000000] 000: percpu: Embedded 9 pages/cpu s15008 r0 d21856 u36864
>>>>> [    0.000000] 000: Built 1 zonelists, mobility grouping on.  Total pages: 522751
>>>>> [    0.000000] 000: Kernel command line: console=ttymxc5,115200n8 ip=::::gdm:eth0:off root=PARTUUID=37199a57-ec72-41f9-ab27-445329d1bd77 rootwait
>>>>> [    0.000000] 000: Dentry cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)
>>>>> [    0.000000] 000: Inode-cache hash table entries: 65536 (order: 6, 262144 bytes, linear)
>>>>> [    0.000000] 000: mem auto-init: stack:off, heap alloc:off, heap free:off
>>>>> [    0.000000] 000: Memory: 1997464K/2097148K available (9216K kernel code, 430K rwdata, 3100K rodata, 1024K init, 733K bss, 34148K reserved, 65536K cma-reserved, 1244448K highmem)
>>>>> [    0.000000] 000: SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1
>>>>> [    0.000000] 000: rcu: Preemptible hierarchical RCU implementation.
>>>>> [    0.000000] 000: rcu: 	RCU priority boosting: priority 1 delay 500 ms.
>>>>> [    0.000000] 000: rcu: 	RCU_SOFTIRQ processing moved to rcuc kthreads.
>>>>> [    0.000000] 000: 	No expedited grace period (rcu_normal_after_boot).
>>>>> [    0.000000] 000: 	Tasks RCU enabled.
>>>>> [    0.000000] 000: rcu: RCU calculated value of scheduler-enlistment delay is 10 jiffies.
>>>>> [    0.000000] 000: NR_IRQS: 16, nr_irqs: 16, preallocated irqs: 16
>>>>> [    0.000000] 000: rcu: 	Offload RCU callbacks from CPUs: (none).
>>>>> [    0.000000] 000: arch_timer: cp15 timer(s) running at 8.00MHz (virt).
>>>>> [    0.000000] 000: clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x1d854df40, max_idle_ns: 440795202120 ns
>>>>> [    0.000001] 000: sched_clock: 56 bits at 8MHz, resolution 125ns, wraps every 2199023255500ns
>>>>> [    0.000013] 000: Switching to timer-based delay loop, resolution 125ns
>>>>> [    0.000426] 000: Switching to timer-based delay loop, resolution 41ns
>>>>> [    0.000438] 000: sched_clock: 32 bits at 24MHz, resolution 41ns, wraps every 89478484971ns
>>>>> [    0.000450] 000: clocksource: mxc_timer1: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 79635851949 ns
>>>>> [    0.001597] 000: Console: colour dummy device 80x30
>>>>> [    0.001642] 000: Calibrating delay loop (skipped), value calculated using timer frequency.. 48.00 BogoMIPS (lpj=240000)
>>>>> [    0.001650] 000: pid_max: default: 32768 minimum: 301
>>>>> [    0.001788] 000: Mount-cache hash table entries: 2048 (order: 1, 8192 bytes, linear)
>>>>> [    0.001807] 000: Mountpoint-cache hash table entries: 2048 (order: 1, 8192 bytes, linear)
>>>>> [    0.002631] 000: CPU: Testing write buffer coherency: ok
>>>>> [    0.059991] 000: Setting up static identity map for 0x80100000 - 0x80100060
>>>>> [    0.079972] 000: rcu: Hierarchical SRCU implementation.
>>>>> [    0.160105] 000: smp: Bringing up secondary CPUs ...
>>>>> [    0.280687] 000: smp: Brought up 1 node, 2 CPUs
>>>>> [    0.280697] 000: SMP: Total of 2 processors activated (96.00 BogoMIPS).
>>>>> [    0.280705] 000: CPU: All CPU(s) started in SVC mode.
>>>>> [    0.281170] 000: devtmpfs: initialized
>>>>> [    0.290961] 000: random: get_random_u32 called from bucket_table_alloc+0x58/0x14c with crng_init=0
>>>>>
>>>>>
>>>>> giorgio
>>>>>
>>>>> _______________________________________________
>>>>> barebox mailing list
>>>>> barebox@lists.infradead.org
>>>>> http://lists.infradead.org/mailman/listinfo/barebox
>>>>>
>>>>
>>>
>>
> 

-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

_______________________________________________
barebox mailing list
barebox@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/barebox