kernel_optimize_test/kernel/sched
Peter Zijlstra d89e588ca4 locking: Introduce smp_mb__after_spinlock()
Since its inception, our understanding of ACQUIRE, esp. as applied to
spinlocks, has changed somewhat. Also, I wonder if, with a simple
change, we cannot make it provide more.

The problem with the comment is that the STORE done by spin_lock isn't
itself ordered by the ACQUIRE, and therefore a later LOAD can pass over
it and cross with any prior STORE, rendering the default WMB
insufficient (pointed out by Alan).

Now, this is only really a problem on PowerPC and ARM64, both of
which already defined smp_mb__before_spinlock() as a smp_mb().

At the same time, we can get a much stronger construct if we place
that same barrier _inside_ the spin_lock(). In that case we upgrade
the RCpc spinlock to an RCsc.  That would make all schedule() calls
fully transitive against one another.

Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Acked-by: Will Deacon <will.deacon@arm.com>
Cc: Alan Stern <stern@rowland.harvard.edu>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Paul McKenney <paulmck@linux.vnet.ibm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2017-08-10 12:29:02 +02:00
..
autogroup.c
autogroup.h
clock.c
completion.c
core.c locking: Introduce smp_mb__after_spinlock() 2017-08-10 12:29:02 +02:00
cpuacct.c
cpuacct.h
cpudeadline.c
cpudeadline.h
cpufreq_schedutil.c cpufreq: schedutil: Fix sugov_start() versus sugov_update_shared() race 2017-07-12 14:47:48 +02:00
cpufreq.c
cpupri.c
cpupri.h
cputime.c sched/cputime: Don't use smp_processor_id() in preemptible context 2017-07-14 10:27:15 +02:00
deadline.c sched/deadline: Fix confusing comments about selection of top pi-waiter 2017-07-14 10:35:16 +02:00
debug.c sched/debug: Expose the number of RT/DL tasks that can migrate 2017-06-30 09:32:07 +02:00
fair.c sched/fair: Fix load_balance() affinity redo path 2017-07-05 16:28:48 +02:00
features.h
idle_task.c
idle.c
loadavg.c
Makefile
rt.c sched/rt: Move RT related code from sched/core.c to sched/rt.c 2017-06-23 10:46:45 +02:00
sched-pelt.h
sched.h sched/rt: Move RT related code from sched/core.c to sched/rt.c 2017-06-23 10:46:45 +02:00
stats.c
stats.h
stop_task.c
swait.c sched/wait: Remove the lockless swait_active() check in swake_up*() 2017-08-10 12:28:53 +02:00
topology.c
wait_bit.c
wait.c