kernel_optimize_test/kernel/sched
Peter Zijlstra 4c4e373156 sched/wake_q: Fix wakeup ordering for wake_q
Notable cmpxchg() does not provide ordering when it fails, however
wake_q_add() requires ordering in this specific case too. Without this
it would be possible for the concurrent wakeup to not observe our
prior state.

Andrea Parri provided:

  C wake_up_q-wake_q_add

  {
	int next = 0;
	int y = 0;
  }

  P0(int *next, int *y)
  {
	int r0;

	/* in wake_up_q() */

	WRITE_ONCE(*next, 1);   /* node->next = NULL */
	smp_mb();               /* implied by wake_up_process() */
	r0 = READ_ONCE(*y);
  }

  P1(int *next, int *y)
  {
	int r1;

	/* in wake_q_add() */

	WRITE_ONCE(*y, 1);      /* wake_cond = true */
	smp_mb__before_atomic();
	r1 = cmpxchg_relaxed(next, 1, 2);
  }

  exists (0:r0=0 /\ 1:r1=0)

  This "exists" clause cannot be satisfied according to the LKMM:

  Test wake_up_q-wake_q_add Allowed
  States 3
  0:r0=0; 1:r1=1;
  0:r0=1; 1:r1=0;
  0:r0=1; 1:r1=1;
  No
  Witnesses
  Positive: 0 Negative: 3
  Condition exists (0:r0=0 /\ 1:r1=0)
  Observation wake_up_q-wake_q_add Never 0 3

Reported-by: Yongji Xie <elohimes@gmail.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Davidlohr Bueso <dave@stgolabs.net>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Waiman Long <longman@redhat.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2019-01-21 11:15:37 +01:00
..
autogroup.c
autogroup.h
clock.c
completion.c
core.c sched/wake_q: Fix wakeup ordering for wake_q 2019-01-21 11:15:37 +01:00
cpuacct.c
cpudeadline.c
cpudeadline.h
cpufreq_schedutil.c Merge branch 'sched-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2018-12-26 14:56:10 -08:00
cpufreq.c sched/cpufreq: Add the SPDX tags 2018-12-11 11:35:25 +01:00
cpupri.c
cpupri.h
cputime.c sched: Fix various typos in comments 2018-12-03 11:55:42 +01:00
deadline.c sched/core: Remove unnecessary unlikely() in push_*_task() 2018-12-11 15:16:57 +01:00
debug.c jump_label: move 'asm goto' support test to Kconfig 2019-01-06 09:46:51 +09:00
fair.c jump_label: move 'asm goto' support test to Kconfig 2019-01-06 09:46:51 +09:00
features.h sched/fair: Disable LB_BIAS by default 2018-10-02 09:45:01 +02:00
idle.c x86/stackprotector: Remove the call to boot_init_stack_canary() from cpu_startup_entry() 2018-10-22 04:07:24 +02:00
isolation.c sched: Fix various typos in comments 2018-12-03 11:55:42 +01:00
loadavg.c sched: loadavg: make calc_load_n() public 2018-10-26 16:26:32 -07:00
Makefile psi: pressure stall information for CPU, memory, and IO 2018-10-26 16:26:32 -07:00
membarrier.c sched/membarrier: synchronize_sched() with synchronize_rcu() 2018-11-27 09:21:43 -08:00
pelt.c sched/fair: Remove setting task's se->runnable_weight during PELT update 2018-10-02 09:45:03 +02:00
pelt.h sched/pelt: Fix warning and clean up IRQ PELT config 2018-10-02 09:45:00 +02:00
psi.c psi: make disabling/enabling easier for vendor kernels 2018-11-30 14:56:14 -08:00
rt.c sched/core: Remove unnecessary unlikely() in push_*_task() 2018-12-11 15:16:57 +01:00
sched-pelt.h
sched.h jump_label: move 'asm goto' support test to Kconfig 2019-01-06 09:46:51 +09:00
stats.c
stats.h psi: make disabling/enabling easier for vendor kernels 2018-11-30 14:56:14 -08:00
stop_task.c
swait.c kernel/sched/: remove caller signal_pending branch predictions 2019-01-04 13:13:48 -08:00
topology.c sched/toplogy: Introduce the 'sched_energy_present' static key 2018-12-11 15:17:01 +01:00
wait_bit.c
wait.c kernel/sched/: remove caller signal_pending branch predictions 2019-01-04 13:13:48 -08:00