kernel_optimize_test/kernel/rcu
Frederic Weisbecker 0836f94040 rcu/exp: Mark current CPU as exp-QS in IPI loop second pass
[ Upstream commit 81f6d49cce2d2fe507e3fddcc4a6db021d9c2e7b ]

Expedited RCU grace periods invoke sync_rcu_exp_select_node_cpus(), which
takes two passes over the leaf rcu_node structure's CPUs.  The first
pass gathers up the current CPU and CPUs that are in dynticks idle mode.
The workqueue will report a quiescent state on their behalf later.
The second pass sends IPIs to the rest of the CPUs, but excludes the
current CPU, incorrectly assuming it has been included in the first
pass's list of CPUs.

Unfortunately the current CPU may have changed between the first and
second pass, due to the fact that the various rcu_node structures'
->lock fields have been dropped, thus momentarily enabling preemption.
This means that if the second pass's CPU was not on the first pass's
list, it will be ignored completely.  There will be no IPI sent to
it, and there will be no reporting of quiescent states on its behalf.
Unfortunately, the expedited grace period will nevertheless be waiting
for that CPU to report a quiescent state, but with that CPU having no
reason to believe that such a report is needed.

The result will be an expedited grace period stall.

Fix this by no longer excluding the current CPU from consideration during
the second pass.

Fixes: b9ad4d6ed1 ("rcu: Avoid self-IPI in sync_rcu_exp_select_node_cpus()")
Reviewed-by: Neeraj Upadhyay <quic_neeraju@quicinc.com>
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Cc: Uladzislau Rezki <urezki@gmail.com>
Cc: Neeraj Upadhyay <quic_neeraju@quicinc.com>
Cc: Boqun Feng <boqun.feng@gmail.com>
Cc: Josh Triplett <josh@joshtriplett.org>
Cc: Joel Fernandes <joel@joelfernandes.org>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
2022-01-27 10:53:55 +01:00
..
Kconfig rcu: Reduce leaf fanout for strict RCU grace periods 2020-08-24 18:40:23 -07:00
Kconfig.debug Merge branch 'strictgp.2020.08.24a' into HEAD 2020-09-03 09:47:42 -07:00
Makefile rcuperf: Change rcuperf to rcuscale 2020-08-24 18:39:24 -07:00
rcu_segcblist.c rcu/segcblist: Prevent useless GP start if no CBs to accelerate 2020-09-03 09:39:59 -07:00
rcu_segcblist.h rcu: Remove kfree_rcu() special casing and lazy-callback handling 2020-01-24 10:24:31 -08:00
rcu.h srcu: Fix broken node geometry after early ssp init 2021-07-20 16:05:38 +02:00
rcuscale.c rcuperf: Change rcuperf to rcuscale 2020-08-24 18:39:24 -07:00
rcutorture.c rcutorture: Avoid problematic critical section nesting on PREEMPT_RT 2021-11-18 14:03:52 +01:00
refscale.c refperf: Avoid null pointer dereference when buf fails to allocate 2020-08-24 18:45:35 -07:00
srcutiny.c srcu: Provide polling interfaces for Tiny SRCU grace periods 2021-09-03 10:09:30 +02:00
srcutree.c srcu: Provide polling interfaces for Tree SRCU grace periods 2021-09-03 10:09:29 +02:00
sync.c rcu/sync: Simplify the state machine 2019-05-28 09:05:23 -07:00
tasks.h rcu-tasks: Move RTGS_WAIT_CBS to beginning of rcu_tasks_kthread() loop 2021-11-18 14:03:53 +01:00
tiny.c rcu: Rename *_kfree_callback/*_kfree_rcu_offset/kfree_call_* 2020-06-29 11:59:25 -07:00
tree_exp.h rcu/exp: Mark current CPU as exp-QS in IPI loop second pass 2022-01-27 10:53:55 +01:00
tree_plugin.h rcu: Always inline rcu_dynticks_task*_{enter,exit}() 2021-11-18 14:04:06 +01:00
tree_stall.h rcu: Fix stall-warning deadlock due to non-release of rcu_node ->lock 2021-09-15 09:50:28 +02:00
tree.c rcu: Mark accesses to rcu_state.n_force_qs 2021-12-22 09:30:59 +01:00
tree.h Merge branch 'strictgp.2020.08.24a' into HEAD 2020-09-03 09:47:42 -07:00
update.c rcu: Reject RCU_LOCKDEP_WARN() false positives 2021-07-20 16:05:38 +02:00