kernel_optimize_test/kernel/rcu
Uladzislau Rezki (Sony) 9b81af9c84 rcu/tree: Defer kvfree_rcu() allocation to a clean context
[ Upstream commit 56292e8609e39537297a7468dda4d87b9bd81d6a ]

The current memmory-allocation interface causes the following difficulties
for kvfree_rcu():

a) If built with CONFIG_PROVE_RAW_LOCK_NESTING, the lockdep will
   complain about violation of the nesting rules, as in "BUG: Invalid
   wait context".  This Kconfig option checks for proper raw_spinlock
   vs. spinlock nesting, in particular, it is not legal to acquire a
   spinlock_t while holding a raw_spinlock_t.

   This is a problem because kfree_rcu() uses raw_spinlock_t whereas the
   "page allocator" internally deals with spinlock_t to access to its
   zones. The code also can be broken from higher level of view:
   <snip>
       raw_spin_lock(&some_lock);
       kfree_rcu(some_pointer, some_field_offset);
   <snip>

b) If built with CONFIG_PREEMPT_RT, spinlock_t is converted into
   sleeplock.  This means that invoking the page allocator from atomic
   contexts results in "BUG: scheduling while atomic".

c) Please note that call_rcu() is already invoked from raw atomic context,
   so it is only reasonable to expaect that kfree_rcu() and kvfree_rcu()
   will also be called from atomic raw context.

This commit therefore defers page allocation to a clean context using the
combination of an hrtimer and a workqueue.  The hrtimer stage is required
in order to avoid deadlocks with the scheduler.  This deferred allocation
is required only when kvfree_rcu()'s per-CPU page cache is empty.

Link: https://lore.kernel.org/lkml/20200630164543.4mdcf6zb4zfclhln@linutronix.de/
Fixes: 3042f83f19 ("rcu: Support reclaim for head-less object")
Reported-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Signed-off-by: Uladzislau Rezki (Sony) <urezki@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
2020-12-30 11:53:17 +01:00
..
Kconfig rcu: Reduce leaf fanout for strict RCU grace periods 2020-08-24 18:40:23 -07:00
Kconfig.debug Merge branch 'strictgp.2020.08.24a' into HEAD 2020-09-03 09:47:42 -07:00
Makefile rcuperf: Change rcuperf to rcuscale 2020-08-24 18:39:24 -07:00
rcu_segcblist.c rcu/segcblist: Prevent useless GP start if no CBs to accelerate 2020-09-03 09:39:59 -07:00
rcu_segcblist.h rcu: Remove kfree_rcu() special casing and lazy-callback handling 2020-01-24 10:24:31 -08:00
rcu.h treewide: Make all debug_obj_descriptors const 2020-09-24 21:56:25 +02:00
rcuscale.c rcuperf: Change rcuperf to rcuscale 2020-08-24 18:39:24 -07:00
rcutorture.c rcutorture: Allow pointer leaks to test diagnostic code 2020-08-24 18:45:36 -07:00
refscale.c refperf: Avoid null pointer dereference when buf fails to allocate 2020-08-24 18:45:35 -07:00
srcutiny.c rcu: Use CONFIG_PREEMPTION where appropriate 2019-12-09 12:37:51 -08:00
srcutree.c srcu: Remove KCSAN stubs 2020-08-24 18:36:03 -07:00
sync.c rcu/sync: Simplify the state machine 2019-05-28 09:05:23 -07:00
tasks.h rcu-tasks: Enclose task-list scan in rcu_read_lock() 2020-09-16 16:32:38 -07:00
tiny.c rcu: Rename *_kfree_callback/*_kfree_rcu_offset/kfree_call_* 2020-06-29 11:59:25 -07:00
tree_exp.h rcu: Initialize at declaration time in rcu_exp_handler() 2020-08-24 18:36:03 -07:00
tree_plugin.h Merge branch 'strictgp.2020.08.24a' into HEAD 2020-09-03 09:47:42 -07:00
tree_stall.h rcu: Don't invoke try_invoke_on_locked_down_task() with irqs disabled 2020-11-10 17:10:38 -08:00
tree.c rcu/tree: Defer kvfree_rcu() allocation to a clean context 2020-12-30 11:53:17 +01:00
tree.h Merge branch 'strictgp.2020.08.24a' into HEAD 2020-09-03 09:47:42 -07:00
update.c Merge tag 'core-rcu-2020-10-12' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2020-10-18 14:34:50 -07:00