kernel_optimize_test/kernel/sched
Frederic Weisbecker 8d495477d6 sched/cputime: Spare a seqcount lock/unlock cycle on context switch
On context switch we are locking the vtime seqcount of the scheduling-out
task twice:

 * On vtime_task_switch_common(), when we flush the pending vtime through
   vtime_account_system()

 * On arch_vtime_task_switch() to reset the vtime state.

This is pointless as these actions can be performed without the need
to unlock/lock in the middle. The reason these steps are separated is to
consolidate a very small amount of common code between
CONFIG_VIRT_CPU_ACCOUNTING_GEN and CONFIG_VIRT_CPU_ACCOUNTING_NATIVE.

Performance in this fast path is definitely a priority over artificial
code factorization so split the task switch code between GEN and
NATIVE and mutualize the parts than can run under a single seqcount
locked block.

As a side effect, vtime_account_idle() becomes included in the seqcount
protection. This happens to be a welcome preparation in order to
properly support kcpustat under vtime in the future and fetch
CPUTIME_IDLE without race.

Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rik van Riel <riel@redhat.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Wanpeng Li <wanpengli@tencent.com>
Cc: Yauheni Kaliuta <yauheni.kaliuta@redhat.com>
Link: https://lkml.kernel.org/r/20191003161745.28464-3-frederic@kernel.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2019-10-09 12:39:26 +02:00
..
autogroup.c
autogroup.h
clock.c
completion.c
core.c Merge branch 'sched-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2019-09-28 12:39:07 -07:00
cpuacct.c
cpudeadline.c
cpudeadline.h
cpufreq_schedutil.c Power management updates for 5.4-rc1 2019-09-17 19:15:14 -07:00
cpufreq.c
cpupri.c
cpupri.h
cputime.c sched/cputime: Spare a seqcount lock/unlock cycle on context switch 2019-10-09 12:39:26 +02:00
deadline.c Merge branch 'timers-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2019-09-17 12:35:15 -07:00
debug.c
fair.c sched/fair: Scale bandwidth quota and period without losing quota/period ratio precision 2019-10-09 12:38:02 +02:00
features.h
idle.c mm: remove quicklist page table caches 2019-09-24 15:54:09 -07:00
isolation.c
loadavg.c
Makefile
membarrier.c membarrier: Fix RCU locking bug caused by faulty merge 2019-10-01 21:27:50 +02:00
pelt.c
pelt.h
psi.c Merge branch 'sched-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2019-09-16 17:25:49 -07:00
rt.c Merge branch 'timers-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2019-09-17 12:35:15 -07:00
sched-pelt.h
sched.h sched/membarrier: Fix p->mm->membarrier_state racy load 2019-09-25 17:42:30 +02:00
stats.c
stats.h
stop_task.c sched: Rework pick_next_task() slow-path 2019-08-08 09:09:31 +02:00
swait.c
topology.c sched/topology: Improve load balancing on AMD EPYC systems 2019-09-03 09:17:37 +02:00
wait_bit.c
wait.c