kernel_optimize_test/kernel
Peter Zijlstra 43128b3eee perf/core: Fix data race between perf_event_set_output() and perf_mmap_close()
[ Upstream commit 68e3c69803dada336893640110cb87221bb01dcf ]

Yang Jihing reported a race between perf_event_set_output() and
perf_mmap_close():

	CPU1					CPU2

	perf_mmap_close(e2)
	  if (atomic_dec_and_test(&e2->rb->mmap_count)) // 1 - > 0
	    detach_rest = true

						ioctl(e1, IOC_SET_OUTPUT, e2)
						  perf_event_set_output(e1, e2)

	  ...
	  list_for_each_entry_rcu(e, &e2->rb->event_list, rb_entry)
	    ring_buffer_attach(e, NULL);
	    // e1 isn't yet added and
	    // therefore not detached

						    ring_buffer_attach(e1, e2->rb)
						      list_add_rcu(&e1->rb_entry,
								   &e2->rb->event_list)

After this; e1 is attached to an unmapped rb and a subsequent
perf_mmap() will loop forever more:

	again:
		mutex_lock(&e->mmap_mutex);
		if (event->rb) {
			...
			if (!atomic_inc_not_zero(&e->rb->mmap_count)) {
				...
				mutex_unlock(&e->mmap_mutex);
				goto again;
			}
		}

The loop in perf_mmap_close() holds e2->mmap_mutex, while the attach
in perf_event_set_output() holds e1->mmap_mutex. As such there is no
serialization to avoid this race.

Change perf_event_set_output() to take both e1->mmap_mutex and
e2->mmap_mutex to alleviate that problem. Additionally, have the loop
in perf_mmap() detach the rb directly, this avoids having to wait for
the concurrent perf_mmap_close() to get around to doing it to make
progress.

Fixes: 9bb5d40cd9 ("perf: Fix mmap() accounting hole")
Reported-by: Yang Jihong <yangjihong1@huawei.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Tested-by: Yang Jihong <yangjihong1@huawei.com>
Link: https://lkml.kernel.org/r/YsQ3jm2GR38SW7uD@worktop.programming.kicks-ass.net
Signed-off-by: Sasha Levin <sashal@kernel.org>
2022-07-29 17:19:11 +02:00
..
bpf bpf: Fix insufficient bounds propagation from adjust_scalar_min_max_vals 2022-07-12 16:32:17 +02:00
cgroup cgroup: Use separate src/dst nodes when preloading css_sets for migration 2022-07-21 21:20:00 +02:00
configs
debug lockdown: also lock down previous kgdb use 2022-05-30 09:33:22 +02:00
dma dma-direct: don't over-decrypt memory 2022-06-22 14:13:20 +02:00
entry KVM: rseq: Update rseq when processing NOTIFY_RESUME on xfer to KVM guest 2021-10-06 15:55:49 +02:00
events perf/core: Fix data race between perf_event_set_output() and perf_mmap_close() 2022-07-29 17:19:11 +02:00
gcov
irq random: remove unused irq_flags argument from add_interrupt_randomness() 2022-05-30 09:33:27 +02:00
kcsan
livepatch livepatch: Fix build failure on 32 bits processors 2022-04-08 14:40:15 +02:00
locking locking/lockdep: Iterate lock_classes directly when reading lockdep files 2022-04-08 14:40:32 +02:00
power PM: suspend: fix return value of __setup handler 2022-04-08 14:40:01 +02:00
printk printk: fix return value of printk.devkmsg __setup handler 2022-04-08 14:40:08 +02:00
rcu rcu: Make TASKS_RUDE_RCU select IRQ_WORK 2022-06-09 10:20:51 +02:00
sched sched/fair: Fix cfs_rq_clock_pelt() for throttled cfs_rq 2022-06-09 10:21:02 +02:00
time fix race between exit_itimers() and /proc/pid/timers 2022-07-21 21:19:59 +02:00
trace tracing/histograms: Fix memory leak problem 2022-07-21 21:19:59 +02:00
.gitignore
acct.c
async.c Revert "module, async: async_synchronize_full() on module init iff async is used" 2022-02-23 12:01:00 +01:00
audit_fsnotify.c
audit_tree.c audit: move put_tree() to avoid trim_trees refcount underflow and UAF 2021-09-03 10:09:31 +02:00
audit_watch.c
audit.c audit: improve audit queue handling when "audit=1" on cmdline 2022-02-08 18:30:34 +01:00
audit.h audit: log AUDIT_TIME_* records only from rules 2022-04-08 14:40:00 +02:00
auditfilter.c
auditsc.c audit: log AUDIT_TIME_* records only from rules 2022-04-08 14:40:00 +02:00
backtracetest.c
bounds.c
capability.c
compat.c
configs.c
context_tracking.c
cpu_pm.c PM: cpu: Make notifier chain use a raw_spinlock_t 2021-09-15 09:50:40 +02:00
cpu.c random: clear fast pool, crng, and batches in cpuhp bring up 2022-05-30 09:33:36 +02:00
crash_core.c
crash_dump.c
cred.c Revert "Add a reference to ucounts for each cred" 2021-09-08 08:49:00 +02:00
delayacct.c
dma.c
exec_domain.c
exit.c fix race between exit_itimers() and /proc/pid/timers 2022-07-21 21:19:59 +02:00
extable.c
fail_function.c
fork.c copy_process(): Move fd_install() out of sighand->siglock critical section 2022-02-23 12:01:08 +01:00
freezer.c
futex.c
gen_kheaders.sh
groups.c
hung_task.c
iomem.c
irq_work.c
jump_label.c jump_label: Fix jump_label_text_reserved() vs __init 2021-07-20 16:05:58 +02:00
kallsyms.c
kcmp.c
Kconfig.freezer
Kconfig.hz
Kconfig.locks
Kconfig.preempt
kcov.c
kexec_core.c
kexec_elf.c
kexec_file.c ima: force signature verification when CONFIG_KEXEC_SIG is configured 2022-07-21 21:20:11 +02:00
kexec_internal.h
kexec.c
kheaders.c
kmod.c
kprobes.c kprobes: Limit max data_size of the kretprobe instances 2021-12-08 09:03:20 +01:00
ksysfs.c
kthread.c kthread: Fix PF_KTHREAD vs to_kthread() race 2021-09-03 10:09:31 +02:00
latencytop.c
Makefile
module_signature.c
module_signing.c
module-internal.h
module.c module: check for exit sections in layout_sections() instead of module_init_section() 2022-05-25 09:18:02 +02:00
notifier.c
nsproxy.c
padata.c
panic.c
params.c
pid_namespace.c memcg: enable accounting for pids in nested pid namespaces 2021-09-18 13:40:36 +02:00
pid.c
profile.c profiling: fix shift-out-of-bounds bugs 2021-09-26 14:08:58 +02:00
ptrace.c ptrace: Reimplement PTRACE_KILL by always sending SIGKILL 2022-06-09 10:20:49 +02:00
range.c
reboot.c
regset.c
relay.c
resource.c
rseq.c rseq: Remove broken uapi field layout on 32-bit little endian 2022-04-08 14:40:03 +02:00
scftorture.c scftorture: Fix distribution of short handler delays 2022-06-09 10:21:01 +02:00
scs.c
seccomp.c seccomp: Fix setting loaded filter count during TSYNC 2021-08-18 08:59:06 +02:00
signal.c signal handling: don't use BUG_ON() for debugging 2022-07-21 21:20:18 +02:00
smp.c smp: Fix offline cpu check in flush_smp_call_function_queue() 2022-04-20 09:23:29 +02:00
smpboot.c
smpboot.h
softirq.c
stackleak.c gcc-plugins/stackleak: Use noinstr in favor of notrace 2022-02-23 12:01:00 +01:00
stacktrace.c
static_call.c static_call: Fix unused variable warn w/o MODULE 2021-09-08 08:49:00 +02:00
stop_machine.c
sys_ni.c
sys.c prctl: allow to setup brk for et_dyn executables 2021-09-26 14:08:57 +02:00
sysctl-test.c
sysctl.c mm: sysctl: fix missing numa_stat when !CONFIG_HUGETLB_PAGE 2022-07-21 21:20:13 +02:00
task_work.c
taskstats.c
test_kprobes.c
torture.c
tracepoint.c tracepoint: Use rcu get state and cond sync for static call updates 2021-09-03 10:09:30 +02:00
tsacct.c taskstats: Cleanup the use of task->exit_code 2022-01-27 10:54:33 +01:00
ucount.c Revert "Add a reference to ucounts for each cred" 2021-09-08 08:49:00 +02:00
uid16.c
uid16.h
umh.c
up.c
user_namespace.c Revert "Add a reference to ucounts for each cred" 2021-09-08 08:49:00 +02:00
user-return-notifier.c
user.c
usermode_driver.c
utsname_sysctl.c
utsname.c
watch_queue.c watch_queue: Free the page array when watch_queue is dismantled 2022-04-08 14:40:41 +02:00
watchdog_hld.c
watchdog.c
workqueue_internal.h
workqueue.c workqueue: Fix unbind_workers() VS wq_worker_running() race 2022-01-16 09:14:22 +01:00