Alexei Starovoitov e21d2b9235 bpf: Fix fexit trampoline. ... [ Upstream commit e21aa341785c679dd409c8cb71f864c00fe6c463 ] The fexit/fmod_ret programs can be attached to kernel functions that can sleep. The synchronize_rcu_tasks() will not wait for such tasks to complete. In such case the trampoline image will be freed and when the task wakes up the return IP will point to freed memory causing the crash. Solve this by adding percpu_ref_get/put for the duration of trampoline and separate trampoline vs its image life times. The "half page" optimization has to be removed, since first_half->second_half->first_half transition cannot be guaranteed to complete in deterministic time. Every trampoline update becomes a new image. The image with fmod_ret or fexit progs will be freed via percpu_ref_kill and call_rcu_tasks. Together they will wait for the original function and trampoline asm to complete. The trampoline is patched from nop to jmp to skip fexit progs. They are freed independently from the trampoline. The image with fentry progs only will be freed via call_rcu_tasks_trace+call_rcu_tasks which will wait for both sleepable and non-sleepable progs to complete. Fixes: fec56f5890 ("bpf: Introduce BPF trampoline") Reported-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Acked-by: Paul E. McKenney <paulmck@kernel.org> # for RCU Link: https://lore.kernel.org/bpf/20210316210007.38949-1-alexei.starovoitov@gmail.com Signed-off-by: Sasha Levin <sashal@kernel.org>		2021-04-07 15:00:03 +02:00
..
preload	bpf: Fix umd memory leak in copy_process()	2021-03-30 14:32:03 +02:00
arraymap.c
bpf_inode_storage.c	bpf: Change inode_storage's lookup_elem return value from NULL to -EBADF	2021-03-30 14:31:56 +02:00
bpf_iter.c	bpf: Fix an unitialized value in bpf_iter	2021-03-04 11:37:33 +01:00
bpf_local_storage.c
bpf_lru_list.c	bpf_lru_list: Read double-checked variable once without lock	2021-03-04 11:37:29 +01:00
bpf_lru_list.h
bpf_lsm.c
bpf_struct_ops_types.h
bpf_struct_ops.c	bpf: Fix fexit trampoline.	2021-04-07 15:00:03 +02:00
btf.c
cgroup.c	bpf, cgroup: Fix problematic bounds check	2021-02-10 09:29:12 +01:00
core.c	bpf: Fix fexit trampoline.	2021-04-07 15:00:03 +02:00
cpumap.c
devmap.c	bpf, devmap: Use GFP_KERNEL for xdp bulk queue allocation	2021-03-04 11:37:33 +01:00
disasm.c
disasm.h
dispatcher.c
hashtab.c
helpers.c	bpf: Fix helper bpf_map_peek_elem_proto pointing to wrong callback	2021-01-23 16:03:59 +01:00
inode.c
local_storage.c
lpm_trie.c
Makefile
map_in_map.c
map_in_map.h
map_iter.c
net_namespace.c
offload.c
percpu_freelist.c
percpu_freelist.h
prog_iter.c
queue_stack_maps.c
reuseport_array.c
ringbuf.c
stackmap.c	bpf: Check for integer overflow when using roundup_pow_of_two()	2021-02-17 11:02:25 +01:00
syscall.c	bpf: Prevent double bpf_prog_put call from bpf_tracing_prog_attach	2021-01-27 11:55:07 +01:00
sysfs_btf.c
task_iter.c	bpf: Save correct stopping point in file seq iteration	2021-01-19 18:27:28 +01:00
tnum.c
trampoline.c	bpf: Fix fexit trampoline.	2021-04-07 15:00:03 +02:00
verifier.c	bpf: Add sanity check for upper ptr_limit	2021-03-20 10:43:43 +01:00