Eric Dumazet a342cb4772 tcp: switch orphan_count to bare per-cpu counters ... [ Upstream commit 19757cebf0c5016a1f36f7fe9810a9f0b33c0832 ] Use of percpu_counter structure to track count of orphaned sockets is causing problems on modern hosts with 256 cpus or more. Stefan Bach reported a serious spinlock contention in real workloads, that I was able to reproduce with a netfilter rule dropping incoming FIN packets. 53.56% server [kernel.kallsyms] [k] queued_spin_lock_slowpath | ---queued_spin_lock_slowpath | --53.51%--_raw_spin_lock_irqsave | --53.51%--__percpu_counter_sum tcp_check_oom | |--39.03%--__tcp_close | tcp_close | inet_release | inet6_release | sock_close | __fput | ____fput | task_work_run | exit_to_usermode_loop | do_syscall_64 | entry_SYSCALL_64_after_hwframe | __GI___libc_close | --14.48%--tcp_out_of_resources tcp_write_timeout tcp_retransmit_timer tcp_write_timer_handler tcp_write_timer call_timer_fn expire_timers __run_timers run_timer_softirq __softirqentry_text_start As explained in commit cf86a086a1 ("net/dst: use a smaller percpu_counter batch for dst entries accounting"), default batch size is too big for the default value of tcp_max_orphans (262144). But even if we reduce batch sizes, there would still be cases where the estimated count of orphans is beyond the limit, and where tcp_too_many_orphans() has to call the expensive percpu_counter_sum_positive(). One solution is to use plain per-cpu counters, and have a timer to periodically refresh this cache. Updating this cache every 100ms seems about right, tcp pressure state is not radically changing over shorter periods. percpu_counter was nice 15 years ago while hosts had less than 16 cpus, not anymore by current standards. v2: Fix the build issue for CONFIG_CRYPTO_DEV_CHELSIO_TLS=m, reported by kernel test robot <lkp@intel.com> Remove unused socket argument from tcp_too_many_orphans() Fixes: dd24c00191 ("net: Use a percpu_counter for orphan_count") Signed-off-by: Eric Dumazet <edumazet@google.com> Reported-by: Stefan Bach <sfb@google.com> Cc: Neal Cardwell <ncardwell@google.com> Acked-by: Neal Cardwell <ncardwell@google.com> Signed-off-by: David S. Miller <davem@davemloft.net> Signed-off-by: Sasha Levin <sashal@kernel.org>		2021-11-18 14:04:08 +01:00
..
ccids	net: dccp: Convert to use the preferred fallthrough macro	2020-08-22 12:38:34 -07:00
ackvec.c	net: dccp: delete repeated words	2020-08-24 17:31:20 -07:00
ackvec.h
ccid.c
ccid.h
dccp.h	tcp: switch orphan_count to bare per-cpu counters	2021-11-18 14:04:08 +01:00
diag.c
feat.c	net: dccp: Convert to use the preferred fallthrough macro	2020-08-22 12:38:34 -07:00
feat.h
input.c	net: dccp: Convert to use the preferred fallthrough macro	2020-08-22 12:38:34 -07:00
ipv4.c	tcp: fix race condition when creating child sockets from syncookies	2020-11-23 16:32:33 -08:00
ipv6.c	ipv6: weaken the v4mapped source check	2021-03-30 14:32:01 +02:00
ipv6.h
Kconfig
Makefile
minisocks.c	dccp: don't duplicate ccid when cloning dccp sock	2021-09-22 12:27:56 +02:00
options.c	net: dccp: Convert to use the preferred fallthrough macro	2020-08-22 12:38:34 -07:00
output.c	net: dccp: Convert to use the preferred fallthrough macro	2020-08-22 12:38:34 -07:00
proto.c	tcp: switch orphan_count to bare per-cpu counters	2021-11-18 14:04:08 +01:00
qpolicy.c
sysctl.c
timer.c	inet: remove icsk_ack.blocked	2020-09-30 14:21:30 -07:00
trace.h