AcoSail/kernel_optimize_test
8
0
Fork 0
forked from luck/tmp_suning_uos_patched
Code Pull Requests Activity
86bba269d0
kernel_optimize_test/net
History
Eric Dumazet 86bba269d0 [PATCH] NET : convert IP route cache garbage collection from softirq processing to a workqueue 
When the periodic IP route cache flush is done (every 600 seconds on
default configuration), some hosts suffer a lot and eventually trigger
the "soft lockup" message.

dst_run_gc() is doing a scan of a possibly huge list of dst_entries,
eventually freeing some (less than 1%) of them, while holding the
dst_lock spinlock for the whole scan.

Then it rearms a timer to redo the full thing 1/10 s later...
The slowdown can last one minute or so, depending on how active are
the tcp sessions.

This second version of the patch converts the processing from a softirq
based one to a workqueue.

Even if the list of entries in garbage_list is huge, host is still
responsive to softirqs and can make progress.

Instead of resetting gc timer to 0.1 second if one entry was freed in a
gc run, we do this if more than 10% of entries were freed.

Before patch :

Aug 16 06:21:37 SRV1 kernel: BUG: soft lockup detected on CPU#0!
Aug 16 06:21:37 SRV1 kernel:
Aug 16 06:21:37 SRV1 kernel: Call Trace:
Aug 16 06:21:37 SRV1 kernel:  <IRQ>  [<ffffffff802286f0>] wake_up_process+0x10/0x20
Aug 16 06:21:37 SRV1 kernel:  [<ffffffff80251e09>] softlockup_tick+0xe9/0x110
Aug 16 06:21:37 SRV1 kernel:  [<ffffffff803cd380>] dst_run_gc+0x0/0x140
Aug 16 06:21:37 SRV1 kernel:  [<ffffffff802376f3>] run_local_timers+0x13/0x20
Aug 16 06:21:37 SRV1 kernel:  [<ffffffff802379c7>] update_process_times+0x57/0x90
Aug 16 06:21:37 SRV1 kernel:  [<ffffffff80216034>] smp_local_timer_interrupt+0x34/0x60
Aug 16 06:21:37 SRV1 kernel:  [<ffffffff802165cc>] smp_apic_timer_interrupt+0x5c/0x80
Aug 16 06:21:37 SRV1 kernel:  [<ffffffff8020a816>] apic_timer_interrupt+0x66/0x70
Aug 16 06:21:37 SRV1 kernel:  [<ffffffff803cd3d3>] dst_run_gc+0x53/0x140
Aug 16 06:21:37 SRV1 kernel:  [<ffffffff803cd3c6>] dst_run_gc+0x46/0x140
Aug 16 06:21:37 SRV1 kernel:  [<ffffffff80237148>] run_timer_softirq+0x148/0x1c0
Aug 16 06:21:37 SRV1 kernel:  [<ffffffff8023340c>] __do_softirq+0x6c/0xe0
Aug 16 06:21:37 SRV1 kernel:  [<ffffffff8020ad6c>] call_softirq+0x1c/0x30
Aug 16 06:21:37 SRV1 kernel:  <EOI>  [<ffffffff8020cb34>] do_softirq+0x34/0x90
Aug 16 06:21:37 SRV1 kernel:  [<ffffffff802331cf>] local_bh_enable_ip+0x3f/0x60
Aug 16 06:21:37 SRV1 kernel:  [<ffffffff80422913>] _spin_unlock_bh+0x13/0x20
Aug 16 06:21:37 SRV1 kernel:  [<ffffffff803dfde8>] rt_garbage_collect+0x1d8/0x320
Aug 16 06:21:37 SRV1 kernel:  [<ffffffff803cd4dd>] dst_alloc+0x1d/0xa0
Aug 16 06:21:37 SRV1 kernel:  [<ffffffff803e1433>] __ip_route_output_key+0x573/0x800
Aug 16 06:21:37 SRV1 kernel:  [<ffffffff803c02e2>] sock_common_recvmsg+0x32/0x50
Aug 16 06:21:37 SRV1 kernel:  [<ffffffff803e16dc>] ip_route_output_flow+0x1c/0x60
Aug 16 06:21:37 SRV1 kernel:  [<ffffffff80400160>] tcp_v4_connect+0x150/0x610
Aug 16 06:21:37 SRV1 kernel:  [<ffffffff803ebf07>] inet_bind_bucket_create+0x17/0x60
Aug 16 06:21:37 SRV1 kernel:  [<ffffffff8040cd16>] inet_stream_connect+0xa6/0x2c0
Aug 16 06:21:37 SRV1 kernel:  [<ffffffff80422981>] _spin_lock_bh+0x11/0x30
Aug 16 06:21:37 SRV1 kernel:  [<ffffffff803c0bdf>] lock_sock_nested+0xcf/0xe0
Aug 16 06:21:37 SRV1 kernel:  [<ffffffff80422981>] _spin_lock_bh+0x11/0x30
Aug 16 06:21:37 SRV1 kernel:  [<ffffffff803be551>] sys_connect+0x71/0xa0
Aug 16 06:21:37 SRV1 kernel:  [<ffffffff803eee3f>] tcp_setsockopt+0x1f/0x30
Aug 16 06:21:37 SRV1 kernel:  [<ffffffff803c030f>] sock_common_setsockopt+0xf/0x20
Aug 16 06:21:37 SRV1 kernel:  [<ffffffff803be4bd>] sys_setsockopt+0x9d/0xc0
Aug 16 06:21:37 SRV1 kernel:  [<ffffffff8028881e>] sys_ioctl+0x5e/0x80
Aug 16 06:21:37 SRV1 kernel:  [<ffffffff80209c4e>] system_call+0x7e/0x83

After patch : (RT_CACHE_DEBUG set to 2 to get following traces)

dst_total: 75469 delayed: 74109 work_perf: 141 expires: 150 elapsed: 8092 us
dst_total: 78725 delayed: 73366 work_perf: 743 expires: 400 elapsed: 8542 us
dst_total: 86126 delayed: 71844 work_perf: 1522 expires: 775 elapsed: 8849 us
dst_total: 100173 delayed: 68791 work_perf: 3053 expires: 1256 elapsed: 9748 us
dst_total: 121798 delayed: 64711 work_perf: 4080 expires: 1997 elapsed: 10146 us
dst_total: 154522 delayed: 58316 work_perf: 6395 expires: 25 elapsed: 11402 us
dst_total: 154957 delayed: 58252 work_perf: 64 expires: 150 elapsed: 6148 us
dst_total: 157377 delayed: 57843 work_perf: 409 expires: 400 elapsed: 6350 us
dst_total: 163745 delayed: 56679 work_perf: 1164 expires: 775 elapsed: 7051 us
dst_total: 176577 delayed: 53965 work_perf: 2714 expires: 1389 elapsed: 8120 us
dst_total: 198993 delayed: 49627 work_perf: 4338 expires: 1997 elapsed: 8909 us
dst_total: 226638 delayed: 46865 work_perf: 2762 expires: 2748 elapsed: 7351 us

I successfully reduced the IP route cache of many hosts by a four factor
thanks to this patch. Previously, I had to disable "ip route flush cache"
to avoid crashes.

Signed-off-by: Eric Dumazet <dada1@cosmosbay.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2007-10-10 16:49:15 -07:00
..
9p 9p: fix bad error path in conversion routines 2007-08-23 10:25:05 -05:00
802 [NET]: Make the device list and device lookups per namespace. 2007-10-10 16:49:10 -07:00
8021q [NET]: Make the device list and device lookups per namespace. 2007-10-10 16:49:10 -07:00
appletalk [NET]: Make the device list and device lookups per namespace. 2007-10-10 16:49:10 -07:00
atm [NET]: Make the device list and device lookups per namespace. 2007-10-10 16:49:10 -07:00
ax25 [NET]: Make the device list and device lookups per namespace. 2007-10-10 16:49:10 -07:00
bluetooth [NET]: Make socket creation namespace safe. 2007-10-10 16:49:07 -07:00
bridge [NET]: Make the device list and device lookups per namespace. 2007-10-10 16:49:10 -07:00
core [PATCH] NET : convert IP route cache garbage collection from softirq processing to a workqueue 2007-10-10 16:49:15 -07:00
dccp [NET]: Make /proc/net per network namespace 2007-10-10 16:49:06 -07:00
decnet [NET]: Make the device list and device lookups per namespace. 2007-10-10 16:49:10 -07:00
econet [NET]: Make the device list and device lookups per namespace. 2007-10-10 16:49:10 -07:00
ethernet [NET] ethernet: optimize memcpy and memset 2007-10-10 16:48:26 -07:00
ieee80211 [NET]: Make /proc/net per network namespace 2007-10-10 16:49:06 -07:00
ipv4 [NET]: Make the device list and device lookups per namespace. 2007-10-10 16:49:10 -07:00
ipv6 [NET]: Make the device list and device lookups per namespace. 2007-10-10 16:49:10 -07:00
ipx [NET]: Make the device list and device lookups per namespace. 2007-10-10 16:49:10 -07:00
irda [NET]: Make the device list and device lookups per namespace. 2007-10-10 16:49:10 -07:00
iucv [NET]: Make socket creation namespace safe. 2007-10-10 16:49:07 -07:00
key [NET]: Make socket creation namespace safe. 2007-10-10 16:49:07 -07:00
lapb [PATCH] remove many unneeded #includes of sched.h 2007-02-14 08:09:54 -08:00
llc [NET]: Make the device list and device lookups per namespace. 2007-10-10 16:49:10 -07:00
mac80211 [NET]: Make the device list and device lookups per namespace. 2007-10-10 16:49:10 -07:00
netfilter [NET]: Disable netfilter sockopts when not in the initial network namespace 2007-10-10 16:49:13 -07:00
netlabel [NetLabel]: add missing rcu_dereference() calls in the LSM domain mapping hash table 2007-08-07 17:53:10 -07:00
netlink [NET]: Support multiple network namespaces with netlink 2007-10-10 16:49:09 -07:00
netrom [NET]: Make the device list and device lookups per namespace. 2007-10-10 16:49:10 -07:00
packet [NET]: Make the device list and device lookups per namespace. 2007-10-10 16:49:10 -07:00
rfkill [NET] RFKILL: Fix whitespace errors. 2007-07-19 10:44:38 +09:00
rose [NET]: Make the device list and device lookups per namespace. 2007-10-10 16:49:10 -07:00
rxrpc [NET]: Make socket creation namespace safe. 2007-10-10 16:49:07 -07:00
sched [NET]: Make the device list and device lookups per namespace. 2007-10-10 16:49:10 -07:00
sctp [NET]: Make the device list and device lookups per namespace. 2007-10-10 16:49:10 -07:00
sunrpc [NET]: Make /proc/net per network namespace 2007-10-10 16:49:06 -07:00
tipc [NET]: Make the device list and device lookups per namespace. 2007-10-10 16:49:10 -07:00
unix [NET]: Make socket creation namespace safe. 2007-10-10 16:49:07 -07:00
wanrouter [NET]: Make /proc/net per network namespace 2007-10-10 16:49:06 -07:00
wireless [NET]: Make the device list and device lookups per namespace. 2007-10-10 16:49:10 -07:00
x25 [NET]: Make the device list and device lookups per namespace. 2007-10-10 16:49:10 -07:00
xfrm [NET]: Support multiple network namespaces with netlink 2007-10-10 16:49:09 -07:00
compat.c O_CLOEXEC for SCM_RIGHTS 2007-07-16 09:05:45 -07:00
Kconfig 9p: Reorganization of 9p file system code 2007-07-14 15:13:40 -05:00
Makefile 9p: Reorganization of 9p file system code 2007-07-14 15:13:40 -05:00
nonet.c [PATCH] Make most file operations structs in fs/ const 2006-03-28 09:16:06 -08:00
socket.c [NET]: Make the device list and device lookups per namespace. 2007-10-10 16:49:10 -07:00
sysctl_net.c Remove obsolete #include <linux/config.h> 2006-06-30 19:25:36 +02:00
TUNABLE