kernel_optimize_test/include/net
Paolo Abeni 7c13f97ffd udp: do fwd memory scheduling on dequeue
A new argument is added to __skb_recv_datagram to provide
an explicit skb destructor, invoked under the receive queue
lock.
The UDP protocol uses such argument to perform memory
reclaiming on dequeue, so that the UDP protocol does not
set anymore skb->desctructor.
Instead explicit memory reclaiming is performed at close() time and
when skbs are removed from the receive queue.
The in kernel UDP protocol users now need to call a
skb_recv_udp() variant instead of skb_recv_datagram() to
properly perform memory accounting on dequeue.

Overall, this allows acquiring only once the receive queue
lock on dequeue.

Tested using pktgen with random src port, 64 bytes packet,
wire-speed on a 10G link as sender and udp_sink as the receiver,
using an l4 tuple rxhash to stress the contention, and one or more
udp_sink instances with reuseport.

nr sinks	vanilla		patched
1		440		560
3		2150		2300
6		3650		3800
9		4450		4600
12		6250		6450

v1 -> v2:
 - do rmem and allocated memory scheduling under the receive lock
 - do bulk scheduling in first_packet_length() and in udp_destruct_sock()
 - avoid the typdef for the dequeue callback

Suggested-by: Eric Dumazet <edumazet@google.com>
Acked-by: Hannes Frederic Sowa <hannes@stressinduktion.org>
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
Acked-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2016-11-07 13:24:41 -05:00
..
9p
bluetooth Bluetooth: Add support for appearance in scan rsp 2016-09-19 20:19:34 +02:00
caif
irda
iucv
netfilter netfilter: move socket lookup infrastructure to nf_socket_ipv{4,6}.c 2016-11-01 20:50:31 +01:00
netns Merge branch 'master' of git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next 2016-09-25 23:34:19 +02:00
nfc NFC: digital: Add support for NFC DEP Response Waiting Time 2016-07-11 02:01:14 +02:00
phonet
sctp sctp: remove the old ttl expires policy 2016-10-13 09:44:14 -04:00
tc_act skbedit: allow the user to specify bitmask for mark 2016-10-27 16:07:25 -04:00
6lowpan.h 6lowpan: add 802.15.4 short addr slaac 2016-06-15 20:41:22 -07:00
act_api.h net_sched: convert tcf_exts from list to pointer array 2016-08-17 19:27:51 -04:00
addrconf.h ipv6: fix a potential deadlock in do_ipv6_setsockopt() 2016-10-21 11:29:02 -04:00
af_ieee802154.h
af_rxrpc.h rxrpc: Rewrite the data and ack handling code 2016-09-08 11:10:12 +01:00
af_unix.h af_unix: split 'u->readlock' into two: 'iolock' and 'bindlock' 2016-09-04 13:29:29 -07:00
af_vsock.h VSOCK: Introduce virtio_vsock_common.ko 2016-08-02 02:57:29 +03:00
ah.h
arp.h
atmclip.h
ax25.h
ax88796.h
bond_3ad.h
bond_alb.h
bond_options.h
bonding.h bonding: quit messing with IOCTL 2016-09-27 21:15:21 -04:00
busy_poll.h
calipso.h calipso: Add a label cache. 2016-06-27 15:06:17 -04:00
cfg80211-wext.h
cfg80211.h Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2016-10-30 12:42:58 -04:00
cfg802154.h ieee802154: add netns support 2016-07-08 12:20:57 +02:00
checksum.h csum: Update csum_block_add to use rotate instead of byteswap 2016-03-13 15:01:00 -04:00
cipso_ipv4.h
cls_cgroup.h cls_cgroup: get sk_classid only from full sockets 2016-04-19 20:09:25 -04:00
codel_impl.h codel: split into multiple files 2016-04-25 16:44:27 -04:00
codel_qdisc.h net_sched: fq_codel: cache skb->truesize into skb->cb 2016-06-25 12:19:35 -04:00
codel.h codel: split into multiple files 2016-04-25 16:44:27 -04:00
compat.h packet: compat support for sock_fprog 2016-06-09 23:41:03 -07:00
datalink.h
dcbevent.h
dcbnl.h
devlink.h devlink: remove unused priv_size 2016-08-26 11:55:18 -07:00
dn_dev.h
dn_fib.h
dn_neigh.h
dn_nsp.h
dn_route.h
dn.h
dsa.h net: dsa: add port fast ageing 2016-09-23 08:38:50 -04:00
dsfield.h
dst_cache.h
dst_metadata.h net/dst: Utility functions to build dst_metadata without supplying an skb 2016-09-10 20:53:55 -07:00
dst_ops.h
dst.h route: move lwtunnel state to a single place 2016-04-25 16:20:09 -04:00
esp.h
ethoc.h
fib_rules.h net: core: add UID to flows, rules, and routes 2016-11-04 14:45:23 -04:00
firewire.h
flow_dissector.h rps: flow_dissector: Add the const for the parameter of flow_keys_have_l4 2016-09-01 16:51:08 -07:00
flow.h net: inet: Support UID-based routing in IP protocols. 2016-11-04 14:45:23 -04:00
flowcache.h
fou.h fou: Add encap ops for IPv6 tunnels 2016-05-20 18:03:16 -04:00
fq_impl.h fq.h: Port memory limit mechanism from fq_codel 2016-09-30 13:29:21 +02:00
fq.h fq.h: Port memory limit mechanism from fq_codel 2016-09-30 13:29:21 +02:00
garp.h
gen_stats.h net: sched: do not acquire qdisc spinlock in qdisc/class stats dump 2016-06-07 16:37:14 -07:00
genetlink.h genetlink: use idr to track families 2016-10-27 16:16:09 -04:00
geneve.h net: Remove deprecated tunnel specific UDP offload functions 2016-06-17 20:23:32 -07:00
gre.h Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2016-08-18 01:17:32 -04:00
gro_cells.h gro_cells: gro_cells_receive now return error code 2016-07-22 00:50:41 -04:00
gtp.h gtp: #define #define _GTP_H_ and not #define _GTP_H 2016-07-25 17:55:43 -07:00
gue.h
hwbm.h net: add a hardware buffer management helper API 2016-03-14 12:19:46 -04:00
icmp.h net: snmp: kill STATS_BH macros 2016-04-27 22:48:25 -04:00
ieee80211_radiotap.h wireless: radiotap: fix timestamp sampling position values 2016-10-19 12:11:36 +02:00
ieee802154_netdev.h
if_inet6.h IPv6: fix DESYNC_FACTOR 2016-10-14 10:59:15 -04:00
ila.h
inet_common.h ip4ip6: Support for GSO/GRO 2016-05-20 18:03:17 -04:00
inet_connection_sock.h tcp/dccp: drop SYN packets if accept queue is full 2016-10-29 15:09:21 -04:00
inet_ecn.h ipv6: suppress sparse warnings in IP6_ECN_set_ce() 2016-08-13 15:08:00 -07:00
inet_frag.h ipv4: namespacify ip fragment max dist sysctl knob 2016-02-16 20:42:54 -05:00
inet_hashtables.h tcp/dccp: do not touch listener sk_refcnt under synflood 2016-04-04 22:11:20 -04:00
inet_sock.h ipv4: add IP_RECVFRAGSIZE cmsg 2016-11-03 15:41:11 -04:00
inet_timewait_sock.h
inet6_connection_sock.h ipv6: remove unused in6_addr struct 2016-03-22 15:45:44 -04:00
inet6_hashtables.h tcp/dccp: do not touch listener sk_refcnt under synflood 2016-04-04 22:11:20 -04:00
inetpeer.h
ip_fib.h switchdev: remove FIB offload infrastructure 2016-09-28 04:48:00 -04:00
ip_tunnels.h ip_tunnel: add collect_md mode to IPIP tunnel 2016-09-17 10:13:07 -04:00
ip_vs.h ipvs: update real-server binding of outgoing connections in SIP-pe 2016-06-06 09:47:25 +09:00
ip.h net/sock: add an explicit sk argument for ip_cmsg_recv_offset() 2016-11-07 13:24:41 -05:00
ip6_checksum.h ipv6: Pass proto to csum_ipv6_magic as __u8 instead of unsigned short 2016-03-13 23:55:13 -04:00
ip6_fib.h net: ipv6: Fix processing of RAs in presence of VRF 2016-10-27 16:30:52 -04:00
ip6_route.h net: inet: Support UID-based routing in IP protocols. 2016-11-04 14:45:23 -04:00
ip6_tunnel.h ip6_tunnel: add collect_md mode to IPv6 tunnels 2016-09-17 10:13:07 -04:00
ipcomp.h
ipconfig.h
ipv6.h Merge branch 'stable-4.8' of git://git.infradead.org/users/pcmoore/selinux into next 2016-07-07 10:15:34 +10:00
ipx.h
iw_handler.h
kcm.h kcm: Use stream parser 2016-08-17 19:36:23 -04:00
l3mdev.h net: ipv4: Do not drop to make_route if oif is l3mdev 2016-10-13 12:05:26 -04:00
lapb.h
lib80211.h
llc_c_ac.h
llc_c_ev.h
llc_c_st.h
llc_conn.h
llc_if.h
llc_pdu.h
llc_s_ac.h
llc_s_ev.h
llc_s_st.h
llc_sap.h
llc.h
lwtunnel.h lwt: Remove unused len field 2016-10-23 17:45:01 -04:00
mac80211.h Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2016-10-30 12:42:58 -04:00
mac802154.h ieee802154: cleanup WARN_ON for fc fetch 2016-07-08 13:23:12 +02:00
mip6.h
mld.h
mpls_iptunnel.h
mpls.h openvswitch: use mpls_hdr 2016-10-03 02:00:22 -04:00
mrp.h
ncsi.h net/ncsi: Introduce ncsi_stop_dev() 2016-10-04 02:11:51 -04:00
ndisc.h 6lowpan: introduce 6lowpan-nd 2016-06-15 20:41:23 -07:00
neighbour.h
net_namespace.h netns: Add a limit on the number of net namespaces 2016-08-08 14:42:04 -05:00
net_ratelimit.h
netevent.h neigh: Send a notification when DELAY_PROBE_TIME changes 2016-07-05 09:06:29 -07:00
netlabel.h netlabel: Implement CALIPSO config functions for SMACK. 2016-06-27 15:06:18 -04:00
netlink.h netlink: Add nla_memdup() to wrap kmemdup() use on nlattr 2016-10-29 14:57:42 -04:00
netprio_cgroup.h
netrom.h
nexthop.h
nl802154.h ieee802154: add netns support 2016-07-08 12:20:57 +02:00
p8022.h
ping.h net: ping: make ping_v6_sendmsg static 2016-03-23 22:09:58 -04:00
pkt_cls.h net/sched: pkt_cls: change tc actions order to be as the user sets 2016-09-28 05:02:44 -04:00
pkt_sched.h net: sched: convert qdisc linked list to hashtable 2016-08-10 17:19:02 -07:00
pptp.h pptp: Refactor the struct and macros of PPTP codes 2016-08-15 10:55:53 -07:00
protocol.h udp: Remove udp_offloads 2016-04-07 16:53:30 -04:00
psnap.h
raw.h net: ip, diag -- Add diag interface for raw sockets 2016-10-23 19:35:24 -04:00
rawv6.h net: ip, diag -- Add diag interface for raw sockets 2016-10-23 19:35:24 -04:00
red.h
regulatory.h
request_sock.h inet: reqsk_alloc() needs to take care of dead listeners 2016-04-04 22:11:19 -04:00
rose.h
route.h net: inet: Support UID-based routing in IP protocols. 2016-11-04 14:45:23 -04:00
rtnetlink.h net: rtnetlink: add support for the IFLA_STATS_LINK_XSTATS_SLAVE attribute 2016-06-30 06:15:04 -04:00
sch_generic.h sched: add and use qdisc_skb_head helpers 2016-09-19 01:47:18 -04:00
scm.h
secure_seq.h
slhc_vj.h
snmp.h net: snmp: fix 64bit stats on 32bit arches 2016-04-28 11:49:45 -04:00
sock_reuseport.h
sock.h net: core: Add a UID field to struct sock. 2016-11-04 14:45:22 -04:00
Space.h
stp.h
strparser.h kcm: Remove TCP specific references from kcm and strparser 2016-08-28 23:32:41 -04:00
switchdev.h switchdev: remove FIB offload infrastructure 2016-09-28 04:48:00 -04:00
tcp_states.h
tcp.h net: Require exact match for TCP socket lookups if dif is l3mdev 2016-10-17 10:17:05 -04:00
timewait_sock.h
transp_v6.h ipv6: add new struct ipcm6_cookie 2016-05-03 16:08:14 -04:00
tso.h
udp_tunnel.h vxlan: Add new UDP encapsulation offload type for VXLAN-GPE 2016-06-17 20:23:32 -07:00
udp.h udp: do fwd memory scheduling on dequeue 2016-11-07 13:24:41 -05:00
udplite.h
vsock_addr.h
vxlan.h vxlan: avoid using stale vxlan socket. 2016-10-29 20:56:31 -04:00
wext.h
wimax.h
x25.h
x25device.h
xfrm.h Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2016-09-23 06:46:57 -04:00