kernel_optimize_test/samples/bpf
brakmo 71634d7f92 bpf: Add support for fq's EDT to HBM
Adds support for fq's Earliest Departure Time to HBM (Host Bandwidth
Manager). Includes a new BPF program supporting EDT, and also updates
corresponding programs.

It will drop packets with an EDT of more than 500us in the future
unless the packet belongs to a flow with less than 2 packets in flight.
This is done so each flow has at least 2 packets in flight, so they
will not starve, and also to help prevent delayed ACK timeouts.

It will also work with ECN enabled traffic, where the packets will be
CE marked if their EDT is more than 50us in the future.

The table below shows some performance numbers. The flows are back to
back RPCS. One server sending to another, either 2 or 4 flows.
One flow is a 10KB RPC, the rest are 1MB RPCs. When there are more
than one flow of a given RPC size, the numbers represent averages.

The rate limit applies to all flows (they are in the same cgroup).
Tests ending with "-edt" ran with the new BPF program supporting EDT.
Tests ending with "-hbt" ran on top HBT qdisc with the specified rate
(i.e. no HBM). The other tests ran with the HBM BPF program included
in the HBM patch-set.

EDT has limited value when using DCTCP, but it helps in many cases when
using Cubic. It usually achieves larger link utilization and lower
99% latencies for the 1MB RPCs.
HBM ends up queueing a lot of packets with its default parameter values,
reducing the goodput of the 10KB RPCs and increasing their latency. Also,
the RTTs seen by the flows are quite large.

                         Aggr              10K  10K  10K   1MB  1MB  1MB
         Limit           rate drops  RTT  rate  P90  P99  rate  P90  P99
Test      rate  Flows    Mbps   %     us  Mbps   us   us  Mbps   ms   ms
--------  ----  -----    ---- -----  ---  ---- ---- ----  ---- ---- ----
cubic       1G    2       904  0.02  108   257  511  539   647 13.4 24.5
cubic-edt   1G    2       982  0.01  156   239  656  967   743 14.0 17.2
dctcp       1G    2       977  0.00  105   324  408  744   653 14.5 15.9
dctcp-edt   1G    2       981  0.01  142   321  417  811   660 15.7 17.0
cubic-htb   1G    2       919  0.00 1825    40 2822 4140   879  9.7  9.9

cubic     200M    2       155  0.30  220    81  532  655    74  283  450
cubic-edt 200M    2       188  0.02  222    87 1035 1095   101   84   85
dctcp     200M    2       188  0.03  111    77  912  939   111   76  325
dctcp-edt 200M    2       188  0.03  217    74 1416 1738   114   76   79
cubic-htb 200M    2       188  0.00 5015     8 14ms 15ms   180   48   50

cubic       1G    4       952  0.03  110   165  516  546   262   38  154
cubic-edt   1G    4       973  0.01  190   111 1034 1314   287   65   79
dctcp       1G    4       951  0.00  103   180  617  905   257   37   38
dctcp-edt   1G    4       967  0.00  163   151  732 1126   272   43   55
cubic-htb   1G    4       914  0.00 3249    13  7ms  8ms   300   29   34

cubic       5G    4      4236  0.00  134   305  490  624  1310   10   17
cubic-edt   5G    4      4865  0.00  156   306  425  759  1520   10   16
dctcp       5G    4      4936  0.00  128   485  221  409  1484    7    9
dctcp-edt   5G    4      4924  0.00  148   390  392  623  1508   11   26

v1 -> v2: Incorporated Andrii's suggestions
v2 -> v3: Incorporated Yonghong's suggestions
v3 -> v4: Removed credit update that is not needed

Signed-off-by: Lawrence Brakmo <brakmo@fb.com>
Acked-by: Yonghong Song <yhs@fb.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
2019-07-03 15:03:00 +02:00
..
.gitignore samples: bpf: add ibumad sample to .gitignore 2019-05-24 19:58:03 -07:00
asm_goto_workaround.h samples/bpf: fix build with new clang 2019-04-05 16:28:36 +02:00
bpf_insn.h selftests: bpf: functional and min/max reasoning unit tests for JMP32 2019-01-26 13:33:02 -08:00
bpf_load.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2019-06-17 20:20:36 -07:00
bpf_load.h samples: bpf: include bpf/bpf.h instead of local libbpf.h 2018-05-14 22:52:10 -07:00
cookie_uid_helper_example.c samples: bpf: rename libbpf.h to bpf_insn.h 2018-05-14 22:52:10 -07:00
cpustat_kern.c samples/bpf: Add program for CPU state statistics 2018-02-26 10:54:02 +01:00
cpustat_user.c samples: bpf: include bpf/bpf.h instead of local libbpf.h 2018-05-14 22:52:10 -07:00
do_hbm_test.sh bpf: Add support for fq's EDT to HBM 2019-07-03 15:03:00 +02:00
fds_example.c samples: bpf: refactor header include path 2019-06-18 00:28:36 +02:00
hash_func01.h samples/bpf: add Paul Hsieh's (LGPL 2.1) hash function SuperFastHash 2018-08-10 16:07:49 +02:00
hbm_edt_kern.c bpf: Add support for fq's EDT to HBM 2019-07-03 15:03:00 +02:00
hbm_kern.h bpf: Add support for fq's EDT to HBM 2019-07-03 15:03:00 +02:00
hbm_out_kern.c bpf: Add more stats to HBM 2019-05-31 16:41:29 -07:00
hbm.c bpf: Add support for fq's EDT to HBM 2019-07-03 15:03:00 +02:00
hbm.h bpf: Add more stats to HBM 2019-05-31 16:41:29 -07:00
ibumad_kern.c samples: bpf: Remove bpf_debug macro in favor of bpf_printk 2019-06-24 18:18:30 -07:00
ibumad_user.c samples: bpf: refactor header include path 2019-06-18 00:28:36 +02:00
lathist_kern.c bpf: BPF based latency tracing 2015-06-23 06:09:58 -07:00
lathist_user.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 206 2019-05-30 11:29:53 -07:00
lwt_len_hist_kern.c bpf: Add tests and samples for LWT-BPF 2016-12-02 10:52:00 -05:00
lwt_len_hist_user.c samples: bpf: include bpf/bpf.h instead of local libbpf.h 2018-05-14 22:52:10 -07:00
lwt_len_hist.sh License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
Makefile bpf: Add support for fq's EDT to HBM 2019-07-03 15:03:00 +02:00
map_perf_test_kern.c samples/bpf: Use getppid instead of getpgrp for array map stress 2017-09-21 11:59:16 -07:00
map_perf_test_user.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 206 2019-05-30 11:29:53 -07:00
offwaketime_kern.c bpf, samples: don't zero data when not needed 2016-04-14 21:40:42 -04:00
offwaketime_user.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 206 2019-05-30 11:29:53 -07:00
parse_ldabs.c bpf: fix samples to add fake KBUILD_MODNAME 2016-10-29 14:46:12 -04:00
parse_simple.c bpf: fix samples to add fake KBUILD_MODNAME 2016-10-29 14:46:12 -04:00
parse_varlen.c samples/bpf: add missing <linux/if_vlan.h> 2018-07-05 09:58:52 +02:00
README.rst samples/bpf: Add documentation on cross compilation 2017-09-21 11:59:16 -07:00
run_cookie_uid_helper_example.sh License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
sampleip_kern.c bpf/samples: Fix PT_REGS_IP on s390x and use it 2016-11-28 16:26:46 -05:00
sampleip_user.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 206 2019-05-30 11:29:53 -07:00
sock_example.c samples: bpf: force IPv4 in ping 2019-03-01 00:53:45 +01:00
sock_example.h samples: bpf: include bpf/bpf.h instead of local libbpf.h 2018-05-14 22:52:10 -07:00
sock_flags_kern.c samples/bpf: Update cgroup socket examples to use uid gid helper 2017-09-01 06:05:15 +01:00
sockex1_kern.c samples: bpf: add skb->field examples and tests 2015-03-15 22:02:28 -04:00
sockex1_user.c samples: bpf: refactor header include path 2019-06-18 00:28:36 +02:00
sockex2_kern.c samples/bpf: fix compilation failure 2018-09-21 22:51:16 +02:00
sockex2_user.c samples: bpf: refactor header include path 2019-06-18 00:28:36 +02:00
sockex3_kern.c samples/bpf: fix compilation failure 2018-09-21 22:51:16 +02:00
sockex3_user.c samples: bpf: force IPv4 in ping 2019-03-01 00:53:45 +01:00
spintest_kern.c samples/bpf: Enable powerpc support 2016-04-06 16:01:29 -04:00
spintest_user.c samples, selftests/bpf: add NULL check for ksym_search 2019-04-04 16:43:47 +02:00
syscall_nrs.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
syscall_tp_kern.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 206 2019-05-30 11:29:53 -07:00
syscall_tp_user.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 206 2019-05-30 11:29:53 -07:00
task_fd_query_kern.c samples: bpf: fix: broken sample regarding removed function 2019-02-27 17:27:22 +01:00
task_fd_query_user.c samples, bpf: suppress compiler warning 2019-05-21 16:55:20 +02:00
tc_l2_redirect_kern.c bpf: fix samples xdp_tx_iptunnel and tc_l2_redirect with fake KBUILD_MODNAME 2017-01-20 12:04:07 -05:00
tc_l2_redirect_user.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 206 2019-05-30 11:29:53 -07:00
tc_l2_redirect.sh License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
tcbpf1_kern.c bpf: fix samples to add fake KBUILD_MODNAME 2016-10-29 14:46:12 -04:00
tcp_basertt_kern.c samples: bpf: Do not define bpf_printk macro 2019-05-24 13:47:17 -07:00
tcp_bpf.readme samples: bpf: remove load_sock_ops in favour of bpftool 2019-03-01 00:53:45 +01:00
tcp_bufs_kern.c samples: bpf: Do not define bpf_printk macro 2019-05-24 13:47:17 -07:00
tcp_clamp_kern.c samples: bpf: Do not define bpf_printk macro 2019-05-24 13:47:17 -07:00
tcp_cong_kern.c samples: bpf: Do not define bpf_printk macro 2019-05-24 13:47:17 -07:00
tcp_iw_kern.c samples: bpf: Do not define bpf_printk macro 2019-05-24 13:47:17 -07:00
tcp_rwnd_kern.c samples: bpf: Do not define bpf_printk macro 2019-05-24 13:47:17 -07:00
tcp_synrto_kern.c samples: bpf: Do not define bpf_printk macro 2019-05-24 13:47:17 -07:00
tcp_tos_reflect_kern.c samples: bpf: Do not define bpf_printk macro 2019-05-24 13:47:17 -07:00
test_cgrp2_array_pin.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 206 2019-05-30 11:29:53 -07:00
test_cgrp2_attach.c samples: bpf: rename libbpf.h to bpf_insn.h 2018-05-14 22:52:10 -07:00
test_cgrp2_sock.c samples: bpf: rename libbpf.h to bpf_insn.h 2018-05-14 22:52:10 -07:00
test_cgrp2_sock.sh samples/bpf: detach prog from cgroup 2018-03-02 00:16:36 +01:00
test_cgrp2_sock2.c samples/bpf: test_cgrp2_sock2: fix an off by one 2018-07-16 15:01:09 -07:00
test_cgrp2_sock2.sh samples/bpf: detach prog from cgroup 2018-03-02 00:16:36 +01:00
test_cgrp2_tc_kern.c bpf: fix samples to add fake KBUILD_MODNAME 2016-10-29 14:46:12 -04:00
test_cgrp2_tc.sh License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
test_cls_bpf.sh License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
test_current_task_under_cgroup_kern.c samples/bpf: Add test_current_task_under_cgroup test 2016-08-12 21:49:42 -07:00
test_current_task_under_cgroup_user.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 206 2019-05-30 11:29:53 -07:00
test_ipip.sh License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
test_lru_dist.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 206 2019-05-30 11:29:53 -07:00
test_lwt_bpf.c bpf: Add tests and samples for LWT-BPF 2016-12-02 10:52:00 -05:00
test_lwt_bpf.sh License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
test_map_in_map_kern.c bpf: Add tests for map-in-map 2017-03-22 15:45:45 -07:00
test_map_in_map_user.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 206 2019-05-30 11:29:53 -07:00
test_overhead_kprobe_kern.c samples/bpf: add tracepoint vs kprobe performance tests 2016-04-07 21:04:27 -04:00
test_overhead_raw_tp_kern.c samples/bpf: raw tracepoint test 2018-03-28 22:55:19 +02:00
test_overhead_tp_kern.c samples/bpf: add tracepoint vs kprobe performance tests 2016-04-07 21:04:27 -04:00
test_overhead_user.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 206 2019-05-30 11:29:53 -07:00
test_override_return.sh samples/bpf: add a test for bpf_override_return 2017-12-12 09:02:40 -08:00
test_probe_write_user_kern.c samples/bpf: Add test/example of using bpf_probe_write_user bpf helper 2016-07-25 18:07:48 -07:00
test_probe_write_user_user.c samples: bpf: include bpf/bpf.h instead of local libbpf.h 2018-05-14 22:52:10 -07:00
trace_event_kern.c samples/bpf: add example to test reading address 2018-03-08 02:22:34 +01:00
trace_event_user.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 206 2019-05-30 11:29:53 -07:00
trace_output_kern.c samples/bpf: fix trace_output example 2016-04-28 17:29:45 -04:00
trace_output_user.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 206 2019-05-30 11:29:53 -07:00
tracex1_kern.c bpf, samples: don't zero data when not needed 2016-04-14 21:40:42 -04:00
tracex1_user.c samples: bpf: include bpf/bpf.h instead of local libbpf.h 2018-05-14 22:52:10 -07:00
tracex2_kern.c bpf, samples: don't zero data when not needed 2016-04-14 21:40:42 -04:00
tracex2_user.c samples: bpf: force IPv4 in ping 2019-03-01 00:53:45 +01:00
tracex3_kern.c samples: bpf: fix: broken sample regarding removed function 2019-02-27 17:27:22 +01:00
tracex3_user.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 206 2019-05-30 11:29:53 -07:00
tracex4_kern.c samples/bpf: Enable powerpc support 2016-04-06 16:01:29 -04:00
tracex4_user.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 206 2019-05-30 11:29:53 -07:00
tracex5_kern.c samples/bpf: Fix tracex5 to work with MIPS syscalls. 2017-06-14 15:03:23 -04:00
tracex5_user.c samples: bpf: include bpf/bpf.h instead of local libbpf.h 2018-05-14 22:52:10 -07:00
tracex6_kern.c bpf: add a test case for helper bpf_perf_event_read_value 2017-10-07 23:05:57 +01:00
tracex6_user.c samples: bpf: include bpf/bpf.h instead of local libbpf.h 2018-05-14 22:52:10 -07:00
tracex7_kern.c samples/bpf: add a test for bpf_override_return 2017-12-12 09:02:40 -08:00
tracex7_user.c samples: bpf: include bpf/bpf.h instead of local libbpf.h 2018-05-14 22:52:10 -07:00
xdp_adjust_tail_kern.c bpf: add bpf_xdp_adjust_tail sample prog 2018-04-18 23:34:17 +02:00
xdp_adjust_tail_user.c samples: bpf: make the use of xdp samples consistent 2019-06-26 15:39:15 +02:00
xdp_fwd_kern.c bpf: Change bpf_fib_lookup to return lookup status 2018-06-29 00:02:02 +02:00
xdp_fwd_user.c samples: bpf: refactor header include path 2019-06-18 00:28:36 +02:00
xdp_monitor_kern.c samples/bpf: xdp_monitor use err code from tracepoint xdp:xdp_devmap_xmit 2018-05-24 18:36:15 -07:00
xdp_monitor_user.c samples/bpf: xdp_monitor use err code from tracepoint xdp:xdp_devmap_xmit 2018-05-24 18:36:15 -07:00
xdp_redirect_cpu_kern.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next 2018-08-13 10:07:23 -07:00
xdp_redirect_cpu_user.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2019-06-22 08:59:24 -04:00
xdp_redirect_kern.c samples/bpf: Fix compilation issue in redirect dummy program 2017-08-31 11:56:57 -07:00
xdp_redirect_map_kern.c samples/bpf: Fix compilation issue in redirect dummy program 2017-08-31 11:56:57 -07:00
xdp_redirect_map_user.c samples: bpf: make the use of xdp samples consistent 2019-06-26 15:39:15 +02:00
xdp_redirect_user.c samples: bpf: make the use of xdp samples consistent 2019-06-26 15:39:15 +02:00
xdp_router_ipv4_kern.c xdp: Sample xdp program implementing ip forward 2017-11-08 10:39:41 +09:00
xdp_router_ipv4_user.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next 2019-06-20 00:06:27 -04:00
xdp_rxq_info_kern.c samples/bpf: xdp_rxq_info action XDP_TX must adjust MAC-addrs 2018-06-28 23:50:20 +02:00
xdp_rxq_info_user.c samples: bpf: refactor header include path 2019-06-18 00:28:36 +02:00
xdp_sample_pkts_kern.c samples: bpf: Do not define bpf_printk macro 2019-05-24 13:47:17 -07:00
xdp_sample_pkts_user.c samples/bpf: Check the prog id before exiting 2019-02-01 23:37:51 +01:00
xdp_tx_iptunnel_common.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 206 2019-05-30 11:29:53 -07:00
xdp_tx_iptunnel_kern.c bpf: fix samples xdp_tx_iptunnel and tc_l2_redirect with fake KBUILD_MODNAME 2017-01-20 12:04:07 -05:00
xdp_tx_iptunnel_user.c samples: bpf: make the use of xdp samples consistent 2019-06-26 15:39:15 +02:00
xdp1_kern.c bpf: make xdp sample variable names more meaningful 2016-07-20 22:07:24 -07:00
xdp1_user.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next 2019-06-20 00:06:27 -04:00
xdp2_kern.c bpf: make xdp sample variable names more meaningful 2016-07-20 22:07:24 -07:00
xdp2skb_meta_kern.c samples/bpf: xdp2skb_meta comment explain why pkt-data pointers are invalidated 2018-01-18 01:49:09 +01:00
xdp2skb_meta.sh samples/bpf: Fix tc and ip paths in xdp2skb_meta.sh 2018-07-10 09:19:01 +02:00
xdpsock_user.c xsk: Change the default frame size to 4096 and allow controlling it 2019-06-27 22:53:26 +02:00

eBPF sample programs
====================

This directory contains a test stubs, verifier test-suite and examples
for using eBPF. The examples use libbpf from tools/lib/bpf.

Build dependencies
==================

Compiling requires having installed:
 * clang >= version 3.4.0
 * llvm >= version 3.7.1

Note that LLVM's tool 'llc' must support target 'bpf', list version
and supported targets with command: ``llc --version``

Kernel headers
--------------

There are usually dependencies to header files of the current kernel.
To avoid installing devel kernel headers system wide, as a normal
user, simply call::

 make headers_install

This will creates a local "usr/include" directory in the git/build top
level directory, that the make system automatically pickup first.

Compiling
=========

For building the BPF samples, issue the below command from the kernel
top level directory::

 make samples/bpf/

Do notice the "/" slash after the directory name.

It is also possible to call make from this directory.  This will just
hide the the invocation of make as above with the appended "/".

Manually compiling LLVM with 'bpf' support
------------------------------------------

Since version 3.7.0, LLVM adds a proper LLVM backend target for the
BPF bytecode architecture.

By default llvm will build all non-experimental backends including bpf.
To generate a smaller llc binary one can use::

 -DLLVM_TARGETS_TO_BUILD="BPF"

Quick sniplet for manually compiling LLVM and clang
(build dependencies are cmake and gcc-c++)::

 $ git clone http://llvm.org/git/llvm.git
 $ cd llvm/tools
 $ git clone --depth 1 http://llvm.org/git/clang.git
 $ cd ..; mkdir build; cd build
 $ cmake .. -DLLVM_TARGETS_TO_BUILD="BPF;X86"
 $ make -j $(getconf _NPROCESSORS_ONLN)

It is also possible to point make to the newly compiled 'llc' or
'clang' command via redefining LLC or CLANG on the make command line::

 make samples/bpf/ LLC=~/git/llvm/build/bin/llc CLANG=~/git/llvm/build/bin/clang

Cross compiling samples
-----------------------
In order to cross-compile, say for arm64 targets, export CROSS_COMPILE and ARCH
environment variables before calling make. This will direct make to build
samples for the cross target.

export ARCH=arm64
export CROSS_COMPILE="aarch64-linux-gnu-"
make samples/bpf/ LLC=~/git/llvm/build/bin/llc CLANG=~/git/llvm/build/bin/clang