kernel_optimize_test/block
Omar Sandoval 34dbad5d26 blk-stat: convert to callback-based statistics reporting
Currently, statistics are gathered in ~0.13s windows, and users grab the
statistics whenever they need them. This is not ideal for both in-tree
users:

1. Writeback throttling wants its own dynamically sized window of
   statistics. Since the blk-stats statistics are reset after every
   window and the wbt windows don't line up with the blk-stats windows,
   wbt doesn't see every I/O.
2. Polling currently grabs the statistics on every I/O. Again, depending
   on how the window lines up, we may miss some I/Os. It's also
   unnecessary overhead to get the statistics on every I/O; the hybrid
   polling heuristic would be just as happy with the statistics from the
   previous full window.

This reworks the blk-stats infrastructure to be callback-based: users
register a callback that they want called at a given time with all of
the statistics from the window during which the callback was active.
Users can dynamically bucketize the statistics. wbt and polling both
currently use read vs. write, but polling can be extended to further
subdivide based on request size.

The callbacks are kept on an RCU list, and each callback has percpu
stats buffers. There will only be a few users, so the overhead on the
I/O completion side is low. The stats flushing is also simplified
considerably: since the timer function is responsible for clearing the
statistics, we don't have to worry about stale statistics.

wbt is a trivial conversion. After the conversion, the windowing problem
mentioned above is fixed.

For polling, we register an extra callback that caches the previous
window's statistics in the struct request_queue for the hybrid polling
heuristic to use.

Since we no longer have a single stats buffer for the request queue,
this also removes the sysfs and debugfs stats entries. To replace those,
we add a debugfs entry for the poll statistics.

Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
2017-03-21 10:03:11 -06:00
..
partitions partitions/efi: Fix integer overflow in GPT size calculation 2017-01-17 09:02:31 -07:00
badblocks.c badblocks: badblocks_set/clear update unacked_exist 2016-10-21 15:45:47 -06:00
bio-integrity.c block: remove bio_is_rw 2016-10-28 08:45:17 -06:00
bio.c blk: Ensure users for current->bio_list can see the full list. 2017-03-11 15:31:37 -07:00
blk-cgroup.c sched/headers: Prepare to move signal wakeup & sigpending methods from <linux/sched.h> into <linux/sched/signal.h> 2017-03-02 08:42:32 +01:00
blk-core.c blk-stat: convert to callback-based statistics reporting 2017-03-21 10:03:11 -06:00
blk-exec.c block: introduce blk_rq_is_passthrough 2017-01-31 14:00:34 -07:00
blk-flush.c block: don't defer flushes on blk-mq + scheduling 2017-02-17 12:35:47 -07:00
blk-integrity.c block: Use pointer to backing_dev_info from request_queue 2017-02-02 08:20:48 -07:00
blk-ioc.c Merge branch 'for-linus' of git://git.kernel.dk/linux-block 2017-03-03 10:53:35 -08:00
blk-lib.c block: don't try Write Same from __blkdev_issue_zeroout 2017-02-06 09:34:46 -07:00
blk-map.c sched/headers: Prepare for new header dependencies before moving code to <linux/sched/task_stack.h> 2017-03-02 08:42:36 +01:00
blk-merge.c block: optionally merge discontiguous discard bios into a single request 2017-02-08 13:43:08 -07:00
blk-mq-cpumap.c blk-mq: export blk_mq_map_queues 2016-11-08 17:30:00 -05:00
blk-mq-debugfs.c blk-stat: convert to callback-based statistics reporting 2017-03-21 10:03:11 -06:00
blk-mq-pci.c blk_mq: linux/blk-mq.h does not include all the headers it depends on 2016-09-19 08:21:51 -06:00
blk-mq-sched.c blk-mq: move update of tags->rqs to __blk_mq_alloc_request() 2017-03-02 08:56:04 -07:00
blk-mq-sched.h blk-mq-sched: separate mark hctx and queue restart operations 2017-02-23 11:55:47 -07:00
blk-mq-sysfs.c blk-mq: free hctx->cpumask in release handler of hctx's kobject 2017-03-08 09:56:12 -07:00
blk-mq-tag.c blk-mq: Fix tagset reinit in the presence of cpu hot-unplug 2017-03-13 08:14:23 -06:00
blk-mq-tag.h blk-mq-sched: Allocate sched reserved tags as specified in the original queue tagset 2017-03-02 08:56:04 -07:00
blk-mq-virtio.c blk-mq: provide a default queue mapping for virtio device 2017-02-27 20:54:05 +02:00
blk-mq.c blk-stat: convert to callback-based statistics reporting 2017-03-21 10:03:11 -06:00
blk-mq.h blk-stat: convert to callback-based statistics reporting 2017-03-21 10:03:11 -06:00
blk-settings.c block: optionally merge discontiguous discard bios into a single request 2017-02-08 13:43:08 -07:00
blk-softirq.c sched/headers: Prepare for new header dependencies before moving code to <linux/sched/topology.h> 2017-03-02 08:42:26 +01:00
blk-stat.c blk-stat: convert to callback-based statistics reporting 2017-03-21 10:03:11 -06:00
blk-stat.h blk-stat: convert to callback-based statistics reporting 2017-03-21 10:03:11 -06:00
blk-sysfs.c blk-stat: convert to callback-based statistics reporting 2017-03-21 10:03:11 -06:00
blk-tag.c blk-mq-sched: add framework for MQ capable IO schedulers 2017-01-17 10:04:20 -07:00
blk-throttle.c scripts/spelling.txt: add "embeded" pattern and fix typo instances 2017-02-27 18:43:47 -08:00
blk-timeout.c block: remove REQ_NO_TIMEOUT flag 2015-12-22 09:38:34 -07:00
blk-wbt.c blk-stat: convert to callback-based statistics reporting 2017-03-21 10:03:11 -06:00
blk-wbt.h blk-stat: convert to callback-based statistics reporting 2017-03-21 10:03:11 -06:00
blk-zoned.c block: Rename blk_queue_zone_size and bdev_zone_size 2017-01-12 07:58:32 -07:00
blk.h block: optionally merge discontiguous discard bios into a single request 2017-02-08 13:43:08 -07:00
bounce.c Merge branch 'for-linus' of git://git.kernel.dk/linux-block 2015-09-19 18:57:09 -07:00
bsg-lib.c block: split scsi_request out of struct request 2017-01-27 15:08:35 -07:00
bsg.c lib/vsprintf.c: remove %Z support 2017-02-27 18:43:47 -08:00
cfq-iosched.c sched/headers: Prepare for new header dependencies before moving code to <linux/sched/clock.h> 2017-03-02 08:42:27 +01:00
cmdline-parser.c block: remove unrelated header files and export symbol 2014-01-21 20:18:26 -08:00
compat_ioctl.c block: Get rid of blk_get_backing_dev_info() 2017-02-02 08:21:32 -07:00
deadline-iosched.c block: enumify ELEVATOR_*_MERGE 2017-02-08 13:43:06 -07:00
elevator.c block: don't call ioc_exit_icq() with the queue lock held for blk-mq 2017-03-02 13:59:08 -07:00
genhd.c Revert "scsi, block: fix duplicate bdi name registration crashes" 2017-03-08 10:55:17 -07:00
ioctl.c block: Get rid of blk_get_backing_dev_info() 2017-02-02 08:21:32 -07:00
ioprio.c sched/headers: Prepare to move the task_lock()/unlock() APIs to <linux/sched/task.h> 2017-03-02 08:42:38 +01:00
Kconfig virtio, vhost: optimizations, fixes 2017-03-02 13:53:13 -08:00
Kconfig.iosched block: get rid of blk-mq default scheduler choice Kconfig entries 2017-02-22 13:19:45 -07:00
Makefile virtio, vhost: optimizations, fixes 2017-03-02 13:53:13 -08:00
mq-deadline.c block: enumify ELEVATOR_*_MERGE 2017-02-08 13:43:06 -07:00
noop-iosched.c block: move existing elevator ops to union 2017-01-17 10:03:33 -07:00
opal_proto.h block/sed-opal: allocate struct opal_dev dynamically 2017-02-17 12:41:47 -07:00
partition-generic.c block: Rename blk_queue_zone_size and bdev_zone_size 2017-01-12 07:58:32 -07:00
scsi_ioctl.c block: fold cmd_type into the REQ_OP_ space 2017-01-31 14:00:44 -07:00
sed-opal.c block/sed: Fix opal user range check and unused variables 2017-03-08 09:56:12 -07:00
t10-pi.c block: Consolidate static integrity profile properties 2015-10-21 14:42:38 -06:00