forked from luck/tmp_suning_uos_patched
3f93aef535
Zstd compression requires different amounts of memory for each level of compression. The prior patches implemented indirection to allow for each compression type to manage their workspaces independently. This patch uses this indirection to implement compression level support for zstd. To manage the additional memory require, each compression level has its own queue of workspaces. A global LRU is used to help with reclaim. Reclaim is done via a timer which provides a mechanism to decrease memory utilization by keeping only workspaces around that are sized appropriately. Forward progress is guaranteed by a preallocated max workspace hidden from the LRU. When getting a workspace, it uses a bitmap to identify the levels that are populated and scans up. If it finds a workspace that is greater than it, it uses it, but does not update the last_used time and the corresponding place in the LRU. If we hit memory pressure, we sleep on the max level workspace. We continue to rescan in case we can use a smaller workspace, but eventually should be able to obtain the max level workspace or allocate one again should memory pressure subside. The memory requirement for decompression is the same as level 1, and therefore can use any of available workspace. The number of workspaces is bound by an upper limit of the workqueue's limit which currently is 2 (percpu limit). The reclaim timer is used to free inactive/improperly sized workspaces and is set to 307s to avoid colliding with transaction commit (every 30s). Repeating the experiment from v2 [1], the Silesia corpus was copied to a btrfs filesystem 10 times and then read back after dropping the caches. The btrfs filesystem was on an SSD. Level Ratio Compression (MB/s) Decompression (MB/s) Memory (KB) 1 2.658 438.47 910.51 780 2 2.744 364.86 886.55 1004 3 2.801 336.33 828.41 1260 4 2.858 286.71 886.55 1260 5 2.916 212.77 556.84 1388 6 2.363 119.82 990.85 1516 7 3.000 154.06 849.30 1516 8 3.011 159.54 875.03 1772 9 3.025 100.51 940.15 1772 10 3.033 118.97 616.26 1772 11 3.036 94.19 802.11 1772 12 3.037 73.45 931.49 1772 13 3.041 55.17 835.26 2284 14 3.087 44.70 716.78 2547 15 3.126 37.30 878.84 2547 [1] https://lore.kernel.org/linux-btrfs/20181031181108.289340-1-terrelln@fb.com/ Cc: Nick Terrell <terrelln@fb.com> Cc: Omar Sandoval <osandov@osandov.com> Signed-off-by: Dennis Zhou <dennis@kernel.org> Reviewed-by: David Sterba <dsterba@suse.com> Signed-off-by: David Sterba <dsterba@suse.com> |
||
|---|---|---|
| .. | ||
| tests | ||
| acl.c | ||
| async-thread.c | ||
| async-thread.h | ||
| backref.c | ||
| backref.h | ||
| btrfs_inode.h | ||
| check-integrity.c | ||
| check-integrity.h | ||
| compression.c | ||
| compression.h | ||
| ctree.c | ||
| ctree.h | ||
| dedupe.h | ||
| delayed-inode.c | ||
| delayed-inode.h | ||
| delayed-ref.c | ||
| delayed-ref.h | ||
| dev-replace.c | ||
| dev-replace.h | ||
| dir-item.c | ||
| disk-io.c | ||
| disk-io.h | ||
| export.c | ||
| export.h | ||
| extent_io.c | ||
| extent_io.h | ||
| extent_map.c | ||
| extent_map.h | ||
| extent-tree.c | ||
| file-item.c | ||
| file.c | ||
| free-space-cache.c | ||
| free-space-cache.h | ||
| free-space-tree.c | ||
| free-space-tree.h | ||
| inode-item.c | ||
| inode-map.c | ||
| inode-map.h | ||
| inode.c | ||
| ioctl.c | ||
| Kconfig | ||
| locking.c | ||
| locking.h | ||
| lzo.c | ||
| Makefile | ||
| math.h | ||
| ordered-data.c | ||
| ordered-data.h | ||
| orphan.c | ||
| print-tree.c | ||
| print-tree.h | ||
| props.c | ||
| props.h | ||
| qgroup.c | ||
| qgroup.h | ||
| raid56.c | ||
| raid56.h | ||
| rcu-string.h | ||
| reada.c | ||
| ref-verify.c | ||
| ref-verify.h | ||
| relocation.c | ||
| root-tree.c | ||
| scrub.c | ||
| send.c | ||
| send.h | ||
| struct-funcs.c | ||
| super.c | ||
| sysfs.c | ||
| sysfs.h | ||
| transaction.c | ||
| transaction.h | ||
| tree-checker.c | ||
| tree-checker.h | ||
| tree-defrag.c | ||
| tree-log.c | ||
| tree-log.h | ||
| ulist.c | ||
| ulist.h | ||
| uuid-tree.c | ||
| volumes.c | ||
| volumes.h | ||
| xattr.c | ||
| xattr.h | ||
| zlib.c | ||
| zstd.c | ||