kernel_optimize_test/fs/xfs
David Chinner 92821e2ba4 [XFS] Lazy Superblock Counters
When we have a couple of hundred transactions on the fly at once, they all
typically modify the on disk superblock in some way.
create/unclink/mkdir/rmdir modify inode counts, allocation/freeing modify
free block counts.

When these counts are modified in a transaction, they must eventually lock
the superblock buffer and apply the mods. The buffer then remains locked
until the transaction is committed into the incore log buffer. The result
of this is that with enough transactions on the fly the incore superblock
buffer becomes a bottleneck.

The result of contention on the incore superblock buffer is that
transaction rates fall - the more pressure that is put on the superblock
buffer, the slower things go.

The key to removing the contention is to not require the superblock fields
in question to be locked. We do that by not marking the superblock dirty
in the transaction. IOWs, we modify the incore superblock but do not
modify the cached superblock buffer. In short, we do not log superblock
modifications to critical fields in the superblock on every transaction.
In fact we only do it just before we write the superblock to disk every
sync period or just before unmount.

This creates an interesting problem - if we don't log or write out the
fields in every transaction, then how do the values get recovered after a
crash? the answer is simple - we keep enough duplicate, logged information
in other structures that we can reconstruct the correct count after log
recovery has been performed.

It is the AGF and AGI structures that contain the duplicate information;
after recovery, we walk every AGI and AGF and sum their individual
counters to get the correct value, and we do a transaction into the log to
correct them. An optimisation of this is that if we have a clean unmount
record, we know the value in the superblock is correct, so we can avoid
the summation walk under normal conditions and so mount/recovery times do
not change under normal operation.

One wrinkle that was discovered during development was that the blocks
used in the freespace btrees are never accounted for in the AGF counters.
This was once a valid optimisation to make; when the filesystem is full,
the free space btrees are empty and consume no space. Hence when it
matters, the "accounting" is correct. But that means the when we do the
AGF summations, we would not have a correct count and xfs_check would
complain. Hence a new counter was added to track the number of blocks used
by the free space btrees. This is an *on-disk format change*.

As a result of this, lazy superblock counters are a mkfs option and at the
moment on linux there is no way to convert an old filesystem. This is
possible - xfs_db can be used to twiddle the right bits and then
xfs_repair will do the format conversion for you. Similarly, you can
convert backwards as well. At some point we'll add functionality to
xfs_admin to do the bit twiddling easily....

SGI-PV: 964999
SGI-Modid: xfs-linux-melb:xfs-kern:28652a

Signed-off-by: David Chinner <dgc@sgi.com>
Signed-off-by: Christoph Hellwig <hch@infradead.org>
Signed-off-by: Tim Shimmin <tes@sgi.com>
2007-07-14 15:28:50 +10:00
..
linux-2.6 [XFS] Lazy Superblock Counters 2007-07-14 15:28:50 +10:00
quota [XFS] Use generic shrinker interfaces in XFS. 2007-07-14 15:23:53 +10:00
support [XFS] reducing the number of random number functions. 2007-05-08 13:49:03 +10:00
Kbuild
Kconfig [PATCH] BLOCK: Make it possible to disable the block layer [try #6] 2006-09-30 20:52:31 +02:00
Makefile
Makefile-linux-2.6 [XFS] 956618: Linux crashes on boot with XFS-DMAPI filesystem when 2006-11-11 18:03:49 +11:00
xfs_acl.c [XFS] Remove unused header files for MAC and CAP checking functionality. 2007-02-10 18:37:28 +11:00
xfs_acl.h
xfs_ag.h [XFS] Lazy Superblock Counters 2007-07-14 15:28:50 +10:00
xfs_alloc_btree.c [XFS] Lazy Superblock Counters 2007-07-14 15:28:50 +10:00
xfs_alloc_btree.h [XFS] Remove unused arguments from the XFS_BTREE_*_ADDR macros. 2007-02-10 18:37:33 +11:00
xfs_alloc.c [XFS] Lazy Superblock Counters 2007-07-14 15:28:50 +10:00
xfs_alloc.h [XFS] Lazy Superblock Counters 2007-07-14 15:28:50 +10:00
xfs_arch.h
xfs_attr_leaf.c [XFS] The last argument "lsn" of xfs_trans_commit() is always called with 2007-05-08 13:48:42 +10:00
xfs_attr_leaf.h
xfs_attr_sf.h
xfs_attr.c [XFS] The last argument "lsn" of xfs_trans_commit() is always called with 2007-05-08 13:48:42 +10:00
xfs_attr.h
xfs_behavior.c
xfs_behavior.h
xfs_bit.c [XFS] Keep stack usage down for 4k stacks by using noinline. 2007-02-10 18:34:56 +11:00
xfs_bit.h
xfs_bmap_btree.c [XFS] Remove a bunch of unused functions from XFS. 2007-02-10 18:37:40 +11:00
xfs_bmap_btree.h [XFS] Remove a bunch of unused functions from XFS. 2007-02-10 18:37:40 +11:00
xfs_bmap.c [XFS] Fix to prevent the notorious 'NULL files' problem after a crash. 2007-05-08 13:49:46 +10:00
xfs_bmap.h [XFS] Remove unused argument to xfs_bmap_finish 2007-02-10 18:37:16 +11:00
xfs_btree.c
xfs_btree.h [XFS] Remove unused arguments from the XFS_BTREE_*_ADDR macros. 2007-02-10 18:37:33 +11:00
xfs_buf_item.c [XFS] Keep stack usage down for 4k stacks by using noinline. 2007-02-10 18:34:56 +11:00
xfs_buf_item.h [XFS] Get rid of old 5.3/6.1 v1 log items. Cleanup patch sent in by Eric 2007-02-10 18:35:02 +11:00
xfs_clnt.h
xfs_da_btree.c [XFS] Remove a bunch of unused functions from XFS. 2007-02-10 18:37:40 +11:00
xfs_da_btree.h [XFS] Remove a bunch of unused functions from XFS. 2007-02-10 18:37:40 +11:00
xfs_dfrag.c [XFS] propogate return codes from flush routines 2007-05-08 13:49:27 +10:00
xfs_dfrag.h
xfs_dinode.h
xfs_dir2_block.c [XFS] the "aendp" arg to xfs_dir2_data_freescan is always NULL, remove it. 2007-05-08 13:48:49 +10:00
xfs_dir2_block.h
xfs_dir2_data.c [XFS] the "aendp" arg to xfs_dir2_data_freescan is always NULL, remove it. 2007-05-08 13:48:49 +10:00
xfs_dir2_data.h [XFS] the "aendp" arg to xfs_dir2_data_freescan is always NULL, remove it. 2007-05-08 13:48:49 +10:00
xfs_dir2_leaf.c [XFS] the "aendp" arg to xfs_dir2_data_freescan is always NULL, remove it. 2007-05-08 13:48:49 +10:00
xfs_dir2_leaf.h
xfs_dir2_node.c [XFS] the "aendp" arg to xfs_dir2_data_freescan is always NULL, remove it. 2007-05-08 13:48:49 +10:00
xfs_dir2_node.h
xfs_dir2_sf.c
xfs_dir2_sf.h
xfs_dir2_trace.c
xfs_dir2_trace.h
xfs_dir2.c [XFS] rename uio_read() to xfs_uio_read() 2006-11-11 18:04:41 +11:00
xfs_dir2.h
xfs_dmapi.h [XFS] Remove KERNEL_VERSION macros from xfs_dmapi.h 2006-11-11 18:05:06 +11:00
xfs_dmops.c
xfs_error.c [XFS] reducing the number of random number functions. 2007-05-08 13:49:03 +10:00
xfs_error.h [XFS] Remove a bunch of unused functions from XFS. 2007-02-10 18:37:40 +11:00
xfs_extfree_item.c [XFS] Keep stack usage down for 4k stacks by using noinline. 2007-02-10 18:34:56 +11:00
xfs_extfree_item.h
xfs_fs.h [XFS] Lazy Superblock Counters 2007-07-14 15:28:50 +10:00
xfs_fsops.c [XFS] Lazy Superblock Counters 2007-07-14 15:28:50 +10:00
xfs_fsops.h
xfs_ialloc_btree.c
xfs_ialloc_btree.h [XFS] Remove unused arguments from the XFS_BTREE_*_ADDR macros. 2007-02-10 18:37:33 +11:00
xfs_ialloc.c [XFS] Lazy Superblock Counters 2007-07-14 15:28:50 +10:00
xfs_ialloc.h [XFS] Lazy Superblock Counters 2007-07-14 15:28:50 +10:00
xfs_iget.c [XFS] Add lockdep support for XFS 2007-05-08 13:50:19 +10:00
xfs_imap.h
xfs_inode_item.c [XFS] Keep stack usage down for 4k stacks by using noinline. 2007-02-10 18:34:56 +11:00
xfs_inode_item.h
xfs_inode.c [XFS] Fix to prevent the notorious 'NULL files' problem after a crash. 2007-05-08 13:49:46 +10:00
xfs_inode.h [XFS] Add lockdep support for XFS 2007-05-08 13:50:19 +10:00
xfs_inum.h
xfs_iocore.c [XFS] Fix to prevent the notorious 'NULL files' problem after a crash. 2007-05-08 13:49:46 +10:00
xfs_iomap.c [XFS] Fix to prevent the notorious 'NULL files' problem after a crash. 2007-05-08 13:49:46 +10:00
xfs_iomap.h [XFS] Fix to prevent the notorious 'NULL files' problem after a crash. 2007-05-08 13:49:46 +10:00
xfs_itable.c Fix occurrences of "the the " 2007-05-09 08:57:56 +02:00
xfs_itable.h [XFS] pv 956241, author: nathans, rv: vapo - make ino validation checks 2006-09-28 11:06:15 +10:00
xfs_log_priv.h [XFS] Fixes the leak in reservation space because we weren't ungranting 2006-09-28 11:04:16 +10:00
xfs_log_recover.c [XFS] Lazy Superblock Counters 2007-07-14 15:28:50 +10:00
xfs_log_recover.h
xfs_log.c [XFS] Lazy Superblock Counters 2007-07-14 15:28:50 +10:00
xfs_log.h
xfs_mount.c [XFS] Lazy Superblock Counters 2007-07-14 15:28:50 +10:00
xfs_mount.h [XFS] Lazy Superblock Counters 2007-07-14 15:28:50 +10:00
xfs_qmops.c [XFS] The last argument "lsn" of xfs_trans_commit() is always called with 2007-05-08 13:48:42 +10:00
xfs_quota.h [XFS] Fix uquota and oquota enforcement problems. 2007-05-08 13:49:33 +10:00
xfs_refcache.h
xfs_rename.c [XFS] The last argument "lsn" of xfs_trans_commit() is always called with 2007-05-08 13:48:42 +10:00
xfs_rtalloc.c [XFS] Don't grow filesystems past the size they can index. 2007-07-14 15:21:29 +10:00
xfs_rtalloc.h [XFS] Remove a bunch of unused functions from XFS. 2007-02-10 18:37:40 +11:00
xfs_rw.c [XFS] The last argument "lsn" of xfs_trans_commit() is always called with 2007-05-08 13:48:42 +10:00
xfs_rw.h [XFS] Make hole punching at EOF atomic. 2007-07-14 15:23:40 +10:00
xfs_sb.h [XFS] Lazy Superblock Counters 2007-07-14 15:28:50 +10:00
xfs_trans_ail.c [XFS] Workaround log space issue by increasing XFS_TRANS_PUSH_AIL_RESTARTS 2007-02-10 18:35:52 +11:00
xfs_trans_buf.c
xfs_trans_extfree.c
xfs_trans_inode.c
xfs_trans_item.c
xfs_trans_priv.h [XFS] Add lock annotations to xfs_trans_update_ail and 2006-09-28 11:04:07 +10:00
xfs_trans_space.h
xfs_trans.c [XFS] Lazy Superblock Counters 2007-07-14 15:28:50 +10:00
xfs_trans.h [XFS] Lazy Superblock Counters 2007-07-14 15:28:50 +10:00
xfs_types.h
xfs_utils.c [XFS] propogate return codes from flush routines 2007-05-08 13:49:27 +10:00
xfs_utils.h
xfs_vfsops.c [XFS] Lazy Superblock Counters 2007-07-14 15:28:50 +10:00
xfs_vnodeops.c [XFS] Make hole punching at EOF atomic. 2007-07-14 15:23:40 +10:00
xfs.h [XFS] 956618: Linux crashes on boot with XFS-DMAPI filesystem when 2006-11-11 18:03:49 +11:00