kernel_optimize_test/arch/sh/include/asm
Linus Torvalds 2b047252d0 Fix TLB gather virtual address range invalidation corner cases
Ben Tebulin reported:

 "Since v3.7.2 on two independent machines a very specific Git
  repository fails in 9/10 cases on git-fsck due to an SHA1/memory
  failures.  This only occurs on a very specific repository and can be
  reproduced stably on two independent laptops.  Git mailing list ran
  out of ideas and for me this looks like some very exotic kernel issue"

and bisected the failure to the backport of commit 53a59fc67f ("mm:
limit mmu_gather batching to fix soft lockups on !CONFIG_PREEMPT").

That commit itself is not actually buggy, but what it does is to make it
much more likely to hit the partial TLB invalidation case, since it
introduces a new case in tlb_next_batch() that previously only ever
happened when running out of memory.

The real bug is that the TLB gather virtual memory range setup is subtly
buggered.  It was introduced in commit 597e1c3580 ("mm/mmu_gather:
enable tlb flush range in generic mmu_gather"), and the range handling
was already fixed at least once in commit e6c495a96c ("mm: fix the TLB
range flushed when __tlb_remove_page() runs out of slots"), but that fix
was not complete.

The problem with the TLB gather virtual address range is that it isn't
set up by the initial tlb_gather_mmu() initialization (which didn't get
the TLB range information), but it is set up ad-hoc later by the
functions that actually flush the TLB.  And so any such case that forgot
to update the TLB range entries would potentially miss TLB invalidates.

Rather than try to figure out exactly which particular ad-hoc range
setup was missing (I personally suspect it's the hugetlb case in
zap_huge_pmd(), which didn't have the same logic as zap_pte_range()
did), this patch just gets rid of the problem at the source: make the
TLB range information available to tlb_gather_mmu(), and initialize it
when initializing all the other tlb gather fields.

This makes the patch larger, but conceptually much simpler.  And the end
result is much more understandable; even if you want to play games with
partial ranges when invalidating the TLB contents in chunks, now the
range information is always there, and anybody who doesn't want to
bother with it won't introduce subtle bugs.

Ben verified that this fixes his problem.

Reported-bisected-and-tested-by: Ben Tebulin <tebulin@googlemail.com>
Build-testing-by: Stephen Rothwell <sfr@canb.auug.org.au>
Build-testing-by: Richard Weinberger <richard.weinberger@gmail.com>
Reviewed-by: Michal Hocko <mhocko@suse.cz>
Acked-by: Peter Zijlstra <peterz@infradead.org>
Cc: stable@vger.kernel.org
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2013-08-16 08:52:46 -07:00
..
adc.h
addrspace.h
alignment.h
asm-offsets.h
atomic-grb.h
atomic-irq.h Disintegrate asm/system.h for SH 2012-03-28 18:30:03 +01:00
atomic-llsc.h
atomic.h sh: Fix up tracepoint build fallout from static key introduction. 2012-04-27 11:12:38 +09:30
barrier.h Disintegrate asm/system.h for SH 2012-03-28 18:30:03 +01:00
bitops-grb.h
bitops-llsc.h
bitops-op32.h
bitops.h Disintegrate asm/system.h for SH 2012-03-28 18:30:03 +01:00
bl_bit_32.h Disintegrate asm/system.h for SH 2012-03-28 18:30:03 +01:00
bl_bit_64.h Disintegrate asm/system.h for SH 2012-03-28 18:30:03 +01:00
bl_bit.h UAPI: (Scripted) Convert #include "..." to #include <path/...> in kernel system headers 2012-10-02 18:01:25 +01:00
bug.h sh: Consolidate die definitions for trap handlers. 2012-06-14 14:18:51 +09:00
bugs.h
cache_insns_32.h Disintegrate asm/system.h for SH 2012-03-28 18:30:03 +01:00
cache_insns_64.h Disintegrate asm/system.h for SH 2012-03-28 18:30:03 +01:00
cache_insns.h UAPI: (Scripted) Convert #include "..." to #include <path/...> in kernel system headers 2012-10-02 18:01:25 +01:00
cache.h
cacheflush.h
checksum_32.h
checksum.h UAPI: (Scripted) Convert #include "..." to #include <path/...> in kernel system headers 2012-10-02 18:01:25 +01:00
clkdev.h
clock.h sh: sh header sh_clk_ops rename 2012-03-12 22:19:10 +01:00
cmpxchg-grb.h
cmpxchg-irq.h Disintegrate asm/system.h for SH 2012-03-28 18:30:03 +01:00
cmpxchg-llsc.h
cmpxchg.h Disintegrate asm/system.h for SH 2012-03-28 18:30:03 +01:00
device.h sh: Defer to asm-generic/device.h. 2012-01-13 16:27:38 +09:00
dma-mapping.h sh: dma_debug: add debug_dma_mapping_error support 2012-11-28 15:29:00 +01:00
dma-register.h
dma.h sh: Kill off MAX_DMA_ADDRESS leftovers. 2012-05-19 18:50:09 +09:00
dmabrg.h
dwarf.h
elf.h sh: Fix FDPIC binary loader 2013-01-11 21:02:57 +09:00
entry-macros.S
fb.h
fixmap.h Revert "sh: Ensure fixmap and store queue space can co-exist." 2012-05-18 19:30:05 +09:00
flat.h
fpu.h
freq.h
ftrace.h
futex-irq.h Disintegrate asm/system.h for SH 2012-03-28 18:30:03 +01:00
futex.h
gpio.h sh: Remove unused sh_pfc_register_info() function 2013-01-25 09:24:30 +09:00
hardirq.h
hd64461.h
heartbeat.h
hugetlb.h mm/hugetlb: add more arch-defined huge_pte functions 2013-04-29 15:54:33 -07:00
hw_breakpoint.h UAPI: (Scripted) Disintegrate arch/sh/include/asm 2012-10-09 09:47:37 +01:00
hw_irq.h
i2c-sh7760.h sh: edosk7760 evt2irq migration. 2012-05-18 14:52:31 +09:00
io_generic.h
io_noioport.h SH: Convert out[bwl] macros to inline functions 2012-07-12 13:12:13 +09:00
io_trapped.h
io.h /dev/mem: use phys_addr_t for physical addresses 2012-10-24 15:32:50 -07:00
irq.h sh: Kill off machvec IRQ hinting. 2012-05-21 17:54:01 +09:00
irqflags.h
Kbuild tracing,x86: Add a TSC trace_clock 2012-11-13 15:48:27 -05:00
kdebug.h sh64: Convert to unwinder API. 2012-05-24 15:03:46 +09:00
kexec.h
kgdb.h sh: kgdb: Unset CACHE_FLUSH_IS_SAFE for SMP. 2012-04-11 10:48:24 +09:00
kmap_types.h
kprobes.h
linkage.h
machvec.h sh: Kill off machvec IRQ hinting. 2012-05-21 17:54:01 +09:00
mc146818rtc.h
mmu_context_32.h
mmu_context_64.h
mmu_context.h UAPI: (Scripted) Convert #include "..." to #include <path/...> in kernel system headers 2012-10-02 18:01:25 +01:00
mmu.h
mmzone.h
module.h Make most arch asm/module.h files use asm-generic/module.h 2012-09-28 14:31:03 +09:30
mutex-llsc.h arch: Make __mutex_fastpath_lock_retval return whether fastpath succeeded or not 2013-06-26 12:10:55 +02:00
mutex.h
page.h sh: Fix cached/uncaced address calculation in 29bit mode 2011-11-04 22:13:50 +09:00
pci.h PCI: collapse pcibios_resource_to_bus 2012-02-23 20:19:04 -07:00
perf_event.h
pgalloc.h
pgtable_32.h
pgtable_64.h sh64: Kill off unused fixed I/O mapping window. 2012-05-14 15:16:11 +09:00
pgtable-2level.h
pgtable-3level.h
pgtable.h consolidate io_remap_pfn_range definitions 2013-06-29 12:46:35 +04:00
posix_types.h UAPI: (Scripted) Disintegrate arch/sh/include/asm 2012-10-09 09:47:37 +01:00
processor_32.h sh: define TASK_UNMAPPED_BASE as a page aligned constant 2013-01-11 20:52:35 +09:00
processor_64.h sh: define TASK_UNMAPPED_BASE as a page aligned constant 2013-01-11 20:52:35 +09:00
processor.h UAPI: (Scripted) Convert #include "..." to #include <path/...> in kernel system headers 2012-10-02 18:01:25 +01:00
ptrace_32.h UAPI: (Scripted) Disintegrate arch/sh/include/asm 2012-10-09 09:47:37 +01:00
ptrace_64.h UAPI: (Scripted) Disintegrate arch/sh/include/asm 2012-10-09 09:47:37 +01:00
ptrace.h UAPI: (Scripted) Disintegrate arch/sh/include/asm 2012-10-09 09:47:37 +01:00
push-switch.h
reboot.h
romimage-macros.h
rtc.h
rwsem.h
seccomp.h
sections.h mtd/uclinux: Use generic __bss_stop instead of _ebss 2012-06-27 09:59:43 +02:00
segment.h
setup.h UAPI: (Scripted) Disintegrate arch/sh/include/asm 2012-10-09 09:47:37 +01:00
sfp-machine.h
sh_bios.h
sh7760fb.h
shmparam.h
siu.h sh: remove unused DMA device pointer from SIU platform data 2012-07-20 11:23:44 +05:30
smc37c93x.h
smp-ops.h
smp.h
sparsemem.h
spi.h
spinlock_types.h
spinlock.h
sram.h
stackprotector.h sh: initial stack protector support. 2012-04-19 15:45:57 +09:00
stacktrace.h
string_32.h
string_64.h
string.h UAPI: (Scripted) Convert #include "..." to #include <path/...> in kernel system headers 2012-10-02 18:01:25 +01:00
suspend.h SH: cpuidle: check error code at init 2013-04-22 00:35:53 +02:00
switch_to_32.h Disintegrate asm/system.h for SH 2012-03-28 18:30:03 +01:00
switch_to_64.h Disintegrate asm/system.h for SH 2012-03-28 18:30:03 +01:00
switch_to.h UAPI: (Scripted) Convert #include "..." to #include <path/...> in kernel system headers 2012-10-02 18:01:25 +01:00
syscall_32.h
syscall_64.h
syscall.h UAPI: (Scripted) Convert #include "..." to #include <path/...> in kernel system headers 2012-10-02 18:01:25 +01:00
syscalls_32.h sh: switch to generic old sigaction() 2013-02-03 18:16:16 -05:00
syscalls_64.h sh: switch to generic fork/vfork/clone 2012-11-28 22:36:47 -05:00
syscalls.h UAPI: (Scripted) Convert #include "..." to #include <path/...> in kernel system headers 2012-10-02 18:01:25 +01:00
thread_info.h arch: Consolidate tsk_is_polling() 2013-04-08 17:39:22 +02:00
timex.h
tlb_64.h
tlb.h Fix TLB gather virtual address range invalidation corner cases 2013-08-16 08:52:46 -07:00
tlbflush.h
topology.h sched/numa: Rewrite the CONFIG_NUMA sched domain support 2012-05-09 15:00:55 +02:00
traps_32.h Disintegrate asm/system.h for SH 2012-03-28 18:30:03 +01:00
traps_64.h sh64: Provide EXPEVT helper. 2012-05-14 13:04:08 +09:00
traps.h UAPI: (Scripted) Convert #include "..." to #include <path/...> in kernel system headers 2012-10-02 18:01:25 +01:00
types.h UAPI: (Scripted) Disintegrate arch/sh/include/asm 2012-10-09 09:47:37 +01:00
uaccess_32.h sh: use the new generic strnlen_user() function 2012-06-13 10:28:37 +09:00
uaccess_64.h sh: use the new generic strnlen_user() function 2012-06-13 10:28:37 +09:00
uaccess.h UAPI: (Scripted) Convert #include "..." to #include <path/...> in kernel system headers 2012-10-02 18:01:25 +01:00
unaligned-sh4a.h
unaligned.h
uncached.h
unistd.h consolidate cond_syscall and SYSCALL_ALIAS declarations 2013-03-03 22:55:19 -05:00
unwinder.h
user.h
vga.h
vmlinux.lds.h
watchdog.h
word-at-a-time.h sh: use the new generic strnlen_user() function 2012-06-13 10:28:37 +09:00