Ali Saidi 82fa9ced35 locking/qrwlock: Fix ordering in queued_write_lock_slowpath() ... [ Upstream commit 84a24bf8c52e66b7ac89ada5e3cfbe72d65c1896 ] While this code is executed with the wait_lock held, a reader can acquire the lock without holding wait_lock. The writer side loops checking the value with the atomic_cond_read_acquire(), but only truly acquires the lock when the compare-and-exchange is completed successfully which isn’t ordered. This exposes the window between the acquire and the cmpxchg to an A-B-A problem which allows reads following the lock acquisition to observe values speculatively before the write lock is truly acquired. We've seen a problem in epoll where the reader does a xchg while holding the read lock, but the writer can see a value change out from under it. Writer | Reader -------------------------------------------------------------------------------- ep_scan_ready_list() | |- write_lock_irq() | |- queued_write_lock_slowpath() | |- atomic_cond_read_acquire() | | read_lock_irqsave(&ep->lock, flags); --> (observes value before unlock) | chain_epi_lockless() | | epi->next = xchg(&ep->ovflist, epi); | | read_unlock_irqrestore(&ep->lock, flags); | | | atomic_cmpxchg_relaxed() | |-- READ_ONCE(ep->ovflist); | A core can order the read of the ovflist ahead of the atomic_cmpxchg_relaxed(). Switching the cmpxchg to use acquire semantics addresses this issue at which point the atomic_cond_read can be switched to use relaxed semantics. Fixes: b519b56e37 ("locking/qrwlock: Use atomic_cond_read_acquire() when spinning in qrwlock") Signed-off-by: Ali Saidi <alisaidi@amazon.com> [peterz: use try_cmpxchg()] Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Reviewed-by: Steve Capper <steve.capper@arm.com> Acked-by: Will Deacon <will@kernel.org> Acked-by: Waiman Long <longman@redhat.com> Tested-by: Steve Capper <steve.capper@arm.com> Signed-off-by: Sasha Levin <sashal@kernel.org>		2021-04-28 13:40:00 +02:00
..
lock_events_list.h
lock_events.c
lock_events.h
lockdep_internals.h	lockdep: Fix usage_traceoverflow	2020-10-09 08:53:08 +02:00
lockdep_proc.c
lockdep_states.h
lockdep.c	lockdep: Add a missing initialization hint to the "INFO: Trying to register non-static key" message	2021-04-21 13:00:51 +02:00
locktorture.c	locktorture: Make function torture_percpu_rwsem_init() static	2020-08-24 18:45:32 -07:00
Makefile
mcs_spinlock.h
mutex-debug.c
mutex-debug.h
mutex.c	locking/ww_mutex: Simplify use_ww_ctx & ww_ctx handling	2021-04-07 15:00:06 +02:00
mutex.h
osq_lock.c
percpu-rwsem.c	locking/percpu-rwsem: Use this_cpu_{inc,dec}() for read_count	2020-09-16 16:26:56 +02:00
qrwlock.c	locking/qrwlock: Fix ordering in queued_write_lock_slowpath()	2021-04-28 13:40:00 +02:00
qspinlock_paravirt.h
qspinlock_stat.h
qspinlock.c
rtmutex_common.h	rtmutex: Remove unused argument from rt_mutex_proxy_unlock()	2021-01-30 13:55:17 +01:00
rtmutex-debug.c
rtmutex-debug.h
rtmutex.c	rtmutex: Remove unused argument from rt_mutex_proxy_unlock()	2021-01-30 13:55:17 +01:00
rtmutex.h
rwsem.c	rwsem: Implement down_read_interruptible	2021-01-09 13:46:24 +01:00
rwsem.h
semaphore.c
spinlock_debug.c
spinlock.c
test-ww_mutex.c