AcoSail/kernel_optimize_test
8
0
Fork 0
forked from luck/tmp_suning_uos_patched
Code Pull Requests Activity

bpf, arm: Fix register clobbering in div/mod implementation

Browse Source 
[ Upstream commit 79e3445b38e0cab94264a3894c0c3d57c930b97e ]

On ARM CPUs that lack div/mod instructions, ALU32 BPF_DIV and BPF_MOD are
implemented using a call to a helper function. Before, the emitted code
for those function calls failed to preserve caller-saved ARM registers.
Since some of those registers happen to be mapped to BPF registers, it
resulted in eBPF register values being overwritten.

This patch emits code to push and pop the remaining caller-saved ARM
registers r2-r3 into the stack during the div/mod function call. ARM
registers r0-r1 are used as arguments and return value, and those were
already saved and restored correctly.

Fixes: 39c13c204b ("arm: eBPF JIT compiler")
Signed-off-by: Johan Almbladh <johan.almbladh@anyfinetworks.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: Sasha Levin <sashal@kernel.org>
This commit is contained in:
Johan Almbladh 2021-09-28 11:13:10 +02:00 committed by Greg Kroah-Hartman
parent 29a19eaeb2
commit 1d8f4447e8
1 changed files with 19 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

19
arch/arm/net/bpf_jit_32.c
View File

@ -36,6 +36,10 @@
* +-----+
* |RSVD | JIT scratchpad
* current ARM_SP => +-----+ <= (BPF_FP - STACK_SIZE + SCRATCH_SIZE)
* | ... | caller-saved registers
* +-----+
* | ... | arguments passed on stack
* ARM_SP during call => +-----|
* | |
* | ... | Function call stack
* | |
@ -63,6 +67,12 @@
*
* When popping registers off the stack at the end of a BPF function, we
* reference them via the current ARM_FP register.
*
* Some eBPF operations are implemented via a call to a helper function.
* Such calls are "invisible" in the eBPF code, so it is up to the calling
* program to preserve any caller-saved ARM registers during the call. The
* JIT emits code to push and pop those registers onto the stack, immediately
* above the callee stack frame.
*/
#define CALLEE_MASK (1 << ARM_R4 | 1 << ARM_R5 | 1 << ARM_R6 | \
1 << ARM_R7 | 1 << ARM_R8 | 1 << ARM_R9 | \
@ -70,6 +80,8 @@
#define CALLEE_PUSH_MASK (CALLEE_MASK | 1 << ARM_LR)
#define CALLEE_POP_MASK (CALLEE_MASK | 1 << ARM_PC)
#define CALLER_MASK (1 << ARM_R0 | 1 << ARM_R1 | 1 << ARM_R2 | 1 << ARM_R3)
enum {
/* Stack layout - these are offsets from (top of stack - 4) */
BPF_R2_HI,
@ -464,6 +476,7 @@ static inline int epilogue_offset(const struct jit_ctx *ctx)
static inline void emit_udivmod(u8 rd, u8 rm, u8 rn, struct jit_ctx *ctx, u8 op)
{
const int exclude_mask = BIT(ARM_R0) | BIT(ARM_R1);
const s8 *tmp = bpf2a32[TMP_REG_1];
#if __LINUX_ARM_ARCH__ == 7
@ -495,11 +508,17 @@ static inline void emit_udivmod(u8 rd, u8 rm, u8 rn, struct jit_ctx *ctx, u8 op)
emit(ARM_MOV_R(ARM_R0, rm), ctx);
}
/* Push caller-saved registers on stack */
emit(ARM_PUSH(CALLER_MASK & ~exclude_mask), ctx);
/* Call appropriate function */
emit_mov_i(ARM_IP, op == BPF_DIV ?
(u32)jit_udiv32 : (u32)jit_mod32, ctx);
emit_blx_r(ARM_IP, ctx);
/* Restore caller-saved registers from stack */
emit(ARM_POP(CALLER_MASK & ~exclude_mask), ctx);
/* Save return value */
if (rd != ARM_R0)
emit(ARM_MOV_R(rd, ARM_R0), ctx);