Commit dd9c9a6
arm64: avoid memcpy() for syscall_get_arguments()
Do not use memcpy() to extract syscall arguments from struct pt_regs but
rather just perform direct assignments.
Update syscall_set_arguments() too to keep syscall_get_arguments() and
syscall_set_arguments() in sync.
With Generic Entry patch[1] and turn on audit, the performance benchmarks
from perf bench basic syscall on kunpeng920 gives roughly a 1% performance
uplift.
| Metric | W/O this patch | With this patch | Change |
| ---------- | -------------- | --------------- | --------- |
| Total time | 2.241 [sec] | 2.211 [sec] | ↓1.36% |
| usecs/op | 0.224157 | 0.221146 | ↓1.36% |
| ops/sec | 4,461,157 | 4,501,409 | ↑0.9% |
Disassembly shows that using direct assignment causes
syscall_set_arguments() to be inlined and cuts the instruction count by
five or six compared to memcpy(). Because __audit_syscall_entry() only
uses four syscall arguments, the compiler has also elided the copy of
regs->regs[4] and regs->regs[5].
Before:
<syscall_get_arguments.constprop.0>:
aa0103e2 mov x2, x1
91002003 add x3, x0, #0x8
f9408804 ldr x4, [x0, torvalds#272]
f8008444 str x4, [x2], torvalds#8
a9409404 ldp x4, x5, [x0, torvalds#8]
a9009424 stp x4, x5, [x1, torvalds#8]
a9418400 ldp x0, x1, [x0, torvalds#24]
a9010440 stp x0, x1, [x2, torvalds#16]
f9401060 ldr x0, [x3, torvalds#32]
f9001040 str x0, [x2, torvalds#32]
d65f03c0 ret
d503201f nop
After:
a9408e82 ldp x2, x3, [x20, torvalds#8]
2a1603e0 mov w0, w22
f9400e84 ldr x4, [x20, torvalds#24]
f9408a81 ldr x1, [x20, torvalds#272]
9401c4ba bl ffff800080215ca8 <__audit_syscall_entry>
This also aligns the implementation with x86 and RISC-V.
Link: https://lkml.kernel.org/r/[email protected]
Link: https://lore.kernel.org/all/[email protected]/ [1]
Signed-off-by: Jinjie Ruan <[email protected]>
Cc: Catalin Marinas <[email protected]>
Cc: Charlie Jenkins <[email protected]>
Cc: Christian Zankel <[email protected]>
Cc: "Dmitry V. Levin" <[email protected]>
Cc: Helge Deller <[email protected]>
Cc: Maciej W. Rozycki <[email protected]>
Cc: Marc Rutland <[email protected]>
Cc: Max Filippov <[email protected]>
Cc: Russell King (Oracle) <[email protected]>
Cc: Steven Rostedt <[email protected]>
Cc: Thomas Gleinxer <[email protected]>
Cc: Will Deacon <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>1 parent 5297774 commit dd9c9a6
1 file changed
+12
-4
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
82 | 82 | | |
83 | 83 | | |
84 | 84 | | |
85 | | - | |
86 | | - | |
87 | | - | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
88 | 90 | | |
89 | 91 | | |
90 | 92 | | |
91 | 93 | | |
92 | 94 | | |
93 | 95 | | |
94 | | - | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
95 | 103 | | |
96 | 104 | | |
97 | 105 | | |
| |||
0 commit comments