Stack overflow during signal handling in AArch64

While working on PR #4397, I found that the burst_flush_aarch64 test crashes with a SIGSEGV when signal_stack_size = 32K (which is the value automatically set by DR after adjustment). The crash is due to a stack overflow and is limited to debug builds.

Note that burst_flush_aarch64 intentionally causes a SIGILL too, which is handled as expected by the test and doesn't cause any crash.

To Reproduce

Remove custom -signal_stack_size 64K in clients/drcachesim/tests/burst_flush_aarch64.cpp.
Build with debug on.
Run clients/bin64/tool.drcacheoff.burst_flush_aarch64

Expected behavior The SIGILL thrown by the test is expected and is handled too. But the crashing SIGSEGV is unexpected.

Screenshots or Pasted Text

Details in GDB:

(gdb) r
Starting program: /home/abhinavas/dr/build/i4328-2_debug/clients/bin64/tool.drcacheoff.burst_flush_aarch64 
pre-DR init
pre-DR start

Program received signal SIGILL, Illegal instruction.
0x0000aaaa6aaba448 in ?? ()
(gdb) c
Continuing.

Program received signal SIGSEGV, Segmentation fault.
safe_read_asm_pre () at /home/abhinavas/dr/src/i4328-2/core/arch/aarch64/aarch64.asm:416
416	        strb     w3, [ARG1]
(gdb) c
Continuing.

Program received signal SIGSEGV, Segmentation fault.
0x0000aaaaaab8c9fc in d_r_notify (priority=SYSLOG_NONE, internal=false, synch=false, substitution_num=0, prefix=0x0, fmt=0x0) at /home/abhinavas/dr/src/i4328-2/core/utils.c:1884
1884	{
(gdb) disassemble
Dump of assembler code for function d_r_notify:
   0x0000aaaaaab8c9f8 <+0>:	sub	sp, sp, #0x900
=> 0x0000aaaaaab8c9fc <+4>:	stp	x29, x30, [sp,#-16]!
   0x0000aaaaaab8ca00 <+8>:	mov	x29, sp

The second SIGSEGV is caused by the unexpected stack overflow in d_r_notify while pushing registers onto the stack. This d_r_notify is invoked at https://github.com/DynamoRIO/dynamorio/blob/70be2df2c17c4a55167715e2b3e95fdd3ff9ba19/core/unix/signal.c#L5126.

Versions

What version of DynamoRIO are you using? At commit 70be2df2
System details

abhinavas@tx1:~/dr/build/i4328-2_debug$ uname -a
Linux tx1.fhahn.com 4.9.0-4-arm64 #1 SMP Debian 4.9.51-1 (2017-09-28) aarch64 GNU/Linux

Additional context

Increasing signal_stack_size to 64K for the burst_flush_aarch64 test solved the issue. But it is unclear why the stack overflowed in the first place, as the stack doesn't seem to be too deep.

https://github.com/DynamoRIO/dynamorio/pull/4397#issuecomment-682155961