Stack Alignment for ia32 (64 bit)

Description

I believe that stack alignment is broken for ia32 using 64 bit addresses for both the base compiler.

Baseline Compiler:
ia32/BaselineCompilerImpl.java
The offsetToFirstArg, offsetToLastArg and the offsetToJavaArg are not reset to their original values on the second iteration of the loop. This means that it will be pointing to the incorrect stack offset and hence it should be jumping into random code.
I propose a fix by remembering the first and the last arg offsets by:

1 2 3 4 5 6 7 8 9 10 11 12 initialOffsetToFirstArg = offsetToFirstArg; initialOffsetToLastArg = offsetToLastArg; for (int j = VM.BuildFor32Addr ? 1 : 0; j < 2; j++) { if (j == 0) { adjustStack(-WORDSIZE, true); offsetToFirstArg = offsetToFirstArg.plus(WORDSIZE); offsetToLastArg = offsetToLastArg.plus(WORDSIZE); } else { if (dontRealignStack != null) dontRealignStack.resolve(asm); offsetToFirstArg = initialOffsetToFirstArg; offsetToLastArg = initialOffsetToLastArg; }


Opt Compiler:
ia32/CallingConvention.Java
The compiler, when aligning the stack, adds to the stack and hence shrinks the stack. This effectively corrupts the stack after it is done. This can be fixed by subtracting from the stack, effectively popping a placeholder value.

1 MIR_UnaryNoRes.setVal(inst, IC(val - WORDSIZE));


Also, I'm not too sure if this is intended or not, but when testing this code (in a 32bit environment), the subsequent code generated by the require_esp and the test in testBlock emitted:

1 2 test esp, 0x8 lea esp, 0xX

This means that rather than testing the code against the stack pointer with the arguments pushed, you are testing it against the stack pointer without.

Environment

Operating System: Linux
Linux CECS4GJG4Y1 4.4.0-112-generic #135-Ubuntu SMP Fri Jan 19 11:48:36 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

Build: x86_64-linux

Status

Assignee

Erik Brangs

Reporter

Brenda Wang

Labels

None

External issue ID

None

Fix versions

Affects versions

git tip

Priority

High
Configure