We're updating the issue view to help you get more done. 

Deadlock situation in the optimizing compiler

Description

I experienced and found a deadlock issue in the optimizing compiler when ObjectConstantOperand.similar() is called and a concurrent thread is waiting to compile a method transitively invoked by similar().

Reminder: to compile a method M, the current thread needs to acquire M's lock, and then acquire the compiler's lock.

ObjectConstantOperand.similar(Operand) is a method invoked while the optimizing compiler is active.
o1.similar(o2) will call o1.value.equals(o2.value), where value.equals() is potentially an application method.

Thus when similar() is called by the optimizing compiler, the current thread T1 may reenter the application. In the meantime, a concurrent thread T2 may want to compile a method M, so it acquires the lock on M, however since the compiler's lock is held by T1, it has to block.
Now, let say T1 reaches a call to M which is still not compiled. T1 tries to acquire M's lock, but it is held by T2, so T1 has to block. T1 and T2 are in a deadlock situation.

I frequently observe the deadlock while running Dacapo 9.12 Xalan.

Environment

None

Status

Assignee

Unassigned

Reporter

quentin sabah

Labels

None

External issue ID

None

Components

Fix versions

Priority

Medium