We're updating the issue view to help you get more done. 

Deadlock in RVMThread code

Description

In RVMThread.java, there is a static Monitor handshakeLock that prevents multiple handshakes from occurring at the same time. This lock must be acquired by the GC thread to block the mutator threads, and it also must be acquired when a new thread is created in order to allocate and initialize a communication lock for the thread.

In RVMThread.assignThreadSlot(), we have this code:

handshakeLock.lockWithHandshake();
if (communicationLockBySlot[threadSlot] == null) {
communicationLockBySlot[threadSlot] = new Monitor();
}
handshakeLock.unlock();

"new Monitor()" allocates, which can cause a GC. If it does cause a GC, it does not release the handshakeLock, so the GC thread cannot acquire the handshakeLock in hardHandshakeSuspend() and the GC cannot proceed.

---------------------- I believe we can fix this by moving the new Monitor allocation outside the lock:

Monitor m = new Monitor();
handshakeLock.lockWithHandshake();
if (communicationLockBySlot[threadSlot] == null) {
communicationLockBySlot[threadSlot] = m;
}
handshakeLock.unlock();

Environment

Status

Assignee

Unassigned

Reporter

Eddie Aftandilian

Fix versions

Affects versions

3.1.0

Priority

Medium