New place fails to connect to dead places

Description

In the following log, a new place 5 failed to connect dead places 1, 2 and 3.

Test program is HelloWholeWorldLoop3.x10 in x10.dist/samples/resiliency.
Run it as "X10_RESILIENT_MODE=1 X10_NPLACES=4 x10 -x10rt yarn -cp x10yarn.jar:HelloWholeWorldLoop3.jar HelloWholeWorldLoop3 hi 10 5000"

Container: container_1426520310948_0098_01_000002 on triloka10.pok.ibm.com_49294
==================================================================================
LogType: HelloWholeWorldLoop3_Place0.stderr
LogLength: 0
Log Contents:

LogType: HelloWholeWorldLoop3_Place0.stdout
LogLength: 1193
Log Contents:
Place(0) sees 4 places
Place(0) says hello and hi 0
I see that Place(1) was removed
I see that Place(3) was removed
Got DeadPlaceException from Place(1) in round 0
Got DeadPlaceException from Place(3) in round 0
Request to add 2 places.
Place(0) sees 2 places
Place(0) says hello and hi 1
I see that Place(2) was removed
Got DeadPlaceException from Place(2) in round 1
I see that Place(5) was added
I see that Place(4) was added
Request to add 1 places.
Place(0) sees 3 places
Place(0) says hello and hi 2
I see that Place(6) was added
I see that Place(4) was removed
Got DeadPlaceException from Place(4) in round 2
Request to add 1 places.
Place(0) sees 3 places
Place(0) says hello and hi 3
I see that Place(7) was added
I see that Place(6) was removed
Got DeadPlaceException from Place(6) in round 3
Request to add 1 places.
Place(0) sees 3 places
Place(0) says hello and hi 4
I see that Place(8) was added
I see that Place(7) was removed
Got DeadPlaceException from Place(7) in round 4
Request to add 1 places.
Place(0) sees 3 places
Place(0) says hello and hi 5
I see that Place(9) was added
I see that Place(0) was removed
I see that Place(9) was removed
I see that Place(9) was removed

Container: container_1426520310948_0098_01_000004 on triloka2.pok.ibm.com_58642
=================================================================================
LogType: HelloWholeWorldLoop3_Place2.stderr
LogLength: 0
Log Contents:

LogType: HelloWholeWorldLoop3_Place2.stdout
LogLength: 49
Log Contents:
Place(2) says hello and hi 0
Place(2) is dying 1

Container: container_1426520310948_0098_01_000008 on triloka2.pok.ibm.com_58642
=================================================================================
LogType: HelloWholeWorldLoop3_Place5.stderr
LogLength: 2124
Log Contents:
java.io.IOException: Place 5 unable to connect to place 1
at x10.x10rt.SocketTransport.initLink(SocketTransport.java:940)
at x10.x10rt.SocketTransport.establishLinks(SocketTransport.java:208)
at x10.x10rt.X10RT.init(X10RT.java:190)
at x10.runtime.impl.java.Runtime.start(Runtime.java:71)
at HelloWholeWorldLoop3$$Main.main(HelloWholeWorldLoop3.java:46)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:88)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
at java.lang.reflect.Method.invoke(Method.java:618)
at x10.x10rt.yarn.X10MainRunner.main(X10MainRunner.java:30)
java.io.IOException: Place 5 unable to connect to place 2
at x10.x10rt.SocketTransport.initLink(SocketTransport.java:940)
at x10.x10rt.SocketTransport.establishLinks(SocketTransport.java:208)
at x10.x10rt.X10RT.init(X10RT.java:190)
at x10.runtime.impl.java.Runtime.start(Runtime.java:71)
at HelloWholeWorldLoop3$$Main.main(HelloWholeWorldLoop3.java:46)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:88)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
at java.lang.reflect.Method.invoke(Method.java:618)
at x10.x10rt.yarn.X10MainRunner.main(X10MainRunner.java:30)
java.io.IOException: Place 5 unable to connect to place 3
at x10.x10rt.SocketTransport.initLink(SocketTransport.java:940)
at x10.x10rt.SocketTransport.establishLinks(SocketTransport.java:208)
at x10.x10rt.X10RT.init(X10RT.java:190)
at x10.runtime.impl.java.Runtime.start(Runtime.java:71)
at HelloWholeWorldLoop3$$Main.main(HelloWholeWorldLoop3.java:46)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:88)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
at java.lang.reflect.Method.invoke(Method.java:618)
at x10.x10rt.yarn.X10MainRunner.main(X10MainRunner.java:30)

LogType: HelloWholeWorldLoop3_Place5.stdout
LogLength: 116
Log Contents:
Place(5) says hello and hi 2
Place(5) says hello and hi 3
Place(5) says hello and hi 4
Place(5) says hello and hi 5

Environment

Managed X10, Elastic X10, YARN

Status

Assignee

Mikio Takeuchi

Reporter

Mikio Takeuchi

Labels

None

Components

Fix versions

Affects versions

Priority

Medium
Configure