Uploaded image for project: 'JikesRVM'
  1. JikesRVM
  2. RVM-91

Modularize threading system (native thread support)

    Details

      Description

      Currently we only support M-to-N threading, that is mapping M Java threads on N virtual processors executing on N pthreads. This causes a number of issues when interacting with native code and we have a number of complicated solutions to this problem. The thread system should be redesigned to support the mapping of M Java threads to M pthreads.

        Gliffy Diagrams

          Attachments

            Issue Links

            1. Only allow clean/expected transitions in thread execStatus Sub-task Open Unassigned
             
            2.
            Implement Thread.setPriority Sub-task Closed Erik Brangs
             
            3.
            Implement PowerPC syscalls Sub-task Closed Ian Rogers
             
            4.
            Rationalize createVM code Sub-task Closed Filip Pizlo
             
            5.
            Native Threads: exception in bootimagewriter on ppc32-aix and ppc64-linux Sub-task Closed Daniel Frampton
             
            6.
            Native Threads: uninterruptibility violations building prototype-opt image on ia32-linux Sub-task Closed Unassigned
             
            7.
            Stack maps seem to be broken Sub-task Closed Unassigned
             
            8.
            mutator contexts are not being collected after thread death Sub-task Closed Daniel Frampton
             
            9.
            Threads occasionally remain IN_JAVA_TO_BLOCK and never reach a safepoint during a GC stop-the-world request Sub-task Closed Filip Pizlo
             
            10.
            Javadoc for a lot of RVMThread methods is missing, in particular methods for entering/exiting the "blocked"states. Sub-task Closed Filip Pizlo
             
            11.
            Rename the "NATIVE" states to something better - perhaps "IN_GC_SAFE"? Sub-task Closed Filip Pizlo
             
            12.
            Add comments and assertions to the block adapter code. Sub-task Closed Filip Pizlo
             
            13.
            Add comments relating to the use and purpose or, and rationale for acctLock Sub-task Closed Filip Pizlo
             
            14.
            Clean up lock classes: HeavyCondLock, Latch, NoYieldPointsCondLock, and SpinLock Sub-task Closed Filip Pizlo
             
            15.
            TimerThread is broken Sub-task Closed Filip Pizlo
             
            16. Decide what 'available' processors means on native threads Sub-task Open Unassigned
             
            17.
            Update userguide to decribe native threads Sub-task Closed Filip Pizlo
             
            18. Figure out why native threads stability is proportional to the number of available processors Sub-task Open Unassigned
             
            19. Figure out if OSR's use of contextRegisters is correct and/or necessary, given their changed meaning in native threads Sub-task Open Unassigned
             
            20.
            handleHandshakeRequest should be called from enterNativeBlockedImpl Sub-task Closed Unassigned
             
            21.
            Adjust sampling mechanism in AOS to account for native threads Sub-task Closed David Grove
             
            22.
            NPE killing Eclipse performance runs Sub-task Closed Daniel Frampton
             
            23.
            Apache Harmony class library completely broken by native thread code Sub-task Closed Daniel Frampton
             
            24. Improve tracing support in RVMThread Sub-task Open Unassigned
             
            25.
            join() with a timeout doesn't time out Sub-task Closed Filip Pizlo
             

              Activity

              Hide
              ianrogers Ian Rogers added a comment -

              I really still don't get it. My belief is that the prototype system is merely a waste of time. If we want to see native threading implementations there are countless other prototypes we could look at (JamVM, Cacao, my own old Dynamite VM). The particular problem we have with the RVM is that we need to adapt the threading interface to be implementation agnostic. In large part this work has been done by moving all thread queuing code into the green thread package. A lot of things remain, OSR and the memory manager interface, but I fail to see how these things can be accomplished faster by starting a new threading API. I fail to see why this new API would be any more solid than the one that exists. Take our very large success at passing JSR166 TCK tests as proof of this. It seems a simply tortuous thing to do to even contemplate having another threading API.

              What I want is to see the assertions that there will be some inherent advantages to the new API backed up by some real examples. Its easy to criticize what exists - the code is there as well as part of a write-up. Without any specific example of what will be achieved by reinventing the wheel I see little point in supporting a course of action to do it.

              Show
              ianrogers Ian Rogers added a comment - I really still don't get it. My belief is that the prototype system is merely a waste of time. If we want to see native threading implementations there are countless other prototypes we could look at (JamVM, Cacao, my own old Dynamite VM). The particular problem we have with the RVM is that we need to adapt the threading interface to be implementation agnostic. In large part this work has been done by moving all thread queuing code into the green thread package. A lot of things remain, OSR and the memory manager interface, but I fail to see how these things can be accomplished faster by starting a new threading API. I fail to see why this new API would be any more solid than the one that exists. Take our very large success at passing JSR166 TCK tests as proof of this. It seems a simply tortuous thing to do to even contemplate having another threading API. What I want is to see the assertions that there will be some inherent advantages to the new API backed up by some real examples. Its easy to criticize what exists - the code is there as well as part of a write-up. Without any specific example of what will be achieved by reinventing the wheel I see little point in supporting a course of action to do it.
              Hide
              steveblackburn Steve Blackburn added a comment -

              In my experience, prototyping is invaluable. Every collector I have implemented has been done that way. I've done a rough-as-guts proof of concept and then rebuilt from scratch. Some of the collectors in MMTk now have been rebuilt from the ground up multiple times over, each time improving abstractions and engineering choices. The truth is that my intuition is too often a poor guide. Implementation and empirical evaluation is the only way I've found of making well founded engineering decisions. But this is besides the point. Tony et al are offering to create a prototype of 1:1 threading. No one loses. Nothing is destroyed.

              Yes, having an implementation agnostic interface is an excellent objective and no one is saying otherwise. The 1:1 threading work is independent and complementary to this objective. It is not intended to speed up this objective and indeed does not address that objective at all. These are essentially independent goals, which is the reason for suggesting two separate projects.

              The 1:1 project is not an attack or critique of what you've done. It is not addressing the issue of API, it is a proof-of-concept prototype, pure and simple. Whether or not the current API is a good one is something we can all discuss separately, but that is an entirely different matter to the one at hand: identifying SoC projects.

              To re iterate, I see two SoC projects right now, which are well matched to two enthusiastic and capable students:

              1. Modular threading. This project is a design project and a software engineering project and follows on from Rahul's MSc work. Ian is the obvious mentor, and as I understand it there is a good student in the offing.

              2. 1:1 threading. This project is an implementation project and will simply develop a rough-cut implementation of 1:1 threading, identifying the Jikes RVM specific problems that stand in the way, and allow some implementation alternatives to be empirically evaluated.

              I really don't see any reason why you should have concerns, Ian. The 1:1 project is not an attack on your API---the API simply doesn't figure in the project at all. Nonetheless outcomes should help us all develop a great modular implementation (by providing a platform for empirical evaluation of alternatives and tradeoffs eg: "Does having an indirection here slow down green threads or 1:1?", or "What is the actual cost of this data structure, etc etc").

              We have two motivated and experienced students. I hope we all feel happy to let the rip ahead and do some exciting work on these two non-conflicting but complimentary projects.

              Show
              steveblackburn Steve Blackburn added a comment - In my experience, prototyping is invaluable. Every collector I have implemented has been done that way. I've done a rough-as-guts proof of concept and then rebuilt from scratch . Some of the collectors in MMTk now have been rebuilt from the ground up multiple times over, each time improving abstractions and engineering choices. The truth is that my intuition is too often a poor guide. Implementation and empirical evaluation is the only way I've found of making well founded engineering decisions. But this is besides the point. Tony et al are offering to create a prototype of 1:1 threading. No one loses. Nothing is destroyed. Yes, having an implementation agnostic interface is an excellent objective and no one is saying otherwise. The 1:1 threading work is independent and complementary to this objective. It is not intended to speed up this objective and indeed does not address that objective at all. These are essentially independent goals, which is the reason for suggesting two separate projects. The 1:1 project is not an attack or critique of what you've done. It is not addressing the issue of API, it is a proof-of-concept prototype, pure and simple. Whether or not the current API is a good one is something we can all discuss separately, but that is an entirely different matter to the one at hand: identifying SoC projects. To re iterate, I see two SoC projects right now, which are well matched to two enthusiastic and capable students: 1. Modular threading. This project is a design project and a software engineering project and follows on from Rahul's MSc work. Ian is the obvious mentor, and as I understand it there is a good student in the offing. 2. 1:1 threading. This project is an implementation project and will simply develop a rough-cut implementation of 1:1 threading, identifying the Jikes RVM specific problems that stand in the way, and allow some implementation alternatives to be empirically evaluated. I really don't see any reason why you should have concerns, Ian. The 1:1 project is not an attack on your API---the API simply doesn't figure in the project at all. Nonetheless outcomes should help us all develop a great modular implementation (by providing a platform for empirical evaluation of alternatives and tradeoffs eg: "Does having an indirection here slow down green threads or 1:1?", or "What is the actual cost of this data structure, etc etc"). We have two motivated and experienced students. I hope we all feel happy to let the rip ahead and do some exciting work on these two non-conflicting but complimentary projects.
              Hide
              dgrove David Grove added a comment -

              I view the 1-1 prototype project as an important step in understanding exactly where m-n threading assumptions have leaked out into the rest of the JVM. I know there are "intrusions" in the adaptive system, in MMTk, and in the JNI and OSR implementations. There may be others that I don't know about. The whole point of the prototyping effort is to identify exactly where code has to be changed to make it work well with native threads. We have to have a complete solution, not just native threads that work unless you want to have the adaptive system perform well or if you disable OSR.

              I think it's critical for us to understand how to make all of those subsystems work well in a 1-1 threading implementation so we can understand what new abstractions need to be added to the threading API so we can get modular threading that supports native threading and more esoteric models well.

              Show
              dgrove David Grove added a comment - I view the 1-1 prototype project as an important step in understanding exactly where m-n threading assumptions have leaked out into the rest of the JVM. I know there are "intrusions" in the adaptive system, in MMTk, and in the JNI and OSR implementations. There may be others that I don't know about. The whole point of the prototyping effort is to identify exactly where code has to be changed to make it work well with native threads. We have to have a complete solution, not just native threads that work unless you want to have the adaptive system perform well or if you disable OSR. I think it's critical for us to understand how to make all of those subsystems work well in a 1-1 threading implementation so we can understand what new abstractions need to be added to the threading API so we can get modular threading that supports native threading and more esoteric models well.
              Hide
              ianrogers Ian Rogers added a comment -

              This issue is now Filip Pizlo's GSoC project. Filip needs a JIRA account to assign this issue to him.

              Show
              ianrogers Ian Rogers added a comment - This issue is now Filip Pizlo's GSoC project. Filip needs a JIRA account to assign this issue to him.
              Hide
              pizlo Filip Pizlo added a comment -

              Native threading is committed in r15395. I'm keeping this issue open until the sub-tasks are closed.

              Show
              pizlo Filip Pizlo added a comment - Native threading is committed in r15395. I'm keeping this issue open until the sub-tasks are closed.

                People

                • Assignee:
                  pizlo Filip Pizlo
                  Reporter:
                  ianrogers Ian Rogers
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  0 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved: