1a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson/* 2a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * Written by Doug Lea with assistance from members of JCP JSR-166 3a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * Expert Group and released to the public domain, as explained at 4a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * http://creativecommons.org/publicdomain/zero/1.0/ 5a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 6a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 7a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilsonpackage java.util.concurrent; 8a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 975a06e56a4cc4599946e21422513e4bafa759509Calin Juravleimport java.lang.Thread.UncaughtExceptionHandler; 10a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilsonimport java.util.ArrayList; 11a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilsonimport java.util.Arrays; 12a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilsonimport java.util.Collection; 13a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilsonimport java.util.Collections; 14a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilsonimport java.util.List; 15a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 16a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson/** 17a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * An {@link ExecutorService} for running {@link ForkJoinTask}s. 18a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * A {@code ForkJoinPool} provides the entry point for submissions 19a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * from non-{@code ForkJoinTask} clients, as well as management and 20a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * monitoring operations. 21a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * 22a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * <p>A {@code ForkJoinPool} differs from other kinds of {@link 23a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * ExecutorService} mainly by virtue of employing 24a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * <em>work-stealing</em>: all threads in the pool attempt to find and 2591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * execute tasks submitted to the pool and/or created by other active 2691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * tasks (eventually blocking waiting for work if none exist). This 2791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * enables efficient processing when most tasks spawn other subtasks 2891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * (as do most {@code ForkJoinTask}s), as well as when many small 2991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * tasks are submitted to the pool from external clients. Especially 3091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * when setting <em>asyncMode</em> to true in constructors, {@code 3191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * ForkJoinPool}s may also be appropriate for use with event-style 3291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * tasks that are never joined. 3391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * 34f73b1089c7236a7268ccce10546cc340a36f8c4bEino-Ville Talvala * <p>A static {@code commonPool()} is available and appropriate for 3591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * most applications. The common pool is used by any ForkJoinTask that 3691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * is not explicitly submitted to a specified pool. Using the common 3791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * pool normally reduces resource usage (its threads are slowly 3891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * reclaimed during periods of non-use, and reinstated upon subsequent 3991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * use). 40a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * 4191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * <p>For applications that require separate or custom pools, a {@code 4291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * ForkJoinPool} may be constructed with a given target parallelism 4391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * level; by default, equal to the number of available processors. The 4491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * pool attempts to maintain enough active (or available) threads by 4591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * dynamically adding, suspending, or resuming internal worker 4675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * threads, even if some tasks are stalled waiting to join others. 4775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * However, no such adjustments are guaranteed in the face of blocked 4875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * I/O or other unmanaged synchronization. The nested {@link 4991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * ManagedBlocker} interface enables extension of the kinds of 50a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * synchronization accommodated. 51a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * 52a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * <p>In addition to execution and lifecycle control methods, this 53a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * class provides status check methods (for example 54a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * {@link #getStealCount}) that are intended to aid in developing, 55a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * tuning, and monitoring fork/join applications. Also, method 56a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * {@link #toString} returns indications of pool state in a 57a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * convenient form for informal monitoring. 58a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * 5991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * <p>As is the case with other ExecutorServices, there are three 6091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * main task execution methods summarized in the following table. 6191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * These are designed to be used primarily by clients not already 6291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * engaged in fork/join computations in the current pool. The main 6391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * forms of these methods accept instances of {@code ForkJoinTask}, 6491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * but overloaded forms also allow mixed execution of plain {@code 65a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * Runnable}- or {@code Callable}- based activities as well. However, 6691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * tasks that are already executing in a pool should normally instead 6791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * use the within-computation forms listed in the table unless using 6891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * async event-style tasks that are not usually joined, in which case 6991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * there is little difference among choice of methods. 70a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * 71a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * <table BORDER CELLPADDING=3 CELLSPACING=1> 7275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * <caption>Summary of task execution methods</caption> 73a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * <tr> 74a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * <td></td> 75a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * <td ALIGN=CENTER> <b>Call from non-fork/join clients</b></td> 76a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * <td ALIGN=CENTER> <b>Call from within fork/join computations</b></td> 77a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * </tr> 78a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * <tr> 7975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * <td> <b>Arrange async execution</b></td> 80a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * <td> {@link #execute(ForkJoinTask)}</td> 81a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * <td> {@link ForkJoinTask#fork}</td> 82a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * </tr> 83a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * <tr> 8475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * <td> <b>Await and obtain result</b></td> 85a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * <td> {@link #invoke(ForkJoinTask)}</td> 86a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * <td> {@link ForkJoinTask#invoke}</td> 87a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * </tr> 88a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * <tr> 8975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * <td> <b>Arrange exec and obtain Future</b></td> 90a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * <td> {@link #submit(ForkJoinTask)}</td> 91a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * <td> {@link ForkJoinTask#fork} (ForkJoinTasks <em>are</em> Futures)</td> 92a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * </tr> 93a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * </table> 94a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * 9591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * <p>The common pool is by default constructed with default 9675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * parameters, but these may be controlled by setting three 9775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * {@linkplain System#getProperty system properties}: 9875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * <ul> 9975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * <li>{@code java.util.concurrent.ForkJoinPool.common.parallelism} 10075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * - the parallelism level, a non-negative integer 10175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * <li>{@code java.util.concurrent.ForkJoinPool.common.threadFactory} 10275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * - the class name of a {@link ForkJoinWorkerThreadFactory} 10375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * <li>{@code java.util.concurrent.ForkJoinPool.common.exceptionHandler} 10475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * - the class name of a {@link UncaughtExceptionHandler} 10575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * </ul> 10675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * The system class loader is used to load these classes. 10775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * Upon any error in establishing these settings, default parameters 10875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * are used. It is possible to disable or limit the use of threads in 10975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * the common pool by setting the parallelism property to zero, and/or 11075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * using a factory that may return {@code null}. 111a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * 112a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * <p><b>Implementation notes</b>: This implementation restricts the 113a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * maximum number of running threads to 32767. Attempts to create 114a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * pools with greater than the maximum number result in 115a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * {@code IllegalArgumentException}. 116a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * 117a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * <p>This implementation rejects submitted tasks (that is, by throwing 118a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * {@link RejectedExecutionException}) only when the pool is shut down 119a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * or internal resources have been exhausted. 120a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * 121a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @since 1.7 122a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @author Doug Lea 123a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 124a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilsonpublic class ForkJoinPool extends AbstractExecutorService { 125a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 126a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /* 127a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * Implementation Overview 128a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * 12991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * This class and its nested classes provide the main 13091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * functionality and control for a set of worker threads: 13191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Submissions from non-FJ threads enter into submission queues. 13291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Workers take these tasks and typically split them into subtasks 13391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * that may be stolen by other workers. Preference rules give 13491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * first priority to processing tasks from their own queues (LIFO 13591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * or FIFO, depending on mode), then to randomized FIFO steals of 13691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * tasks in other queues. 13791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * 13891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * WorkQueues 13991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * ========== 14091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * 14191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Most operations occur within work-stealing queues (in nested 14291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * class WorkQueue). These are special forms of Deques that 14391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * support only three of the four possible end-operations -- push, 14491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * pop, and poll (aka steal), under the further constraints that 14591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * push and pop are called only from the owning thread (or, as 14691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * extended here, under a lock), while poll may be called from 14791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * other threads. (If you are unfamiliar with them, you probably 14891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * want to read Herlihy and Shavit's book "The Art of 14991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Multiprocessor programming", chapter 16 describing these in 15091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * more detail before proceeding.) The main work-stealing queue 15191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * design is roughly similar to those in the papers "Dynamic 15291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Circular Work-Stealing Deque" by Chase and Lev, SPAA 2005 15391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * (http://research.sun.com/scalable/pubs/index.html) and 15491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * "Idempotent work stealing" by Michael, Saraswat, and Vechev, 15591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * PPoPP 2009 (http://portal.acm.org/citation.cfm?id=1504186). 15675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * See also "Correct and Efficient Work-Stealing for Weak Memory 15775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * Models" by Le, Pop, Cohen, and Nardelli, PPoPP 2013 15875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * (http://www.di.ens.fr/~zappa/readings/ppopp13.pdf) for an 15975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * analysis of memory ordering (atomic, volatile etc) issues. The 16075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * main differences ultimately stem from GC requirements that we 16175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * null out taken slots as soon as we can, to maintain as small a 16275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * footprint as possible even in programs generating huge numbers 16375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * of tasks. To accomplish this, we shift the CAS arbitrating pop 16475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * vs poll (steal) from being on the indices ("base" and "top") to 16575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * the slots themselves. So, both a successful pop and poll 16675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * mainly entail a CAS of a slot from non-null to null. Because 16775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * we rely on CASes of references, we do not need tag bits on base 16875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * or top. They are simple ints as used in any circular 16975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * array-based queue (see for example ArrayDeque). Updates to the 17075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * indices must still be ordered in a way that guarantees that top 17175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * == base means the queue is empty, but otherwise may err on the 17275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * side of possibly making the queue appear nonempty when a push, 17375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * pop, or poll have not fully committed. Note that this means 17475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * that the poll operation, considered individually, is not 17575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * wait-free. One thief cannot successfully continue until another 17675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * in-progress one (or, if previously empty, a push) completes. 17775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * However, in the aggregate, we ensure at least probabilistic 17875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * non-blockingness. If an attempted steal fails, a thief always 17975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * chooses a different random victim target to try next. So, in 18075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * order for one thief to progress, it suffices for any 18175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * in-progress poll or new push on any empty queue to 18275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * complete. (This is why we normally use method pollAt and its 18375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * variants that try once at the apparent base index, else 18475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * consider alternative actions, rather than method poll.) 18591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * 18691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * This approach also enables support of a user mode in which local 18791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * task processing is in FIFO, not LIFO order, simply by using 18891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * poll rather than pop. This can be useful in message-passing 18991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * frameworks in which tasks are never joined. However neither 19091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * mode considers affinities, loads, cache localities, etc, so 19191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * rarely provide the best possible performance on a given 19291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * machine, but portably provide good throughput by averaging over 19391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * these factors. (Further, even if we did try to use such 19491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * information, we do not usually have a basis for exploiting it. 19591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * For example, some sets of tasks profit from cache affinities, 19691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * but others are harmed by cache pollution effects.) 19791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * 19891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * WorkQueues are also used in a similar way for tasks submitted 19991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * to the pool. We cannot mix these tasks in the same queues used 20091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * for work-stealing (this would contaminate lifo/fifo 20191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * processing). Instead, we randomly associate submission queues 20291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * with submitting threads, using a form of hashing. The 20375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * Submitter probe value serves as a hash code for 20475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * choosing existing queues, and may be randomly repositioned upon 20575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * contention with other submitters. In essence, submitters act 20675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * like workers except that they are restricted to executing local 2076d7851f45fd35ee156adb7c34c03daa7f01ab32fNarayan Kamath * tasks that they submitted. However, because most 20875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * shared/external queue operations are more expensive than 20975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * internal, and because, at steady state, external submitters 21075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * will compete for CPU with workers, ForkJoinTask.join and 21175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * related methods disable them from repeatedly helping to process 21275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * tasks if all workers are active. Insertion of tasks in shared 21375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * mode requires a lock (mainly to protect in the case of 21491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * resizing) but we use only a simple spinlock (using bits in 21591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * field qlock), because submitters encountering a busy queue move 21691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * on to try or create other queues -- they block only when 21791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * creating and registering new queues. 21891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * 21991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Management 22091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * ========== 221a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * 222a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * The main throughput advantages of work-stealing stem from 223a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * decentralized control -- workers mostly take tasks from 224a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * themselves or each other. We cannot negate this in the 225a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * implementation of other management responsibilities. The main 226a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * tactic for avoiding bottlenecks is packing nearly all 22791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * essentially atomic control state into two volatile variables 22891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * that are by far most often read (not written) as status and 22991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * consistency checks. 23091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * 23191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Field "ctl" contains 64 bits holding all the information needed 23291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * to atomically decide to add, inactivate, enqueue (on an event 23391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * queue), dequeue, and/or re-activate workers. To enable this 23491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * packing, we restrict maximum parallelism to (1<<15)-1 (which is 23591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * far in excess of normal operating range) to allow ids, counts, 23691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * and their negations (used for thresholding) to fit into 16bit 23791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * fields. 23891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * 23991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Field "plock" is a form of sequence lock with a saturating 24091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * shutdown bit (similarly for per-queue "qlocks"), mainly 24191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * protecting updates to the workQueues array, as well as to 24291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * enable shutdown. When used as a lock, it is normally only very 24391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * briefly held, so is nearly always available after at most a 24491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * brief spin, but we use a monitor-based backup strategy to 24591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * block when needed. 24691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * 24791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Recording WorkQueues. WorkQueues are recorded in the 24891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * "workQueues" array that is created upon first use and expanded 24991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * if necessary. Updates to the array while recording new workers 25091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * and unrecording terminated ones are protected from each other 25191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * by a lock but the array is otherwise concurrently readable, and 25291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * accessed directly. To simplify index-based operations, the 25391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * array size is always a power of two, and all readers must 25491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * tolerate null slots. Worker queues are at odd indices. Shared 25591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * (submission) queues are at even indices, up to a maximum of 64 25691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * slots, to limit growth even if array needs to expand to add 25791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * more workers. Grouping them together in this way simplifies and 25891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * speeds up task scanning. 25991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * 26091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * All worker thread creation is on-demand, triggered by task 26191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * submissions, replacement of terminated workers, and/or 26291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * compensation for blocked workers. However, all other support 26391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * code is set up to work with other policies. To ensure that we 26491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * do not hold on to worker references that would prevent GC, ALL 26591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * accesses to workQueues are via indices into the workQueues 26691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * array (which is one source of some of the messy code 26791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * constructions here). In essence, the workQueues array serves as 26891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * a weak reference mechanism. Thus for example the wait queue 26991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * field of ctl stores indices, not references. Access to the 27091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * workQueues in associated methods (for example signalWork) must 27191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * both index-check and null-check the IDs. All such accesses 27291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * ignore bad IDs by returning out early from what they are doing, 27391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * since this can only be associated with termination, in which 27491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * case it is OK to give up. All uses of the workQueues array 27591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * also check that it is non-null (even if previously 27691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * non-null). This allows nulling during termination, which is 27791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * currently not necessary, but remains an option for 27891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * resource-revocation-based shutdown schemes. It also helps 27991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * reduce JIT issuance of uncommon-trap code, which tends to 28091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * unnecessarily complicate control flow in some methods. 28191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * 28291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Event Queuing. Unlike HPC work-stealing frameworks, we cannot 283a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * let workers spin indefinitely scanning for tasks when none can 284a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * be found immediately, and we cannot start/resume workers unless 285a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * there appear to be tasks available. On the other hand, we must 286a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * quickly prod them into action when new tasks are submitted or 28791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * generated. In many usages, ramp-up time to activate workers is 28891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * the main limiting factor in overall performance (this is 28991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * compounded at program start-up by JIT compilation and 29091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * allocation). So we try to streamline this as much as possible. 29191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * We park/unpark workers after placing in an event wait queue 29291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * when they cannot find work. This "queue" is actually a simple 29391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Treiber stack, headed by the "id" field of ctl, plus a 15bit 29491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * counter value (that reflects the number of times a worker has 29591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * been inactivated) to avoid ABA effects (we need only as many 29691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * version numbers as worker threads). Successors are held in 29791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * field WorkQueue.nextWait. Queuing deals with several intrinsic 29891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * races, mainly that a task-producing thread can miss seeing (and 299a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * signalling) another thread that gave up looking for work but 300a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * has not yet entered the wait queue. We solve this by requiring 30191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * a full sweep of all workers (via repeated calls to method 30291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * scan()) both before and after a newly waiting worker is added 30375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * to the wait queue. Because enqueued workers may actually be 30475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * rescanning rather than waiting, we set and clear the "parker" 30575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * field of WorkQueues to reduce unnecessary calls to unpark. 30675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * (This requires a secondary recheck to avoid missed signals.) 30775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * Note the unusual conventions about Thread.interrupts 30875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * surrounding parking and other blocking: Because interrupts are 30975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * used solely to alert threads to check termination, which is 31075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * checked anyway upon blocking, we clear status (using 31175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * Thread.interrupted) before any call to park, so that park does 31275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * not immediately return due to status being set via some other 31375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * unrelated call to interrupt in user code. 314a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * 315a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * Signalling. We create or wake up workers only when there 316a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * appears to be at least one task they might be able to find and 31775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * execute. When a submission is added or another worker adds a 31875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * task to a queue that has fewer than two tasks, they signal 31975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * waiting workers (or trigger creation of new ones if fewer than 32075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * the given parallelism level -- signalWork). These primary 32175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * signals are buttressed by others whenever other threads remove 32275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * a task from a queue and notice that there are other tasks there 32375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * as well. So in general, pools will be over-signalled. On most 32475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * platforms, signalling (unpark) overhead time is noticeably 32591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * long, and the time between signalling a thread and it actually 32691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * making progress can be very noticeably long, so it is worth 32791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * offloading these delays from critical paths as much as 32875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * possible. Additionally, workers spin-down gradually, by staying 32975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * alive so long as they see the ctl state changing. Similar 33075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * stability-sensing techniques are also used before blocking in 33175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * awaitJoin and helpComplete. 332a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * 333a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * Trimming workers. To release resources after periods of lack of 334a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * use, a worker starting to wait when the pool is quiescent will 33591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * time out and terminate if the pool has remained quiescent for a 33691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * given period -- a short period if there are more threads than 33791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * parallelism, longer as the number of threads decreases. This 33891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * will slowly propagate, eventually terminating all workers after 33991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * periods of non-use. 34091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * 34191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Shutdown and Termination. A call to shutdownNow atomically sets 34291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * a plock bit and then (non-atomically) sets each worker's 34391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * qlock status, cancels all unprocessed tasks, and wakes up 34491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * all waiting workers. Detecting whether termination should 34591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * commence after a non-abrupt shutdown() call requires more work 34691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * and bookkeeping. We need consensus about quiescence (i.e., that 34791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * there is no more work). The active count provides a primary 34891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * indication but non-abrupt shutdown still requires a rechecking 34991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * scan for any workers that are inactive but not queued. 35091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * 35191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Joining Tasks 35291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * ============= 35391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * 35491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Any of several actions may be taken when one worker is waiting 35591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * to join a task stolen (or always held) by another. Because we 35691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * are multiplexing many tasks on to a pool of workers, we can't 35791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * just let them block (as in Thread.join). We also cannot just 35891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * reassign the joiner's run-time stack with another and replace 35991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * it later, which would be a form of "continuation", that even if 36091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * possible is not necessarily a good idea since we sometimes need 36191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * both an unblocked task and its continuation to progress. 36291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Instead we combine two tactics: 363a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * 364a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * Helping: Arranging for the joiner to execute some task that it 36591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * would be running if the steal had not occurred. 366a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * 367a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * Compensating: Unless there are already enough live threads, 36891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * method tryCompensate() may create or re-activate a spare 36991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * thread to compensate for blocked joiners until they unblock. 37091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * 37191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * A third form (implemented in tryRemoveAndExec) amounts to 37291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * helping a hypothetical compensator: If we can readily tell that 37391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * a possible action of a compensator is to steal and execute the 37491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * task being joined, the joining thread can do so directly, 37591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * without the need for a compensation thread (although at the 37691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * expense of larger run-time stacks, but the tradeoff is 37791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * typically worthwhile). 378a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * 379a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * The ManagedBlocker extension API can't use helping so relies 380a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * only on compensation in method awaitBlocker. 381a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * 38291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * The algorithm in tryHelpStealer entails a form of "linear" 38391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * helping: Each worker records (in field currentSteal) the most 38491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * recent task it stole from some other worker. Plus, it records 38591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * (in field currentJoin) the task it is currently actively 38691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * joining. Method tryHelpStealer uses these markers to try to 38791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * find a worker to help (i.e., steal back a task from and execute 38891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * it) that could hasten completion of the actively joined task. 38991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * In essence, the joiner executes a task that would be on its own 39091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * local deque had the to-be-joined task not been stolen. This may 39191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * be seen as a conservative variant of the approach in Wagner & 39291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Calder "Leapfrogging: a portable technique for implementing 39391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * efficient futures" SIGPLAN Notices, 1993 39491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * (http://portal.acm.org/citation.cfm?id=155354). It differs in 39591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * that: (1) We only maintain dependency links across workers upon 39691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * steals, rather than use per-task bookkeeping. This sometimes 39791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * requires a linear scan of workQueues array to locate stealers, 39891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * but often doesn't because stealers leave hints (that may become 39991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * stale/wrong) of where to locate them. It is only a hint 40091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * because a worker might have had multiple steals and the hint 40191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * records only one of them (usually the most current). Hinting 40291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * isolates cost to when it is needed, rather than adding to 40391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * per-task overhead. (2) It is "shallow", ignoring nesting and 40491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * potentially cyclic mutual steals. (3) It is intentionally 40591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * racy: field currentJoin is updated only while actively joining, 40691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * which means that we miss links in the chain during long-lived 40791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * tasks, GC stalls etc (which is OK since blocking in such cases 40891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * is usually a good idea). (4) We bound the number of attempts 40991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * to find work (see MAX_HELP) and fall back to suspending the 41091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * worker and if necessary replacing it with another. 41191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * 412a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * It is impossible to keep exactly the target parallelism number 413a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * of threads running at any given time. Determining the 414a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * existence of conservatively safe helping targets, the 415a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * availability of already-created spares, and the apparent need 41691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * to create new spares are all racy, so we rely on multiple 41791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * retries of each. Compensation in the apparent absence of 41891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * helping opportunities is challenging to control on JVMs, where 41991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * GC and other activities can stall progress of tasks that in 42091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * turn stall out many other dependent tasks, without us being 42191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * able to determine whether they will ever require compensation. 42291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Even though work-stealing otherwise encounters little 42391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * degradation in the presence of more threads than cores, 42491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * aggressively adding new threads in such cases entails risk of 42591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * unwanted positive feedback control loops in which more threads 42691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * cause more dependent stalls (as well as delayed progress of 42791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * unblocked threads to the point that we know they are available) 42891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * leading to more situations requiring more threads, and so 42991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * on. This aspect of control can be seen as an (analytically 43091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * intractable) game with an opponent that may choose the worst 43191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * (for us) active thread to stall at any time. We take several 43291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * precautions to bound losses (and thus bound gains), mainly in 43391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * methods tryCompensate and awaitJoin. 43491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * 43591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Common Pool 43691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * =========== 43791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * 43875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * The static common pool always exists after static 43991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * initialization. Since it (or any other created pool) need 44091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * never be used, we minimize initial construction overhead and 44191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * footprint to the setup of about a dozen fields, with no nested 44291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * allocation. Most bootstrapping occurs within method 44391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * fullExternalPush during the first submission to the pool. 44491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * 44591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * When external threads submit to the common pool, they can 44675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * perform subtask processing (see externalHelpJoin and related 44775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * methods). This caller-helps policy makes it sensible to set 44875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * common pool parallelism level to one (or more) less than the 44975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * total number of available cores, or even zero for pure 45075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * caller-runs. We do not need to record whether external 45191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * submissions are to the common pool -- if not, externalHelpJoin 45291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * returns quickly (at the most helping to signal some common pool 45391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * workers). These submitters would otherwise be blocked waiting 45491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * for completion, so the extra effort (with liberally sprinkled 45591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * task status checks) in inapplicable cases amounts to an odd 45691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * form of limited spin-wait before blocking in ForkJoinTask.join. 45791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * 45891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Style notes 45991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * =========== 46091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * 46191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * There is a lot of representation-level coupling among classes 46291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * ForkJoinPool, ForkJoinWorkerThread, and ForkJoinTask. The 46391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * fields of WorkQueue maintain data structures managed by 46491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * ForkJoinPool, so are directly accessed. There is little point 465a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * trying to reduce this, since any associated future changes in 466a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * representations will need to be accompanied by algorithmic 46791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * changes anyway. Several methods intrinsically sprawl because 46891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * they must accumulate sets of consistent reads of volatiles held 46991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * in local variables. Methods signalWork() and scan() are the 47091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * main bottlenecks, so are especially heavily 47191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * micro-optimized/mangled. There are lots of inline assignments 47291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * (of form "while ((local = field) != 0)") which are usually the 47391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * simplest way to ensure the required read orderings (which are 47491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * sometimes critical). This leads to a "C"-like style of listing 47591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * declarations of these locals at the heads of methods or blocks. 47691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * There are several occurrences of the unusual "do {} while 47791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * (!cas...)" which is the simplest way to force an update of a 47891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * CAS'ed variable. There are also other coding oddities (including 47991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * several unnecessary-looking hoisted null checks) that help 48091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * some methods perform reasonably even when interpreted (not 48191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * compiled). 48291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * 48391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * The order of declarations in this file is: 48491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * (1) Static utility functions 48591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * (2) Nested (static) classes 48691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * (3) Static fields 48791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * (4) Fields, along with constants used when unpacking some of them 48891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * (5) Internal control methods 48991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * (6) Callbacks and other support for ForkJoinTask methods 49091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * (7) Exported methods 49191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * (8) Static block initializing statics in minimally dependent order 49291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle */ 493edf43d27e240d82106f39ae91404963c23987234Narayan Kamath // android-note: Removed references to CountedCompleters. 49491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle 49591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle // Static utilities 49691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle 49791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle /** 49891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * If there is a security manager, makes sure caller has 49991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * permission to modify threads. 500a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 50191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle private static void checkPermission() { 50291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle SecurityManager security = System.getSecurityManager(); 50391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (security != null) 50491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle security.checkPermission(modifyThreadPermission); 50591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 50691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle 50791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle // Nested classes 508a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 509a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 510a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * Factory for creating new {@link ForkJoinWorkerThread}s. 511a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * A {@code ForkJoinWorkerThreadFactory} must be defined and used 512a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * for {@code ForkJoinWorkerThread} subclasses that extend base 513a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * functionality or initialize threads with different contexts. 514a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 515a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson public static interface ForkJoinWorkerThreadFactory { 516a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 517a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * Returns a new worker thread operating in the given pool. 518a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * 519a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @param pool the pool this thread works in 52075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * @return the new worker thread 521edf43d27e240d82106f39ae91404963c23987234Narayan Kamath * @throws NullPointerException if the pool is null 522a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 523a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson public ForkJoinWorkerThread newThread(ForkJoinPool pool); 524a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 525a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 526a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 527a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * Default ForkJoinWorkerThreadFactory implementation; creates a 528a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * new ForkJoinWorkerThread. 529a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 53091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle static final class DefaultForkJoinWorkerThreadFactory 531a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson implements ForkJoinWorkerThreadFactory { 53291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle public final ForkJoinWorkerThread newThread(ForkJoinPool pool) { 533a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson return new ForkJoinWorkerThread(pool); 534a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 535a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 536a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 537a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 53891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Class for artificial tasks that are used to replace the target 53991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * of local joins if they are removed from an interior queue slot 54091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * in WorkQueue.tryRemoveAndExec. We don't need the proxy to 54191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * actually do anything beyond having a unique identity. 54291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle */ 54391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle static final class EmptyTask extends ForkJoinTask<Void> { 54491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle private static final long serialVersionUID = -7721805057305804111L; 54591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle EmptyTask() { status = ForkJoinTask.NORMAL; } // force done 54691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle public final Void getRawResult() { return null; } 54791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle public final void setRawResult(Void x) {} 54891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle public final boolean exec() { return true; } 54991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 55091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle 55191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle /** 55291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Queues supporting work-stealing as well as external task 55391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * submission. See above for main rationale and algorithms. 55491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Implementation relies heavily on "Unsafe" intrinsics 55591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * and selective use of "volatile": 55691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * 55791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Field "base" is the index (mod array.length) of the least valid 55891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * queue slot, which is always the next position to steal (poll) 55991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * from if nonempty. Reads and writes require volatile orderings 56091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * but not CAS, because updates are only performed after slot 56191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * CASes. 56291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * 56391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Field "top" is the index (mod array.length) of the next queue 56491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * slot to push to or pop from. It is written only by owner thread 56591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * for push, or under lock for external/shared push, and accessed 56691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * by other threads only after reading (volatile) base. Both top 56791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * and base are allowed to wrap around on overflow, but (top - 56891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * base) (or more commonly -(base - top) to force volatile read of 56991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * base before top) still estimates size. The lock ("qlock") is 57091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * forced to -1 on termination, causing all further lock attempts 57191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * to fail. (Note: we don't need CAS for termination state because 57291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * upon pool shutdown, all shared-queues will stop being used 57391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * anyway.) Nearly all lock bodies are set up so that exceptions 57491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * within lock bodies are "impossible" (modulo JVM errors that 57591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * would cause failure anyway.) 57691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * 57791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * The array slots are read and written using the emulation of 57891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * volatiles/atomics provided by Unsafe. Insertions must in 57991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * general use putOrderedObject as a form of releasing store to 58091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * ensure that all writes to the task object are ordered before 58191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * its publication in the queue. All removals entail a CAS to 58291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * null. The array is always a power of two. To ensure safety of 58391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Unsafe array operations, all accesses perform explicit null 58491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * checks and implicit bounds checks via power-of-two masking. 58591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * 58691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * In addition to basic queuing support, this class contains 58791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * fields described elsewhere to control execution. It turns out 58891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * to work better memory-layout-wise to include them in this class 58991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * rather than a separate class. 59091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * 59191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Performance on most platforms is very sensitive to placement of 59291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * instances of both WorkQueues and their arrays -- we absolutely 59391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * do not want multiple WorkQueue instances or multiple queue 59491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * arrays sharing cache lines. (It would be best for queue objects 59591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * and their arrays to share, but there is nothing available to 59675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * help arrange that). The @Contended annotation alerts JVMs to 59775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * try to keep instances apart. 59891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle */ 59991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle static final class WorkQueue { 60091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle /** 60191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Capacity of work-stealing queue array upon initialization. 60291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Must be a power of two; at least 4, but should be larger to 60391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * reduce or eliminate cacheline sharing among queues. 60491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Currently, it is much larger, as a partial workaround for 60591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * the fact that JVMs often place arrays in locations that 60691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * share GC bookkeeping (especially cardmarks) such that 60791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * per-write accesses encounter serious memory contention. 60891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle */ 60991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle static final int INITIAL_QUEUE_CAPACITY = 1 << 13; 610a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 61191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle /** 61291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Maximum size for queue arrays. Must be a power of two less 61391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * than or equal to 1 << (31 - width of array entry) to ensure 61491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * lack of wraparound of index calculations, but defined to a 61591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * value a bit less than this to help users trap runaway 61691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * programs before saturating systems. 61791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle */ 61891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle static final int MAXIMUM_QUEUE_CAPACITY = 1 << 26; // 64M 61991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle 62091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle // Heuristic padding to ameliorate unfortunate memory placements 62191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle volatile long pad00, pad01, pad02, pad03, pad04, pad05, pad06; 62291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle 62391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle volatile int eventCount; // encoded inactivation count; < 0 if inactive 62491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle int nextWait; // encoded record of next event waiter 62591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle int nsteals; // number of steals 62675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle int hint; // steal index hint 62775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle short poolIndex; // index of this queue in pool 62875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle final short mode; // 0: lifo, > 0: fifo, < 0: shared 62991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle volatile int qlock; // 1: locked, -1: terminate; else 0 63091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle volatile int base; // index of next slot for poll 63191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle int top; // index of next slot for push 63291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle ForkJoinTask<?>[] array; // the elements (initially unallocated) 63391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle final ForkJoinPool pool; // the containing pool (may be null) 63491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle final ForkJoinWorkerThread owner; // owning thread or null if shared 63591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle volatile Thread parker; // == owner during call to park; else null 63691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle volatile ForkJoinTask<?> currentJoin; // task being joined in awaitJoin 63791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle ForkJoinTask<?> currentSteal; // current non-local task being executed 63891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle 63991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle volatile Object pad10, pad11, pad12, pad13, pad14, pad15, pad16, pad17; 64091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle volatile Object pad18, pad19, pad1a, pad1b, pad1c, pad1d; 64191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle 64291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle WorkQueue(ForkJoinPool pool, ForkJoinWorkerThread owner, int mode, 64391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle int seed) { 64491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle this.pool = pool; 64591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle this.owner = owner; 64675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle this.mode = (short)mode; 64775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle this.hint = seed; // store initial seed for runWorker 64891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle // Place indices in the center of array (that is not yet allocated) 64991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle base = top = INITIAL_QUEUE_CAPACITY >>> 1; 65091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 651a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 65291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle /** 65391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Returns the approximate number of tasks in the queue. 65491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle */ 65591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle final int queueSize() { 65691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle int n = base - top; // non-owner callers must read base first 65791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle return (n >= 0) ? 0 : -n; // ignore transient negative 65891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 65991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle 66075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle /** 66191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Provides a more accurate estimate of whether this queue has 66291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * any tasks than does queueSize, by checking whether a 66391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * near-empty queue has at least one unclaimed task. 66491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle */ 66591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle final boolean isEmpty() { 66691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle ForkJoinTask<?>[] a; int m, s; 66791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle int n = base - (s = top); 66891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle return (n >= 0 || 66991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle (n == -1 && 67091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle ((a = array) == null || 67191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle (m = a.length - 1) < 0 || 67291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle U.getObject 67391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle (a, (long)((m & (s - 1)) << ASHIFT) + ABASE) == null))); 67491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 67591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle 67691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle /** 67791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Pushes a task. Call only by owner in unshared queues. (The 67891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * shared-queue version is embedded in method externalPush.) 67991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * 68091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * @param task the task. Caller must ensure non-null. 68191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * @throws RejectedExecutionException if array cannot be resized 68291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle */ 68391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle final void push(ForkJoinTask<?> task) { 68491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle ForkJoinTask<?>[] a; ForkJoinPool p; 68575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle int s = top, n; 68691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if ((a = array) != null) { // ignore if queue removed 68775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle int m = a.length - 1; 68875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle U.putOrderedObject(a, ((m & s) << ASHIFT) + ABASE, task); 68975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if ((n = (top = s + 1) - base) <= 2) 69075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle (p = pool).signalWork(p.workQueues, this); 69191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle else if (n >= m) 69291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle growArray(); 69391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 69491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 69591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle 69675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle /** 69791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Initializes or doubles the capacity of array. Call either 69891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * by owner or with lock held -- it is OK for base, but not 69991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * top, to move while resizings are in progress. 70091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle */ 70191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle final ForkJoinTask<?>[] growArray() { 70291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle ForkJoinTask<?>[] oldA = array; 70391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle int size = oldA != null ? oldA.length << 1 : INITIAL_QUEUE_CAPACITY; 70491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (size > MAXIMUM_QUEUE_CAPACITY) 70591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle throw new RejectedExecutionException("Queue capacity exceeded"); 70691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle int oldMask, t, b; 70791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle ForkJoinTask<?>[] a = array = new ForkJoinTask<?>[size]; 70891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (oldA != null && (oldMask = oldA.length - 1) >= 0 && 70991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle (t = top) - (b = base) > 0) { 71091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle int mask = size - 1; 71191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle do { 71291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle ForkJoinTask<?> x; 71391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle int oldj = ((b & oldMask) << ASHIFT) + ABASE; 71491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle int j = ((b & mask) << ASHIFT) + ABASE; 71591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle x = (ForkJoinTask<?>)U.getObjectVolatile(oldA, oldj); 71691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (x != null && 71791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle U.compareAndSwapObject(oldA, oldj, x, null)) 71891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle U.putObjectVolatile(a, j, x); 71991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } while (++b != t); 72091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 72191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle return a; 72291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 72391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle 72491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle /** 72591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Takes next task, if one exists, in LIFO order. Call only 72691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * by owner in unshared queues. 72791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle */ 72891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle final ForkJoinTask<?> pop() { 72991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle ForkJoinTask<?>[] a; ForkJoinTask<?> t; int m; 73091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if ((a = array) != null && (m = a.length - 1) >= 0) { 73191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle for (int s; (s = top - 1) - base >= 0;) { 73291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle long j = ((m & s) << ASHIFT) + ABASE; 73391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if ((t = (ForkJoinTask<?>)U.getObject(a, j)) == null) 73491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle break; 73591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (U.compareAndSwapObject(a, j, t, null)) { 73691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle top = s; 73791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle return t; 73891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 73991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 74091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 74191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle return null; 74291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 74391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle 74491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle /** 74591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Takes a task in FIFO order if b is base of queue and a task 74691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * can be claimed without contention. Specialized versions 74791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * appear in ForkJoinPool methods scan and tryHelpStealer. 74891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle */ 74991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle final ForkJoinTask<?> pollAt(int b) { 75091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle ForkJoinTask<?> t; ForkJoinTask<?>[] a; 75191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if ((a = array) != null) { 75291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle int j = (((a.length - 1) & b) << ASHIFT) + ABASE; 75391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if ((t = (ForkJoinTask<?>)U.getObjectVolatile(a, j)) != null && 75475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle base == b && U.compareAndSwapObject(a, j, t, null)) { 75575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle U.putOrderedInt(this, QBASE, b + 1); 75691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle return t; 75791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 75891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 75991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle return null; 76091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 76191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle 76291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle /** 76391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Takes next task, if one exists, in FIFO order. 76491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle */ 76591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle final ForkJoinTask<?> poll() { 76691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle ForkJoinTask<?>[] a; int b; ForkJoinTask<?> t; 76791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle while ((b = base) - top < 0 && (a = array) != null) { 76891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle int j = (((a.length - 1) & b) << ASHIFT) + ABASE; 76991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle t = (ForkJoinTask<?>)U.getObjectVolatile(a, j); 77091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (t != null) { 77175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (U.compareAndSwapObject(a, j, t, null)) { 77275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle U.putOrderedInt(this, QBASE, b + 1); 77391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle return t; 77491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 77591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 77691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle else if (base == b) { 77791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (b + 1 == top) 77891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle break; 77991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle Thread.yield(); // wait for lagging update (very rare) 78091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 78191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 78291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle return null; 78391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 78491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle 78591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle /** 78691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Takes next task, if one exists, in order specified by mode. 78791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle */ 78891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle final ForkJoinTask<?> nextLocalTask() { 78991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle return mode == 0 ? pop() : poll(); 79091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 79191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle 79291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle /** 79391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Returns next task, if one exists, in order specified by mode. 79491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle */ 79591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle final ForkJoinTask<?> peek() { 79691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle ForkJoinTask<?>[] a = array; int m; 79791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (a == null || (m = a.length - 1) < 0) 79891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle return null; 79991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle int i = mode == 0 ? top - 1 : base; 80091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle int j = ((i & m) << ASHIFT) + ABASE; 80191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle return (ForkJoinTask<?>)U.getObjectVolatile(a, j); 80291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 80391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle 80491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle /** 80591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Pops the given task only if it is at the current top. 80691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * (A shared version is available only via FJP.tryExternalUnpush) 80791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle */ 80891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle final boolean tryUnpush(ForkJoinTask<?> t) { 80991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle ForkJoinTask<?>[] a; int s; 81091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if ((a = array) != null && (s = top) != base && 81191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle U.compareAndSwapObject 81291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle (a, (((a.length - 1) & --s) << ASHIFT) + ABASE, t, null)) { 81391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle top = s; 81491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle return true; 81591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 81691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle return false; 81791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 81891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle 81991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle /** 82091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Removes and cancels all known tasks, ignoring any exceptions. 82191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle */ 82291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle final void cancelAll() { 82391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle ForkJoinTask.cancelIgnoringExceptions(currentJoin); 82491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle ForkJoinTask.cancelIgnoringExceptions(currentSteal); 82591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle for (ForkJoinTask<?> t; (t = poll()) != null; ) 82691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle ForkJoinTask.cancelIgnoringExceptions(t); 82791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 82891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle 82991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle // Specialized execution methods 83091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle 83191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle /** 83275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * Polls and runs tasks until empty. 83391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle */ 83475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle final void pollAndExecAll() { 83575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle for (ForkJoinTask<?> t; (t = poll()) != null;) 83675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle t.doExec(); 83791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 83891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle 83991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle /** 84075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * Executes a top-level task and any local tasks remaining 84175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * after execution. 84291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle */ 84375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle final void runTask(ForkJoinTask<?> task) { 84475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if ((currentSteal = task) != null) { 84575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle task.doExec(); 84675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle ForkJoinTask<?>[] a = array; 84775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle int md = mode; 84875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle ++nsteals; 84975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle currentSteal = null; 85075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (md != 0) 85175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle pollAndExecAll(); 85275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle else if (a != null) { 85375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle int s, m = a.length - 1; 85475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle while ((s = top - 1) - base >= 0) { 85575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle long i = ((m & s) << ASHIFT) + ABASE; 85675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle ForkJoinTask<?> t = (ForkJoinTask<?>)U.getObject(a, i); 85775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (t == null) 85875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle break; 85975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (U.compareAndSwapObject(a, i, t, null)) { 86075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle top = s; 86175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle t.doExec(); 86275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle } 86375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle } 86475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle } 86575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle } 86691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 86791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle 86891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle /** 86991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * If present, removes from queue and executes the given task, 87091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * or any other cancelled task. Returns (true) on any CAS 87191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * or consistency check failure so caller can retry. 87291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * 87391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * @return false if no progress can be made, else true 87491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle */ 87591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle final boolean tryRemoveAndExec(ForkJoinTask<?> task) { 87675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle boolean stat; 87791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle ForkJoinTask<?>[] a; int m, s, b, n; 87875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (task != null && (a = array) != null && (m = a.length - 1) >= 0 && 87991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle (n = (s = top) - (b = base)) > 0) { 88075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle boolean removed = false, empty = true; 88175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle stat = true; 88291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle for (ForkJoinTask<?> t;;) { // traverse from s to b 88375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle long j = ((--s & m) << ASHIFT) + ABASE; 88475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle t = (ForkJoinTask<?>)U.getObject(a, j); 88591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (t == null) // inconsistent length 88691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle break; 88791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle else if (t == task) { 88891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (s + 1 == top) { // pop 88991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (!U.compareAndSwapObject(a, j, task, null)) 89091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle break; 89191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle top = s; 89291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle removed = true; 89391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 89491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle else if (base == b) // replace with proxy 89591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle removed = U.compareAndSwapObject(a, j, task, 89691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle new EmptyTask()); 89791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle break; 89891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 89991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle else if (t.status >= 0) 90091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle empty = false; 90191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle else if (s + 1 == top) { // pop and throw away 90291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (U.compareAndSwapObject(a, j, t, null)) 90391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle top = s; 90491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle break; 90591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 90691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (--n == 0) { 90791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (!empty && base == b) 90891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle stat = false; 90991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle break; 91091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 91191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 91275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (removed) 91375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle task.doExec(); 91491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 91575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle else 91675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle stat = false; 91791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle return stat; 91891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 91991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle 92091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle /** 92175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * Tries to poll for and execute the given task or any other 92275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * task in its CountedCompleter computation. 92391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle */ 92475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle final boolean pollAndExecCC(CountedCompleter<?> root) { 92575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle ForkJoinTask<?>[] a; int b; Object o; CountedCompleter<?> t, r; 92675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if ((b = base) - top < 0 && (a = array) != null) { 92791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle long j = (((a.length - 1) & b) << ASHIFT) + ABASE; 92875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if ((o = U.getObjectVolatile(a, j)) == null) 92975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle return true; // retry 93075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (o instanceof CountedCompleter) { 93175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle for (t = (CountedCompleter<?>)o, r = t;;) { 93275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (r == root) { 93375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (base == b && 93475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle U.compareAndSwapObject(a, j, t, null)) { 93575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle U.putOrderedInt(this, QBASE, b + 1); 93675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle t.doExec(); 93775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle } 93891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle return true; 93991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 94075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle else if ((r = r.completer) == null) 94175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle break; // not part of root computation 94291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 94391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 94491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 94591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle return false; 94691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 94791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle 94891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle /** 94975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * Tries to pop and execute the given task or any other task 95075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * in its CountedCompleter computation. 95191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle */ 95275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle final boolean externalPopAndExecCC(CountedCompleter<?> root) { 95375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle ForkJoinTask<?>[] a; int s; Object o; CountedCompleter<?> t, r; 95475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (base - (s = top) < 0 && (a = array) != null) { 95575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle long j = (((a.length - 1) & (s - 1)) << ASHIFT) + ABASE; 95675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if ((o = U.getObject(a, j)) instanceof CountedCompleter) { 95775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle for (t = (CountedCompleter<?>)o, r = t;;) { 95875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (r == root) { 95975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (U.compareAndSwapInt(this, QLOCK, 0, 1)) { 96075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (top == s && array == a && 96175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle U.compareAndSwapObject(a, j, t, null)) { 96275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle top = s - 1; 96375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle qlock = 0; 96475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle t.doExec(); 96575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle } 96675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle else 96775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle qlock = 0; 96875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle } 96975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle return true; 97075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle } 97175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle else if ((r = r.completer) == null) 97275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle break; 97375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle } 97491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 97591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 97675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle return false; 97791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 97891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle 97991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle /** 98075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * Internal version 98191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle */ 98275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle final boolean internalPopAndExecCC(CountedCompleter<?> root) { 98375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle ForkJoinTask<?>[] a; int s; Object o; CountedCompleter<?> t, r; 98475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (base - (s = top) < 0 && (a = array) != null) { 98575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle long j = (((a.length - 1) & (s - 1)) << ASHIFT) + ABASE; 98675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if ((o = U.getObject(a, j)) instanceof CountedCompleter) { 98775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle for (t = (CountedCompleter<?>)o, r = t;;) { 98875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (r == root) { 98975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (U.compareAndSwapObject(a, j, t, null)) { 99075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle top = s - 1; 99175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle t.doExec(); 99275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle } 99375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle return true; 99475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle } 99575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle else if ((r = r.completer) == null) 99675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle break; 99775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle } 99875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle } 99991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 100075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle return false; 100191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 100291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle 100391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle /** 100491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Returns true if owned and not known to be blocked. 100591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle */ 100691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle final boolean isApparentlyUnblocked() { 100791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle Thread wt; Thread.State s; 100891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle return (eventCount >= 0 && 100991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle (wt = owner) != null && 101091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle (s = wt.getState()) != Thread.State.BLOCKED && 101191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle s != Thread.State.WAITING && 101291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle s != Thread.State.TIMED_WAITING); 101391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 101491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle 101591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle // Unsafe mechanics 101691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle private static final sun.misc.Unsafe U; 101775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle private static final long QBASE; 101891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle private static final long QLOCK; 101991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle private static final int ABASE; 102091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle private static final int ASHIFT; 102191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle static { 102291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle try { 102391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle U = sun.misc.Unsafe.getUnsafe(); 102491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle Class<?> k = WorkQueue.class; 102591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle Class<?> ak = ForkJoinTask[].class; 102675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle QBASE = U.objectFieldOffset 102775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle (k.getDeclaredField("base")); 102891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle QLOCK = U.objectFieldOffset 102991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle (k.getDeclaredField("qlock")); 103091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle ABASE = U.arrayBaseOffset(ak); 103191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle int scale = U.arrayIndexScale(ak); 103291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if ((scale & (scale - 1)) != 0) 103391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle throw new Error("data type scale not a power of two"); 103491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle ASHIFT = 31 - Integer.numberOfLeadingZeros(scale); 103591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } catch (Exception e) { 103691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle throw new Error(e); 103791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 103891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 1039a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 1040a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 104191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle // static fields (initialized in static initializer below) 104291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle 1043a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 104491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Per-thread submission bookkeeping. Shared across all pools 104591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * to reduce ThreadLocal pollution and because random motion 104691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * to avoid contention in one pool is likely to hold for others. 104791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Lazily initialized on first submission (but null-checked 104891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * in other contexts to avoid unnecessary initialization). 1049a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 105091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle static final ThreadLocal<Submitter> submitters; 1051a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 1052a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 105375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * Creates a new ForkJoinWorkerThread. This factory is used unless 105475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * overridden in ForkJoinPool constructors. 105575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle */ 105675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle public static final ForkJoinWorkerThreadFactory 105775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle defaultForkJoinWorkerThreadFactory; 105875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle 105975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle /** 106091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Permission required for callers of methods that may start or 106191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * kill threads. 1062a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 106391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle private static final RuntimePermission modifyThreadPermission; 1064a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 1065a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 106691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Common (static) pool. Non-null for public use unless a static 106791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * construction exception, but internal usages null-check on use 106891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * to paranoically avoid potential initialization circularities 106991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * as well as to simplify generated code. 1070a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 107175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle static final ForkJoinPool common; 1072a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 1073a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 107475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * Common pool parallelism. To allow simpler use and management 107575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * when common pool threads are disabled, we allow the underlying 107675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * common.parallelism field to be zero, but in that case still report 107775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * parallelism as 1 to reflect resulting caller-runs mechanics. 1078a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 107975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle static final int commonParallelism; 1080a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 1081a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 108291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Sequence number for creating workerNamePrefix. 1083a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 108491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle private static int poolNumberSequence; 1085a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 1086a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 108791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Returns the next sequence number. We don't expect this to 108891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * ever contend, so use simple builtin sync. 1089a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 109091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle private static final synchronized int nextPoolId() { 109191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle return ++poolNumberSequence; 109291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 109391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle 109491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle // static constants 1095a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 1096a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 109791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Initial timeout value (in nanoseconds) for the thread 109891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * triggering quiescence to park waiting for new work. On timeout, 109991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * the thread will instead try to shrink the number of 110091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * workers. The value should be large enough to avoid overly 110191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * aggressive shrinkage during most transient stalls (long GCs 110291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * etc). 1103a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 110491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle private static final long IDLE_TIMEOUT = 2000L * 1000L * 1000L; // 2sec 1105a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 1106a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 110791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Timeout value when there are more threads than parallelism level 1108a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 110991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle private static final long FAST_IDLE_TIMEOUT = 200L * 1000L * 1000L; 1110a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 1111a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 111291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Tolerance for idle timeouts, to cope with timer undershoots 1113a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 111475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle private static final long TIMEOUT_SLOP = 2000000L; 1115a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 1116a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 111791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * The maximum stolen->joining link depth allowed in method 111891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * tryHelpStealer. Must be a power of two. Depths for legitimate 111991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * chains are unbounded, but we use a fixed constant to avoid 112091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * (otherwise unchecked) cycles and to bound staleness of 112191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * traversal parameters at the expense of sometimes blocking when 112291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * we could be helping. 1123a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 112491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle private static final int MAX_HELP = 64; 1125a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 1126a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 112791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Increment for seed generators. See class ThreadLocal for 112891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * explanation. 1129a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 113091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle private static final int SEED_INCREMENT = 0x61c88647; 1131a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 113291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle /* 113391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Bits and masks for control variables 113491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * 113591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Field ctl is a long packed with: 1136a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * AC: Number of active running workers minus target parallelism (16 bits) 1137a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * TC: Number of total workers minus target parallelism (16 bits) 1138a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * ST: true if pool is terminating (1 bit) 1139a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * EC: the wait count of top waiting thread (15 bits) 114091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * ID: poolIndex of top of Treiber stack of waiters (16 bits) 1141a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * 1142a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * When convenient, we can extract the upper 32 bits of counts and 1143a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * the lower 32 bits of queue state, u = (int)(ctl >>> 32) and e = 1144a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * (int)ctl. The ec field is never accessed alone, but always 1145a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * together with id and st. The offsets of counts by the target 1146a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * parallelism and the positionings of fields makes it possible to 1147a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * perform the most common checks via sign tests of fields: When 1148a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * ac is negative, there are not enough active workers, when tc is 114991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * negative, there are not enough total workers, and when e is 1150a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * negative, the pool is terminating. To deal with these possibly 1151a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * negative fields, we use casts in and out of "short" and/or 1152a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * signed shifts to maintain signedness. 115391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * 115491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * When a thread is queued (inactivated), its eventCount field is 115591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * set negative, which is the only way to tell if a worker is 115691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * prevented from executing tasks, even though it must continue to 115791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * scan for them to avoid queuing races. Note however that 115891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * eventCount updates lag releases so usage requires care. 115991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * 116091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Field plock is an int packed with: 116191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * SHUTDOWN: true if shutdown is enabled (1 bit) 116291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * SEQ: a sequence lock, with PL_LOCK bit set if locked (30 bits) 116391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * SIGNAL: set when threads may be waiting on the lock (1 bit) 116491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * 116591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * The sequence number enables simple consistency checks: 116691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Staleness of read-only operations on the workQueues array can 116791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * be checked by comparing plock before vs after the reads. 1168a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 1169a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 1170a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson // bit positions/shifts for fields 1171a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson private static final int AC_SHIFT = 48; 1172a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson private static final int TC_SHIFT = 32; 1173a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson private static final int ST_SHIFT = 31; 1174a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson private static final int EC_SHIFT = 16; 1175a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 1176a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson // bounds 117791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle private static final int SMASK = 0xffff; // short bits 117891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle private static final int MAX_CAP = 0x7fff; // max #workers - 1 117991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle private static final int EVENMASK = 0xfffe; // even short bits 118091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle private static final int SQMASK = 0x007e; // max 64 (even) slots 1181a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson private static final int SHORT_SIGN = 1 << 15; 1182a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson private static final int INT_SIGN = 1 << 31; 1183a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 1184a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson // masks 1185a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson private static final long STOP_BIT = 0x0001L << ST_SHIFT; 1186a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson private static final long AC_MASK = ((long)SMASK) << AC_SHIFT; 1187a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson private static final long TC_MASK = ((long)SMASK) << TC_SHIFT; 1188a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 1189a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson // units for incrementing and decrementing 1190a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson private static final long TC_UNIT = 1L << TC_SHIFT; 1191a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson private static final long AC_UNIT = 1L << AC_SHIFT; 1192a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 1193a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson // masks and units for dealing with u = (int)(ctl >>> 32) 1194a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson private static final int UAC_SHIFT = AC_SHIFT - 32; 1195a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson private static final int UTC_SHIFT = TC_SHIFT - 32; 1196a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson private static final int UAC_MASK = SMASK << UAC_SHIFT; 1197a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson private static final int UTC_MASK = SMASK << UTC_SHIFT; 1198a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson private static final int UAC_UNIT = 1 << UAC_SHIFT; 1199a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson private static final int UTC_UNIT = 1 << UTC_SHIFT; 1200a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 1201a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson // masks and units for dealing with e = (int)ctl 120291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle private static final int E_MASK = 0x7fffffff; // no STOP_BIT 120391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle private static final int E_SEQ = 1 << EC_SHIFT; 1204a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 120591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle // plock bits 120691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle private static final int SHUTDOWN = 1 << 31; 120791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle private static final int PL_LOCK = 2; 120891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle private static final int PL_SIGNAL = 1; 120991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle private static final int PL_SPINS = 1 << 8; 1210a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 121191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle // access mode for WorkQueue 121291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle static final int LIFO_QUEUE = 0; 121391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle static final int FIFO_QUEUE = 1; 121491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle static final int SHARED_QUEUE = -1; 1215a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 121691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle // Heuristic padding to ameliorate unfortunate memory placements 121791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle volatile long pad00, pad01, pad02, pad03, pad04, pad05, pad06; 121891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle 121975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle // Instance fields 122091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle volatile long stealCount; // collects worker counts 122191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle volatile long ctl; // main pool control 122291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle volatile int plock; // shutdown status and seqLock 122391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle volatile int indexSeed; // worker/submitter index seed 122475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle final short parallelism; // parallelism level 122575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle final short mode; // LIFO/FIFO 122691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle WorkQueue[] workQueues; // main registry 122791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle final ForkJoinWorkerThreadFactory factory; 122875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle final UncaughtExceptionHandler ueh; // per-worker UEH 122991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle final String workerNamePrefix; // to create worker name string 123091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle 123191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle volatile Object pad10, pad11, pad12, pad13, pad14, pad15, pad16, pad17; 123291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle volatile Object pad18, pad19, pad1a, pad1b; 123391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle 123491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle /** 123591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Acquires the plock lock to protect worker array and related 123691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * updates. This method is called only if an initial CAS on plock 123791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * fails. This acts as a spinlock for normal cases, but falls back 123891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * to builtin monitor to block when (rarely) needed. This would be 123991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * a terrible idea for a highly contended lock, but works fine as 124091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * a more conservative alternative to a pure spinlock. 124191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle */ 124291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle private int acquirePlock() { 124375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle int spins = PL_SPINS, ps, nps; 124491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle for (;;) { 124591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (((ps = plock) & PL_LOCK) == 0 && 124691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle U.compareAndSwapInt(this, PLOCK, ps, nps = ps + PL_LOCK)) 124791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle return nps; 124891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle else if (spins >= 0) { 124975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (ThreadLocalRandom.current().nextInt() >= 0) 125091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle --spins; 125191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 125291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle else if (U.compareAndSwapInt(this, PLOCK, ps, ps | PL_SIGNAL)) { 125391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle synchronized (this) { 125491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if ((plock & PL_SIGNAL) != 0) { 125591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle try { 125691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle wait(); 125791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } catch (InterruptedException ie) { 125891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle try { 125991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle Thread.currentThread().interrupt(); 126091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } catch (SecurityException ignore) { 126191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 126291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 126391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 126491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle else 126591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle notifyAll(); 126691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 126791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 126891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 126991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 1270a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 1271a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 127291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Unlocks and signals any thread waiting for plock. Called only 127391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * when CAS of seq value for unlock fails. 1274a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 127591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle private void releasePlock(int ps) { 127691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle plock = ps; 127791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle synchronized (this) { notifyAll(); } 127891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 1279a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 1280a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 128191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Tries to create and start one worker if fewer than target 128291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * parallelism level exist. Adjusts counts etc on failure. 128391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle */ 128491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle private void tryAddWorker() { 128575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle long c; int u, e; 128691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle while ((u = (int)((c = ctl) >>> 32)) < 0 && 128775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle (u & SHORT_SIGN) != 0 && (e = (int)c) >= 0) { 128875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle long nc = ((long)(((u + UTC_UNIT) & UTC_MASK) | 128975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle ((u + UAC_UNIT) & UAC_MASK)) << 32) | (long)e; 129091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (U.compareAndSwapLong(this, CTL, c, nc)) { 129191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle ForkJoinWorkerThreadFactory fac; 129291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle Throwable ex = null; 129391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle ForkJoinWorkerThread wt = null; 129491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle try { 129591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if ((fac = factory) != null && 129691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle (wt = fac.newThread(this)) != null) { 129791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle wt.start(); 129891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle break; 129991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 130075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle } catch (Throwable rex) { 130175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle ex = rex; 130291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 130391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle deregisterWorker(wt, ex); 130491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle break; 130591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 1306a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 1307a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 1308a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 130991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle // Registering and deregistering workers 1310a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 1311a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 131291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Callback from ForkJoinWorkerThread to establish and record its 131391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * WorkQueue. To avoid scanning bias due to packing entries in 131491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * front of the workQueues array, we treat the array as a simple 131591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * power-of-two hash table using per-thread seed as hash, 131691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * expanding as needed. 131791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * 131891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * @param wt the worker thread 131991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * @return the worker's queue 1320a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 132191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle final WorkQueue registerWorker(ForkJoinWorkerThread wt) { 132275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle UncaughtExceptionHandler handler; WorkQueue[] ws; int s, ps; 132391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle wt.setDaemon(true); 132491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if ((handler = ueh) != null) 132591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle wt.setUncaughtExceptionHandler(handler); 132691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle do {} while (!U.compareAndSwapInt(this, INDEXSEED, s = indexSeed, 132791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle s += SEED_INCREMENT) || 132891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle s == 0); // skip 0 132975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle WorkQueue w = new WorkQueue(this, wt, mode, s); 133091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (((ps = plock) & PL_LOCK) != 0 || 133191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle !U.compareAndSwapInt(this, PLOCK, ps, ps += PL_LOCK)) 133291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle ps = acquirePlock(); 133391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle int nps = (ps & SHUTDOWN) | ((ps + PL_LOCK) & ~SHUTDOWN); 133491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle try { 133591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if ((ws = workQueues) != null) { // skip if shutting down 133691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle int n = ws.length, m = n - 1; 133791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle int r = (s << 1) | 1; // use odd-numbered indices 133891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (ws[r &= m] != null) { // collision 133991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle int probes = 0; // step by approx half size 134091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle int step = (n <= 4) ? 2 : ((n >>> 1) & EVENMASK) + 2; 134191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle while (ws[r = (r + step) & m] != null) { 134291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (++probes >= n) { 134391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle workQueues = ws = Arrays.copyOf(ws, n <<= 1); 134491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle m = n - 1; 134591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle probes = 0; 134691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 134791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 1348a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 134975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle w.poolIndex = (short)r; 135075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle w.eventCount = r; // volatile write orders 135191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle ws[r] = w; 1352a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 135391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } finally { 135491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (!U.compareAndSwapInt(this, PLOCK, ps, nps)) 135591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle releasePlock(nps); 135691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 135775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle wt.setName(workerNamePrefix.concat(Integer.toString(w.poolIndex >>> 1))); 135891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle return w; 135991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 136091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle 136191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle /** 136291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Final callback from terminating worker, as well as upon failure 136391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * to construct or start a worker. Removes record of worker from 136491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * array, and adjusts counts. If pool is shutting down, tries to 136591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * complete termination. 136691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * 136775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * @param wt the worker thread, or null if construction failed 136891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * @param ex the exception causing failure, or null if none 136991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle */ 137091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle final void deregisterWorker(ForkJoinWorkerThread wt, Throwable ex) { 137191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle WorkQueue w = null; 137291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (wt != null && (w = wt.workQueue) != null) { 137375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle int ps; long sc; 137491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle w.qlock = -1; // ensure set 137591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle do {} while (!U.compareAndSwapLong(this, STEALCOUNT, 137675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle sc = stealCount, 137775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle sc + w.nsteals)); 137891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (((ps = plock) & PL_LOCK) != 0 || 137991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle !U.compareAndSwapInt(this, PLOCK, ps, ps += PL_LOCK)) 138091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle ps = acquirePlock(); 138191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle int nps = (ps & SHUTDOWN) | ((ps + PL_LOCK) & ~SHUTDOWN); 138291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle try { 138391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle int idx = w.poolIndex; 138491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle WorkQueue[] ws = workQueues; 138591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (ws != null && idx >= 0 && idx < ws.length && ws[idx] == w) 138691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle ws[idx] = null; 138791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } finally { 138891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (!U.compareAndSwapInt(this, PLOCK, ps, nps)) 138991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle releasePlock(nps); 1390a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 1391a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 1392a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 139391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle long c; // adjust ctl counts 139491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle do {} while (!U.compareAndSwapLong 139591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle (this, CTL, c = ctl, (((c - AC_UNIT) & AC_MASK) | 139691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle ((c - TC_UNIT) & TC_MASK) | 139791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle (c & ~(AC_MASK|TC_MASK))))); 139891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle 139991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (!tryTerminate(false, false) && w != null && w.array != null) { 140091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle w.cancelAll(); // cancel remaining tasks 140191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle WorkQueue[] ws; WorkQueue v; Thread p; int u, i, e; 140291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle while ((u = (int)((c = ctl) >>> 32)) < 0 && (e = (int)c) >= 0) { 140391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (e > 0) { // activate or create replacement 140491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if ((ws = workQueues) == null || 140591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle (i = e & SMASK) >= ws.length || 140675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle (v = ws[i]) == null) 140791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle break; 140891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle long nc = (((long)(v.nextWait & E_MASK)) | 140991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle ((long)(u + UAC_UNIT) << 32)); 141091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (v.eventCount != (e | INT_SIGN)) 141191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle break; 141291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (U.compareAndSwapLong(this, CTL, c, nc)) { 141391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle v.eventCount = (e + E_SEQ) & E_MASK; 141491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if ((p = v.parker) != null) 141591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle U.unpark(p); 141691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle break; 141791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 141891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 141991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle else { 142091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if ((short)u < 0) 142191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle tryAddWorker(); 142291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle break; 142391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 142491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 1425a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 142691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (ex == null) // help clean refs on way out 142791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle ForkJoinTask.helpExpungeStaleExceptions(); 142891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle else // rethrow 142991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle ForkJoinTask.rethrow(ex); 1430a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 1431a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 143291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle // Submissions 1433a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 1434a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 143575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * Per-thread records for threads that submit to pools. Currently 143675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * holds only pseudo-random seed / index that is used to choose 143775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * submission queues in method externalPush. In the future, this may 143875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * also incorporate a means to implement different task rejection 143975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * and resubmission policies. 144075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * 144175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * Seeds for submitters and workers/workQueues work in basically 144275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * the same way but are initialized and updated using slightly 144375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * different mechanics. Both are initialized using the same 144475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * approach as in class ThreadLocal, where successive values are 144575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * unlikely to collide with previous values. Seeds are then 144675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * randomly modified upon collisions using xorshifts, which 144775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * requires a non-zero seed. 144875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle */ 144975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle static final class Submitter { 145075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle int seed; 145175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle Submitter(int s) { seed = s; } 145275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle } 145375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle 145475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle /** 145591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Unless shutting down, adds the given task to a submission queue 145691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * at submitter's current queue index (modulo submission 145791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * range). Only the most common path is directly handled in this 145891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * method. All others are relayed to fullExternalPush. 145991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * 146091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * @param task the task. Caller must ensure non-null. 146191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle */ 146291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle final void externalPush(ForkJoinTask<?> task) { 146375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle Submitter z = submitters.get(); 146475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle WorkQueue q; int r, m, s, n, am; ForkJoinTask<?>[] a; 146575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle int ps = plock; 146675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle WorkQueue[] ws = workQueues; 146775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (z != null && ps > 0 && ws != null && (m = (ws.length - 1)) >= 0 && 146875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle (q = ws[m & (r = z.seed) & SQMASK]) != null && r != 0 && 146991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle U.compareAndSwapInt(q, QLOCK, 0, 1)) { // lock 147075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if ((a = q.array) != null && 147175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle (am = a.length - 1) > (n = (s = q.top) - q.base)) { 147275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle int j = ((am & s) << ASHIFT) + ABASE; 147391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle U.putOrderedObject(a, j, task); 147491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle q.top = s + 1; // push on to deque 147591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle q.qlock = 0; 147675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (n <= 1) 147775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle signalWork(ws, q); 147891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle return; 147991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 148091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle q.qlock = 0; 148191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 148291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle fullExternalPush(task); 148391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 148491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle 148591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle /** 148691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Full version of externalPush. This method is called, among 148791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * other times, upon the first submission of the first task to the 148875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * pool, so must perform secondary initialization. It also 148975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * detects first submission by an external thread by looking up 149075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * its ThreadLocal, and creates a new shared queue if the one at 149175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * index if empty or contended. The plock lock body must be 149275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * exception-free (so no try/finally) so we optimistically 149375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * allocate new queues outside the lock and throw them away if 149475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * (very rarely) not needed. 149575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * 149675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * Secondary initialization occurs when plock is zero, to create 149775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * workQueue array and set plock to a valid value. This lock body 149875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * must also be exception-free. Because the plock seq value can 149975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * eventually wrap around zero, this method harmlessly fails to 150075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * reinitialize if workQueues exists, while still advancing plock. 150191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle */ 150291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle private void fullExternalPush(ForkJoinTask<?> task) { 150391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle int r = 0; // random index seed 150491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle for (Submitter z = submitters.get();;) { 150591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle WorkQueue[] ws; WorkQueue q; int ps, m, k; 150691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (z == null) { 150791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (U.compareAndSwapInt(this, INDEXSEED, r = indexSeed, 150891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle r += SEED_INCREMENT) && r != 0) 150991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle submitters.set(z = new Submitter(r)); 151091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 151175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle else if (r == 0) { // move to a different index 151291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle r = z.seed; 151375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle r ^= r << 13; // same xorshift as WorkQueues 151491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle r ^= r >>> 17; 151575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle z.seed = r ^= (r << 5); 151691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 151775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if ((ps = plock) < 0) 151891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle throw new RejectedExecutionException(); 151991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle else if (ps == 0 || (ws = workQueues) == null || 152075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle (m = ws.length - 1) < 0) { // initialize workQueues 152175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle int p = parallelism; // find power of two table size 152275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle int n = (p > 1) ? p - 1 : 1; // ensure at least 2 slots 152375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle n |= n >>> 1; n |= n >>> 2; n |= n >>> 4; 152475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle n |= n >>> 8; n |= n >>> 16; n = (n + 1) << 1; 152575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle WorkQueue[] nws = ((ws = workQueues) == null || ws.length == 0 ? 152675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle new WorkQueue[n] : null); 152775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (((ps = plock) & PL_LOCK) != 0 || 152875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle !U.compareAndSwapInt(this, PLOCK, ps, ps += PL_LOCK)) 152975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle ps = acquirePlock(); 153075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (((ws = workQueues) == null || ws.length == 0) && nws != null) 153175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle workQueues = nws; 153275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle int nps = (ps & SHUTDOWN) | ((ps + PL_LOCK) & ~SHUTDOWN); 153375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (!U.compareAndSwapInt(this, PLOCK, ps, nps)) 153475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle releasePlock(nps); 153575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle } 153691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle else if ((q = ws[k = r & m & SQMASK]) != null) { 153791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (q.qlock == 0 && U.compareAndSwapInt(q, QLOCK, 0, 1)) { 153891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle ForkJoinTask<?>[] a = q.array; 153991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle int s = q.top; 154091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle boolean submitted = false; 154191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle try { // locked version of push 154291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if ((a != null && a.length > s + 1 - q.base) || 154391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle (a = q.growArray()) != null) { // must presize 154491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle int j = (((a.length - 1) & s) << ASHIFT) + ABASE; 154591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle U.putOrderedObject(a, j, task); 154691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle q.top = s + 1; 154791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle submitted = true; 154891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 154991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } finally { 155091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle q.qlock = 0; // unlock 155191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 155291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (submitted) { 155375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle signalWork(ws, q); 155491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle return; 155591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 1556a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 155791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle r = 0; // move on failure 1558a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 155991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle else if (((ps = plock) & PL_LOCK) == 0) { // create new queue 156091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle q = new WorkQueue(this, null, SHARED_QUEUE, r); 156175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle q.poolIndex = (short)k; 156291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (((ps = plock) & PL_LOCK) != 0 || 156391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle !U.compareAndSwapInt(this, PLOCK, ps, ps += PL_LOCK)) 156491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle ps = acquirePlock(); 156591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if ((ws = workQueues) != null && k < ws.length && ws[k] == null) 156691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle ws[k] = q; 156791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle int nps = (ps & SHUTDOWN) | ((ps + PL_LOCK) & ~SHUTDOWN); 156891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (!U.compareAndSwapInt(this, PLOCK, ps, nps)) 156991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle releasePlock(nps); 1570a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 1571a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson else 157275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle r = 0; 1573a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 157491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 157591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle 157691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle // Maintaining ctl counts 157791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle 157891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle /** 157991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Increments active count; mainly called upon return from blocking. 158091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle */ 158191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle final void incrementActiveCount() { 158291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle long c; 158375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle do {} while (!U.compareAndSwapLong 158475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle (this, CTL, c = ctl, ((c & ~AC_MASK) | 158575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle ((c & AC_MASK) + AC_UNIT)))); 158691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 158791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle 158891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle /** 158991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Tries to create or activate a worker if too few are active. 159091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * 159175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * @param ws the worker array to use to find signallees 159275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * @param q if non-null, the queue holding tasks to be processed 159391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle */ 159475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle final void signalWork(WorkQueue[] ws, WorkQueue q) { 159575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle for (;;) { 159675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle long c; int e, u, i; WorkQueue w; Thread p; 159775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if ((u = (int)((c = ctl) >>> 32)) >= 0) 159875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle break; 159975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if ((e = (int)c) <= 0) { 160091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if ((short)u < 0) 160191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle tryAddWorker(); 160291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle break; 1603a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 160475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (ws == null || ws.length <= (i = e & SMASK) || 160575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle (w = ws[i]) == null) 160675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle break; 160775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle long nc = (((long)(w.nextWait & E_MASK)) | 160875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle ((long)(u + UAC_UNIT)) << 32); 160975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle int ne = (e + E_SEQ) & E_MASK; 161075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (w.eventCount == (e | INT_SIGN) && 161175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle U.compareAndSwapLong(this, CTL, c, nc)) { 161275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle w.eventCount = ne; 161375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if ((p = w.parker) != null) 161475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle U.unpark(p); 161575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle break; 161675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle } 161775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (q != null && q.base >= q.top) 161875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle break; 1619a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 1620a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 1621a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 162291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle // Scanning for tasks 162391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle 162491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle /** 162591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Top-level runloop for workers, called by ForkJoinWorkerThread.run. 162691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle */ 162791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle final void runWorker(WorkQueue w) { 162891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle w.growArray(); // allocate queue 162975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle for (int r = w.hint; scan(w, r) == 0; ) { 163075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle r ^= r << 13; r ^= r >>> 17; r ^= r << 5; // xorshift 163175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle } 163291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 163391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle 1634a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 163575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * Scans for and, if found, runs one task, else possibly 163691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * inactivates the worker. This method operates on single reads of 163791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * volatile state and is designed to be re-invoked continuously, 163891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * in part because it returns upon detecting inconsistencies, 163991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * contention, or state changes that indicate possible success on 164091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * re-invocation. 1641a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * 164275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * The scan searches for tasks across queues starting at a random 164375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * index, checking each at least twice. The scan terminates upon 164475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * either finding a non-empty queue, or completing the sweep. If 164575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * the worker is not inactivated, it takes and runs a task from 164675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * this queue. Otherwise, if not activated, it tries to activate 164775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * itself or some other worker by signalling. On failure to find a 164875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * task, returns (for retry) if pool state may have changed during 164975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * an empty scan, or tries to inactivate if active, else possibly 165075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * blocks or terminates via method awaitWork. 165191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * 165291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * @param w the worker (via its WorkQueue) 165375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * @param r a random seed 165475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * @return worker qlock status if would have waited, else 0 165591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle */ 165675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle private final int scan(WorkQueue w, int r) { 165791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle WorkQueue[] ws; int m; 165875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle long c = ctl; // for consistency check 165975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if ((ws = workQueues) != null && (m = ws.length - 1) >= 0 && w != null) { 166075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle for (int j = m + m + 1, ec = w.eventCount;;) { 166175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle WorkQueue q; int b, e; ForkJoinTask<?>[] a; ForkJoinTask<?> t; 166275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if ((q = ws[(r - j) & m]) != null && 166375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle (b = q.base) - q.top < 0 && (a = q.array) != null) { 166475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle long i = (((a.length - 1) & b) << ASHIFT) + ABASE; 166575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if ((t = ((ForkJoinTask<?>) 166675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle U.getObjectVolatile(a, i))) != null) { 166775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (ec < 0) 166875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle helpRelease(c, ws, w, q, b); 166975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle else if (q.base == b && 167075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle U.compareAndSwapObject(a, i, t, null)) { 167175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle U.putOrderedInt(q, QBASE, b + 1); 167275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if ((b + 1) - q.top < 0) 167375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle signalWork(ws, q); 167475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle w.runTask(t); 167575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle } 1676a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 167775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle break; 1678a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 167975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle else if (--j < 0) { 168075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if ((ec | (e = (int)c)) < 0) // inactive or terminating 168175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle return awaitWork(w, c, ec); 168275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle else if (ctl == c) { // try to inactivate and enqueue 168375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle long nc = (long)ec | ((c - AC_UNIT) & (AC_MASK|TC_MASK)); 168475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle w.nextWait = e; 168591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle w.eventCount = ec | INT_SIGN; 168675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (!U.compareAndSwapLong(this, CTL, c, nc)) 168775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle w.eventCount = ec; // back out 168891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 168975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle break; 1690a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 1691a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 1692a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 169375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle return 0; 1694a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 1695a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 1696a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 169775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * A continuation of scan(), possibly blocking or terminating 169875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * worker w. Returns without blocking if pool state has apparently 169975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * changed since last invocation. Also, if inactivating w has 170075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * caused the pool to become quiescent, checks for pool 170175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * termination, and, so long as this is not the only worker, waits 170275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * for event for up to a given duration. On timeout, if ctl has 170375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * not changed, terminates the worker, which will in turn wake up 170475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * another worker to possibly repeat this process. 1705a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * 1706a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @param w the calling worker 170775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * @param c the ctl value on entry to scan 170875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * @param ec the worker's eventCount on entry to scan 170975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle */ 171075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle private final int awaitWork(WorkQueue w, long c, int ec) { 171175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle int stat, ns; long parkTime, deadline; 171275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if ((stat = w.qlock) >= 0 && w.eventCount == ec && ctl == c && 171375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle !Thread.interrupted()) { 171475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle int e = (int)c; 171575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle int u = (int)(c >>> 32); 171675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle int d = (u >> UAC_SHIFT) + parallelism; // active count 171775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle 171875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (e < 0 || (d <= 0 && tryTerminate(false, false))) 171975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle stat = w.qlock = -1; // pool is terminating 172075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle else if ((ns = w.nsteals) != 0) { // collect steals and retry 172175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle long sc; 172275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle w.nsteals = 0; 172375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle do {} while (!U.compareAndSwapLong(this, STEALCOUNT, 172475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle sc = stealCount, sc + ns)); 172575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle } 172675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle else { 172775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle long pc = ((d > 0 || ec != (e | INT_SIGN)) ? 0L : 172875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle ((long)(w.nextWait & E_MASK)) | // ctl to restore 172975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle ((long)(u + UAC_UNIT)) << 32); 173075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (pc != 0L) { // timed wait if last waiter 173175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle int dc = -(short)(c >>> TC_SHIFT); 173275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle parkTime = (dc < 0 ? FAST_IDLE_TIMEOUT: 173375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle (dc + 1) * IDLE_TIMEOUT); 173475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle deadline = System.nanoTime() + parkTime - TIMEOUT_SLOP; 173575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle } 173675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle else 173775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle parkTime = deadline = 0L; 173875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (w.eventCount == ec && ctl == c) { 173975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle Thread wt = Thread.currentThread(); 174075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle U.putObject(wt, PARKBLOCKER, this); 174175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle w.parker = wt; // emulate LockSupport.park 174275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (w.eventCount == ec && ctl == c) 174375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle U.park(false, parkTime); // must recheck before park 174475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle w.parker = null; 174575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle U.putObject(wt, PARKBLOCKER, null); 174675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (parkTime != 0L && ctl == c && 174775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle deadline - System.nanoTime() <= 0L && 174875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle U.compareAndSwapLong(this, CTL, c, pc)) 174975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle stat = w.qlock = -1; // shrink pool 1750a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 1751a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 1752a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 175375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle return stat; 1754a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 1755a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 1756a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 175775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * Possibly releases (signals) a worker. Called only from scan() 175875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * when a worker with apparently inactive status finds a non-empty 175975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * queue. This requires revalidating all of the associated state 176075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * from caller. 176175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle */ 176275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle private final void helpRelease(long c, WorkQueue[] ws, WorkQueue w, 176375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle WorkQueue q, int b) { 176475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle WorkQueue v; int e, i; Thread p; 176575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (w != null && w.eventCount < 0 && (e = (int)c) > 0 && 176675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle ws != null && ws.length > (i = e & SMASK) && 176775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle (v = ws[i]) != null && ctl == c) { 176875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle long nc = (((long)(v.nextWait & E_MASK)) | 176975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle ((long)((int)(c >>> 32) + UAC_UNIT)) << 32); 177075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle int ne = (e + E_SEQ) & E_MASK; 177175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (q != null && q.base == b && w.eventCount < 0 && 177275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle v.eventCount == (e | INT_SIGN) && 177375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle U.compareAndSwapLong(this, CTL, c, nc)) { 177475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle v.eventCount = ne; 177575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if ((p = v.parker) != null) 177675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle U.unpark(p); 1777a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 1778a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 1779a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 1780a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 1781a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 178291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Tries to locate and execute tasks for a stealer of the given 178391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * task, or in turn one of its stealers, Traces currentSteal -> 178491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * currentJoin links looking for a thread working on a descendant 178591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * of the given task and with a non-empty queue to steal back and 178691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * execute tasks from. The first call to this method upon a 178791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * waiting join will often entail scanning/search, (which is OK 178891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * because the joiner has nothing better to do), but this method 178991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * leaves hints in workers to speed up subsequent calls. The 179091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * implementation is very branchy to cope with potential 179191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * inconsistencies or loops encountering chains that are stale, 179291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * unknown, or so long that they are likely cyclic. 179391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * 179491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * @param joiner the joining worker 179591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * @param task the task to join 179691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * @return 0 if no progress can be made, negative if task 179791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * known complete, else positive 179891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle */ 179991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle private int tryHelpStealer(WorkQueue joiner, ForkJoinTask<?> task) { 180091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle int stat = 0, steps = 0; // bound to avoid cycles 180175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (task != null && joiner != null && 180275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle joiner.base - joiner.top >= 0) { // hoist checks 180391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle restart: for (;;) { 180491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle ForkJoinTask<?> subtask = task; // current target 180591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle for (WorkQueue j = joiner, v;;) { // v is stealer of subtask 180691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle WorkQueue[] ws; int m, s, h; 180791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if ((s = task.status) < 0) { 180891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle stat = s; 180991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle break restart; 1810a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 181191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if ((ws = workQueues) == null || (m = ws.length - 1) <= 0) 181291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle break restart; // shutting down 181391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if ((v = ws[h = (j.hint | 1) & m]) == null || 181491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle v.currentSteal != subtask) { 181591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle for (int origin = h;;) { // find stealer 181691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (((h = (h + 2) & m) & 15) == 1 && 181791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle (subtask.status < 0 || j.currentJoin != subtask)) 181891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle continue restart; // occasional staleness check 181991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if ((v = ws[h]) != null && 182091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle v.currentSteal == subtask) { 182191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle j.hint = h; // save hint 182291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle break; 182391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 182491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (h == origin) 182591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle break restart; // cannot find stealer 182691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 182791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 182891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle for (;;) { // help stealer or descend to its stealer 182975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle ForkJoinTask[] a; int b; 183091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (subtask.status < 0) // surround probes with 183191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle continue restart; // consistency checks 183291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if ((b = v.base) - v.top < 0 && (a = v.array) != null) { 183391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle int i = (((a.length - 1) & b) << ASHIFT) + ABASE; 183491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle ForkJoinTask<?> t = 183591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle (ForkJoinTask<?>)U.getObjectVolatile(a, i); 183691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (subtask.status < 0 || j.currentJoin != subtask || 183791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle v.currentSteal != subtask) 183891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle continue restart; // stale 183991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle stat = 1; // apparent progress 184075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (v.base == b) { 184175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (t == null) 184275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle break restart; 184375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (U.compareAndSwapObject(a, i, t, null)) { 184475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle U.putOrderedInt(v, QBASE, b + 1); 184575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle ForkJoinTask<?> ps = joiner.currentSteal; 184675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle int jt = joiner.top; 184775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle do { 184875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle joiner.currentSteal = t; 184975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle t.doExec(); // clear local tasks too 185075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle } while (task.status >= 0 && 185175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle joiner.top != jt && 185275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle (t = joiner.pop()) != null); 185375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle joiner.currentSteal = ps; 185475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle break restart; 185575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle } 185691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 185791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 185891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle else { // empty -- try to descend 185991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle ForkJoinTask<?> next = v.currentJoin; 186091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (subtask.status < 0 || j.currentJoin != subtask || 186191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle v.currentSteal != subtask) 186291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle continue restart; // stale 186391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle else if (next == null || ++steps == MAX_HELP) 186491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle break restart; // dead-end or maybe cyclic 186591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle else { 186691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle subtask = next; 186791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle j = v; 186891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle break; 186991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 187091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 1871a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 1872a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 187391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 1874a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 187591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle return stat; 1876a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 1877a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 1878a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 187991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Analog of tryHelpStealer for CountedCompleters. Tries to steal 188091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * and run tasks within the target's computation. 1881a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * 188291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * @param task the task to join 188375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle */ 188475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle private int helpComplete(WorkQueue joiner, CountedCompleter<?> task) { 188575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle WorkQueue[] ws; int m; 188675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle int s = 0; 188775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if ((ws = workQueues) != null && (m = ws.length - 1) >= 0 && 188875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle joiner != null && task != null) { 188975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle int j = joiner.poolIndex; 189075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle int scans = m + m + 1; 189175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle long c = 0L; // for stability check 189275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle for (int k = scans; ; j += 2) { 189375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle WorkQueue q; 189491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if ((s = task.status) < 0) 189575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle break; 189675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle else if (joiner.internalPopAndExecCC(task)) 189775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle k = scans; 189875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle else if ((s = task.status) < 0) 189975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle break; 190075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle else if ((q = ws[j & m]) != null && q.pollAndExecCC(task)) 190175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle k = scans; 190275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle else if (--k < 0) { 190375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (c == (c = ctl)) 190491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle break; 190575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle k = scans; 190691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 190791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 190891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 190975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle return s; 191091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 191191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle 191291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle /** 191391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Tries to decrement active count (sometimes implicitly) and 191491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * possibly release or create a compensating worker in preparation 191591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * for blocking. Fails on contention or termination. Otherwise, 191691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * adds a new thread if no idle workers are available and pool 191791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * may become starved. 191875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * 191975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * @param c the assumed ctl value 192091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle */ 192175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle final boolean tryCompensate(long c) { 192275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle WorkQueue[] ws = workQueues; 192375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle int pc = parallelism, e = (int)c, m, tc; 192475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (ws != null && (m = ws.length - 1) >= 0 && e >= 0 && ctl == c) { 192575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle WorkQueue w = ws[e & m]; 192675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (e != 0 && w != null) { 192775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle Thread p; 192891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle long nc = ((long)(w.nextWait & E_MASK) | 192991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle (c & (AC_MASK|TC_MASK))); 193075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle int ne = (e + E_SEQ) & E_MASK; 193175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (w.eventCount == (e | INT_SIGN) && 193275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle U.compareAndSwapLong(this, CTL, c, nc)) { 193375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle w.eventCount = ne; 193491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if ((p = w.parker) != null) 193591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle U.unpark(p); 193691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle return true; // replace with idle worker 193791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 193891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 193991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle else if ((tc = (short)(c >>> TC_SHIFT)) >= 0 && 194091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle (int)(c >> AC_SHIFT) + pc > 1) { 194191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle long nc = ((c - AC_UNIT) & AC_MASK) | (c & ~AC_MASK); 194291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (U.compareAndSwapLong(this, CTL, c, nc)) 194391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle return true; // no compensation 194491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 194591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle else if (tc + pc < MAX_CAP) { 194691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle long nc = ((c + TC_UNIT) & TC_MASK) | (c & ~TC_MASK); 194791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (U.compareAndSwapLong(this, CTL, c, nc)) { 194891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle ForkJoinWorkerThreadFactory fac; 194991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle Throwable ex = null; 195091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle ForkJoinWorkerThread wt = null; 195191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle try { 195291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if ((fac = factory) != null && 195391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle (wt = fac.newThread(this)) != null) { 195491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle wt.start(); 195591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle return true; 195691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 195791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } catch (Throwable rex) { 195891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle ex = rex; 195991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 196091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle deregisterWorker(wt, ex); // clean up and return false 196191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 1962a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 1963a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 196491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle return false; 1965a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 1966a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 1967a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 196891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Helps and/or blocks until the given task is done. 1969a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * 197091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * @param joiner the joining worker 197191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * @param task the task 197291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * @return task status on exit 197391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle */ 197491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle final int awaitJoin(WorkQueue joiner, ForkJoinTask<?> task) { 197591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle int s = 0; 197675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (task != null && (s = task.status) >= 0 && joiner != null) { 197791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle ForkJoinTask<?> prevJoin = joiner.currentJoin; 197891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle joiner.currentJoin = task; 197975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle do {} while (joiner.tryRemoveAndExec(task) && // process local tasks 198075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle (s = task.status) >= 0); 198175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (s >= 0 && (task instanceof CountedCompleter)) 198275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle s = helpComplete(joiner, (CountedCompleter<?>)task); 198375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle long cc = 0; // for stability checks 198491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle while (s >= 0 && (s = task.status) >= 0) { 198575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if ((s = tryHelpStealer(joiner, task)) == 0 && 198691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle (s = task.status) >= 0) { 198775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (!tryCompensate(cc)) 198875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle cc = ctl; 198975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle else { 199091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (task.trySetSignal() && (s = task.status) >= 0) { 199191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle synchronized (task) { 199291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (task.status >= 0) { 199391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle try { // see ForkJoinTask 199491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle task.wait(); // for explanation 199591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } catch (InterruptedException ie) { 199691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 199791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 199891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle else 199991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle task.notifyAll(); 200091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 200191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 200275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle long c; // reactivate 200391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle do {} while (!U.compareAndSwapLong 200475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle (this, CTL, c = ctl, 200575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle ((c & ~AC_MASK) | 200675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle ((c & AC_MASK) + AC_UNIT)))); 2007a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2008a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2009a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 201091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle joiner.currentJoin = prevJoin; 2011a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 201291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle return s; 2013a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2014a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 2015a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 201691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Stripped-down variant of awaitJoin used by timed joins. Tries 201791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * to help join only while there is continuous progress. (Caller 201891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * will then enter a timed wait.) 201991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * 202091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * @param joiner the joining worker 202191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * @param task the task 2022a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 202391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle final void helpJoinOnce(WorkQueue joiner, ForkJoinTask<?> task) { 202491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle int s; 202591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (joiner != null && task != null && (s = task.status) >= 0) { 202691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle ForkJoinTask<?> prevJoin = joiner.currentJoin; 202791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle joiner.currentJoin = task; 202875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle do {} while (joiner.tryRemoveAndExec(task) && // process local tasks 202975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle (s = task.status) >= 0); 203075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (s >= 0) { 203175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (task instanceof CountedCompleter) 203275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle helpComplete(joiner, (CountedCompleter<?>)task); 203391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle do {} while (task.status >= 0 && 203491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle tryHelpStealer(joiner, task) > 0); 203591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 203691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle joiner.currentJoin = prevJoin; 2037a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2038a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2039a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 2040a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 204191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Returns a (probably) non-empty steal queue, if one is found 204275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * during a scan, else null. This method must be retried by 204375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * caller if, by the time it tries to use the queue, it is empty. 204475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle */ 204575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle private WorkQueue findNonEmptyStealQueue() { 204675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle int r = ThreadLocalRandom.current().nextInt(); 204775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle for (;;) { 204875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle int ps = plock, m; WorkQueue[] ws; WorkQueue q; 204975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if ((ws = workQueues) != null && (m = ws.length - 1) >= 0) { 205075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle for (int j = (m + 1) << 2; j >= 0; --j) { 205175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if ((q = ws[(((r - j) << 1) | 1) & m]) != null && 205275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle q.base - q.top < 0) 205375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle return q; 205491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 205591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 205675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (plock == ps) 205775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle return null; 2058a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2059a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2060a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 2061a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 206291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Runs tasks until {@code isQuiescent()}. We piggyback on 206391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * active count ctl maintenance, but rather than blocking 206491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * when tasks cannot be found, we rescan until all others cannot 206591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * find tasks either. 206691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle */ 206791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle final void helpQuiescePool(WorkQueue w) { 206875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle ForkJoinTask<?> ps = w.currentSteal; 206991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle for (boolean active = true;;) { 207075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle long c; WorkQueue q; ForkJoinTask<?> t; int b; 207175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle while ((t = w.nextLocalTask()) != null) 207275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle t.doExec(); 207375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if ((q = findNonEmptyStealQueue()) != null) { 207491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (!active) { // re-establish active count 207591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle active = true; 207691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle do {} while (!U.compareAndSwapLong 207775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle (this, CTL, c = ctl, 207875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle ((c & ~AC_MASK) | 207975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle ((c & AC_MASK) + AC_UNIT)))); 208075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle } 208175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if ((b = q.base) - q.top < 0 && (t = q.pollAt(b)) != null) { 208275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle (w.currentSteal = t).doExec(); 208375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle w.currentSteal = ps; 208491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 208591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 2086edf43d27e240d82106f39ae91404963c23987234Narayan Kamath else if (active) { // decrement active count without queuing 208775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle long nc = ((c = ctl) & ~AC_MASK) | ((c & AC_MASK) - AC_UNIT); 208875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if ((int)(nc >> AC_SHIFT) + parallelism == 0) 208975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle break; // bypass decrement-then-increment 209075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (U.compareAndSwapLong(this, CTL, c, nc)) 209191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle active = false; 209291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 209375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle else if ((int)((c = ctl) >> AC_SHIFT) + parallelism <= 0 && 209475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle U.compareAndSwapLong 209575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle (this, CTL, c, ((c & ~AC_MASK) | 209675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle ((c & AC_MASK) + AC_UNIT)))) 209775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle break; 2098a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2099a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2100a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 2101a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 210291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Gets and removes a local or stolen task for the given worker. 2103a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * 210491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * @return a task, if available 2105a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 210691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle final ForkJoinTask<?> nextTaskFor(WorkQueue w) { 210791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle for (ForkJoinTask<?> t;;) { 210891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle WorkQueue q; int b; 210991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if ((t = w.nextLocalTask()) != null) 211091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle return t; 211175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if ((q = findNonEmptyStealQueue()) == null) 211291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle return null; 211391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if ((b = q.base) - q.top < 0 && (t = q.pollAt(b)) != null) 211491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle return t; 2115a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2116a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2117a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 2118a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 211991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Returns a cheap heuristic guide for task partitioning when 212091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * programmers, frameworks, tools, or languages have little or no 212191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * idea about task granularity. In essence by offering this 212291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * method, we ask users only about tradeoffs in overhead vs 212391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * expected throughput and its variance, rather than how finely to 212491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * partition tasks. 212591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * 212691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * In a steady state strict (tree-structured) computation, each 212791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * thread makes available for stealing enough tasks for other 212891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * threads to remain active. Inductively, if all threads play by 212991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * the same rules, each thread should make available only a 213091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * constant number of tasks. 213191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * 213291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * The minimum useful constant is just 1. But using a value of 1 213391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * would require immediate replenishment upon each steal to 213491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * maintain enough tasks, which is infeasible. Further, 213591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * partitionings/granularities of offered tasks should minimize 213691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * steal rates, which in general means that threads nearer the top 213791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * of computation tree should generate more than those nearer the 213891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * bottom. In perfect steady state, each thread is at 213991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * approximately the same level of computation tree. However, 214091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * producing extra tasks amortizes the uncertainty of progress and 214191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * diffusion assumptions. 214291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * 214391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * So, users will want to use values larger (but not much larger) 214491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * than 1 to both smooth over transient shortages and hedge 214591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * against uneven progress; as traded off against the cost of 214691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * extra task overhead. We leave the user to pick a threshold 214791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * value to compare with the results of this call to guide 214891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * decisions, but recommend values such as 3. 214991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * 215091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * When all threads are active, it is on average OK to estimate 215191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * surplus strictly locally. In steady-state, if one thread is 215291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * maintaining say 2 surplus tasks, then so are others. So we can 215391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * just use estimated queue length. However, this strategy alone 215491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * leads to serious mis-estimates in some non-steady-state 215591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * conditions (ramp-up, ramp-down, other stalls). We can detect 215691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * many of these by further considering the number of "idle" 215791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * threads, that are known to have zero queued tasks, so 215891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * compensate by a factor of (#idle/#active) threads. 215991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * 216091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Note: The approximation of #busy workers as #active workers is 216191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * not very good under current signalling scheme, and should be 216291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * improved. 216391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle */ 216491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle static int getSurplusQueuedTaskCount() { 216591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle Thread t; ForkJoinWorkerThread wt; ForkJoinPool pool; WorkQueue q; 216691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (((t = Thread.currentThread()) instanceof ForkJoinWorkerThread)) { 216775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle int p = (pool = (wt = (ForkJoinWorkerThread)t).pool).parallelism; 216891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle int n = (q = wt.workQueue).top - q.base; 216991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle int a = (int)(pool.ctl >> AC_SHIFT) + p; 217091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle return n - (a > (p >>>= 1) ? 0 : 217191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle a > (p >>>= 1) ? 1 : 217291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle a > (p >>>= 1) ? 2 : 217391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle a > (p >>>= 1) ? 4 : 217491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle 8); 2175a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 217691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle return 0; 2177a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2178a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 217991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle // Termination 2180a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 2181a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 218291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Possibly initiates and/or completes termination. The caller 218391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * triggering termination runs three passes through workQueues: 218491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * (0) Setting termination status, followed by wakeups of queued 218591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * workers; (1) cancelling all tasks; (2) interrupting lagging 218691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * threads (likely in external tasks, but possibly also blocked in 218791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * joins). Each pass repeats previous steps because of potential 218891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * lagging thread creation. 2189a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * 2190a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @param now if true, unconditionally terminate, else only 219191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * if no work and no active workers 219291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * @param enable if true, enable shutdown when next possible 2193a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @return true if now terminating or terminated 2194a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 219591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle private boolean tryTerminate(boolean now, boolean enable) { 219675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle int ps; 219775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (this == common) // cannot shut down 219891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle return false; 219975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if ((ps = plock) >= 0) { // enable by setting plock 220075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (!enable) 220175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle return false; 220275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if ((ps & PL_LOCK) != 0 || 220375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle !U.compareAndSwapInt(this, PLOCK, ps, ps += PL_LOCK)) 220475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle ps = acquirePlock(); 220575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle int nps = ((ps + PL_LOCK) & ~SHUTDOWN) | SHUTDOWN; 220675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (!U.compareAndSwapInt(this, PLOCK, ps, nps)) 220775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle releasePlock(nps); 220875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle } 220991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle for (long c;;) { 221075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (((c = ctl) & STOP_BIT) != 0) { // already terminating 221175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if ((short)(c >>> TC_SHIFT) + parallelism <= 0) { 221291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle synchronized (this) { 221375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle notifyAll(); // signal when 0 workers 221491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 2215a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 221691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle return true; 2217a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 221875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (!now) { // check if idle & no tasks 221975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle WorkQueue[] ws; WorkQueue w; 222075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if ((int)(c >> AC_SHIFT) + parallelism > 0) 222191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle return false; 222275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if ((ws = workQueues) != null) { 222375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle for (int i = 0; i < ws.length; ++i) { 222475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if ((w = ws[i]) != null && 222575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle (!w.isEmpty() || 222675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle ((i & 1) != 0 && w.eventCount >= 0))) { 222775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle signalWork(ws, w); 222891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle return false; 222975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle } 223091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 223191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 223291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 223391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (U.compareAndSwapLong(this, CTL, c, c | STOP_BIT)) { 223491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle for (int pass = 0; pass < 3; ++pass) { 223575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle WorkQueue[] ws; WorkQueue w; Thread wt; 223675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if ((ws = workQueues) != null) { 223791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle int n = ws.length; 223891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle for (int i = 0; i < n; ++i) { 223991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if ((w = ws[i]) != null) { 224091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle w.qlock = -1; 224191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (pass > 0) { 224291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle w.cancelAll(); 224391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (pass > 1 && (wt = w.owner) != null) { 224491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (!wt.isInterrupted()) { 224591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle try { 224691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle wt.interrupt(); 224775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle } catch (Throwable ignore) { 224891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 224991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 225091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle U.unpark(wt); 225191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 2252a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2253a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2254a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 225591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle // Wake up workers parked on event queue 225691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle int i, e; long cc; Thread p; 225791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle while ((e = (int)(cc = ctl) & E_MASK) != 0 && 225875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle (i = e & SMASK) < n && i >= 0 && 225991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle (w = ws[i]) != null) { 226091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle long nc = ((long)(w.nextWait & E_MASK) | 226191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle ((cc + AC_UNIT) & AC_MASK) | 226291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle (cc & (TC_MASK|STOP_BIT))); 226391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (w.eventCount == (e | INT_SIGN) && 226491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle U.compareAndSwapLong(this, CTL, cc, nc)) { 226591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle w.eventCount = (e + E_SEQ) & E_MASK; 226691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle w.qlock = -1; 226791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if ((p = w.parker) != null) 226891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle U.unpark(p); 226991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 227091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 2271a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2272a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2273a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2274a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2275a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2276a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 227791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle // external operations on common pool 227891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle 2279a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 228091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Returns common pool queue for a thread that has submitted at 228191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * least one task. 2282a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 228391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle static WorkQueue commonSubmitterQueue() { 228475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle Submitter z; ForkJoinPool p; WorkQueue[] ws; int m, r; 228591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle return ((z = submitters.get()) != null && 228675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle (p = common) != null && 228791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle (ws = p.workQueues) != null && 228891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle (m = ws.length - 1) >= 0) ? 228991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle ws[m & z.seed & SQMASK] : null; 2290a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2291a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 2292a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 229391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Tries to pop the given task from submitter's queue in common pool. 229491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle */ 229575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle final boolean tryExternalUnpush(ForkJoinTask<?> task) { 229675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle WorkQueue joiner; ForkJoinTask<?>[] a; int m, s; 229775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle Submitter z = submitters.get(); 229875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle WorkQueue[] ws = workQueues; 229975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle boolean popped = false; 230075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (z != null && ws != null && (m = ws.length - 1) >= 0 && 230175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle (joiner = ws[z.seed & m & SQMASK]) != null && 230275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle joiner.base != (s = joiner.top) && 230375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle (a = joiner.array) != null) { 230491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle long j = (((a.length - 1) & (s - 1)) << ASHIFT) + ABASE; 230575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (U.getObject(a, j) == task && 230675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle U.compareAndSwapInt(joiner, QLOCK, 0, 1)) { 230775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (joiner.top == s && joiner.array == a && 230875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle U.compareAndSwapObject(a, j, task, null)) { 230975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle joiner.top = s - 1; 231075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle popped = true; 2311a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 231275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle joiner.qlock = 0; 2313a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2314a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 231575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle return popped; 2316a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2317a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 231875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle final int externalHelpComplete(CountedCompleter<?> task) { 231975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle WorkQueue joiner; int m, j; 232075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle Submitter z = submitters.get(); 232175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle WorkQueue[] ws = workQueues; 232275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle int s = 0; 232375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (z != null && ws != null && (m = ws.length - 1) >= 0 && 232475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle (joiner = ws[(j = z.seed) & m & SQMASK]) != null && task != null) { 232575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle int scans = m + m + 1; 232675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle long c = 0L; // for stability check 232775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle j |= 1; // poll odd queues 232875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle for (int k = scans; ; j += 2) { 232975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle WorkQueue q; 233075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if ((s = task.status) < 0) 233191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle break; 233275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle else if (joiner.externalPopAndExecCC(task)) 233375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle k = scans; 233475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle else if ((s = task.status) < 0) 233591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle break; 233675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle else if ((q = ws[j & m]) != null && q.pollAndExecCC(task)) 233775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle k = scans; 233875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle else if (--k < 0) { 233975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (c == (c = ctl)) 234075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle break; 234175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle k = scans; 234291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 234391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 234491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 234575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle return s; 2346a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2347a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 2348a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson // Exported methods 2349a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 2350a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson // Constructors 2351a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 2352a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 2353a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * Creates a {@code ForkJoinPool} with parallelism equal to {@link 2354a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * java.lang.Runtime#availableProcessors}, using the {@linkplain 2355a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * #defaultForkJoinWorkerThreadFactory default thread factory}, 2356a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * no UncaughtExceptionHandler, and non-async LIFO processing mode. 2357a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 2358a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson public ForkJoinPool() { 235991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle this(Math.min(MAX_CAP, Runtime.getRuntime().availableProcessors()), 2360a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson defaultForkJoinWorkerThreadFactory, null, false); 2361a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2362a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 2363a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 2364a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * Creates a {@code ForkJoinPool} with the indicated parallelism 2365a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * level, the {@linkplain 2366a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * #defaultForkJoinWorkerThreadFactory default thread factory}, 2367a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * no UncaughtExceptionHandler, and non-async LIFO processing mode. 2368a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * 2369a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @param parallelism the parallelism level 2370a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @throws IllegalArgumentException if parallelism less than or 2371a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * equal to zero, or greater than implementation limit 2372a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 2373a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson public ForkJoinPool(int parallelism) { 2374a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson this(parallelism, defaultForkJoinWorkerThreadFactory, null, false); 2375a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2376a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 2377a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 2378a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * Creates a {@code ForkJoinPool} with the given parameters. 2379a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * 2380a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @param parallelism the parallelism level. For default value, 2381a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * use {@link java.lang.Runtime#availableProcessors}. 2382a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @param factory the factory for creating new threads. For default value, 2383a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * use {@link #defaultForkJoinWorkerThreadFactory}. 2384a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @param handler the handler for internal worker threads that 2385a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * terminate due to unrecoverable errors encountered while executing 2386a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * tasks. For default value, use {@code null}. 2387a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @param asyncMode if true, 2388a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * establishes local first-in-first-out scheduling mode for forked 2389a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * tasks that are never joined. This mode may be more appropriate 2390a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * than default locally stack-based mode in applications in which 2391a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * worker threads only process event-style asynchronous tasks. 2392a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * For default value, use {@code false}. 2393a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @throws IllegalArgumentException if parallelism less than or 2394a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * equal to zero, or greater than implementation limit 2395a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @throws NullPointerException if the factory is null 2396a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 2397a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson public ForkJoinPool(int parallelism, 2398a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson ForkJoinWorkerThreadFactory factory, 239975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle UncaughtExceptionHandler handler, 2400a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson boolean asyncMode) { 240175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle this(checkParallelism(parallelism), 240275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle checkFactory(factory), 240375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle handler, 240475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle (asyncMode ? FIFO_QUEUE : LIFO_QUEUE), 240575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle "ForkJoinPool-" + nextPoolId() + "-worker-"); 2406a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson checkPermission(); 240775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle } 240875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle 240975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle private static int checkParallelism(int parallelism) { 241091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (parallelism <= 0 || parallelism > MAX_CAP) 2411a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson throw new IllegalArgumentException(); 241275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle return parallelism; 241375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle } 241475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle 241575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle private static ForkJoinWorkerThreadFactory checkFactory 241675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle (ForkJoinWorkerThreadFactory factory) { 241775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (factory == null) 241875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle throw new NullPointerException(); 241975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle return factory; 2420a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2421a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 242291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle /** 242375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * Creates a {@code ForkJoinPool} with the given parameters, without 242475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * any security checks or parameter validation. Invoked directly by 242575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * makeCommonPool. 242691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle */ 242775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle private ForkJoinPool(int parallelism, 242875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle ForkJoinWorkerThreadFactory factory, 242975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle UncaughtExceptionHandler handler, 243075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle int mode, 243175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle String workerNamePrefix) { 243275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle this.workerNamePrefix = workerNamePrefix; 243391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle this.factory = factory; 243491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle this.ueh = handler; 243575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle this.mode = (short)mode; 243675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle this.parallelism = (short)parallelism; 243775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle long np = (long)(-parallelism); // offset ctl counts 243875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle this.ctl = ((np << AC_SHIFT) & AC_MASK) | ((np << TC_SHIFT) & TC_MASK); 243991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 244091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle 244191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle /** 244275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * Returns the common pool instance. This pool is statically 244375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * constructed; its run state is unaffected by attempts to {@link 244475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * #shutdown} or {@link #shutdownNow}. However this pool and any 244575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * ongoing processing are automatically terminated upon program 244675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * {@link System#exit}. Any program that relies on asynchronous 244775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * task processing to complete before program termination should 244875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * invoke {@code commonPool().}{@link #awaitQuiescence awaitQuiescence}, 244975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * before exit. 245091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * 245191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * @return the common pool instance 245291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * @since 1.8 245391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * @hide 245491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle */ 245591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle public static ForkJoinPool commonPool() { 245675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle // assert common != null : "static init error"; 245775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle return common; 245891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 245991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle 2460a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson // Execution methods 2461a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 2462a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 2463a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * Performs the given task, returning its result upon completion. 2464a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * If the computation encounters an unchecked Exception or Error, 2465a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * it is rethrown as the outcome of this invocation. Rethrown 2466a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * exceptions behave in the same way as regular exceptions, but, 2467a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * when possible, contain stack traces (as displayed for example 2468a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * using {@code ex.printStackTrace()}) of both the current thread 2469a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * as well as the thread actually encountering the exception; 2470a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * minimally only the latter. 2471a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * 2472a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @param task the task 2473a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @return the task's result 2474a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @throws NullPointerException if the task is null 2475a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @throws RejectedExecutionException if the task cannot be 2476a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * scheduled for execution 2477a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 2478a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson public <T> T invoke(ForkJoinTask<T> task) { 2479a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson if (task == null) 2480a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson throw new NullPointerException(); 248191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle externalPush(task); 248291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle return task.join(); 2483a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2484a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 2485a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 2486a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * Arranges for (asynchronous) execution of the given task. 2487a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * 2488a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @param task the task 2489a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @throws NullPointerException if the task is null 2490a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @throws RejectedExecutionException if the task cannot be 2491a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * scheduled for execution 2492a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 2493a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson public void execute(ForkJoinTask<?> task) { 2494a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson if (task == null) 2495a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson throw new NullPointerException(); 249691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle externalPush(task); 2497a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2498a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 2499a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson // AbstractExecutorService methods 2500a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 2501a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 2502a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @throws NullPointerException if the task is null 2503a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @throws RejectedExecutionException if the task cannot be 2504a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * scheduled for execution 2505a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 2506a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson public void execute(Runnable task) { 2507a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson if (task == null) 2508a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson throw new NullPointerException(); 2509a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson ForkJoinTask<?> job; 2510a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson if (task instanceof ForkJoinTask<?>) // avoid re-wrap 2511a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson job = (ForkJoinTask<?>) task; 2512a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson else 251375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle job = new ForkJoinTask.RunnableExecuteAction(task); 251491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle externalPush(job); 2515a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2516a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 2517a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 2518a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * Submits a ForkJoinTask for execution. 2519a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * 2520a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @param task the task to submit 2521a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @return the task 2522a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @throws NullPointerException if the task is null 2523a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @throws RejectedExecutionException if the task cannot be 2524a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * scheduled for execution 2525a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 2526a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson public <T> ForkJoinTask<T> submit(ForkJoinTask<T> task) { 2527a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson if (task == null) 2528a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson throw new NullPointerException(); 252991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle externalPush(task); 2530a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson return task; 2531a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2532a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 2533a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 2534a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @throws NullPointerException if the task is null 2535a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @throws RejectedExecutionException if the task cannot be 2536a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * scheduled for execution 2537a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 2538a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson public <T> ForkJoinTask<T> submit(Callable<T> task) { 253991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle ForkJoinTask<T> job = new ForkJoinTask.AdaptedCallable<T>(task); 254091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle externalPush(job); 2541a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson return job; 2542a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2543a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 2544a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 2545a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @throws NullPointerException if the task is null 2546a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @throws RejectedExecutionException if the task cannot be 2547a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * scheduled for execution 2548a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 2549a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson public <T> ForkJoinTask<T> submit(Runnable task, T result) { 255091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle ForkJoinTask<T> job = new ForkJoinTask.AdaptedRunnable<T>(task, result); 255191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle externalPush(job); 2552a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson return job; 2553a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2554a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 2555a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 2556a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @throws NullPointerException if the task is null 2557a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @throws RejectedExecutionException if the task cannot be 2558a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * scheduled for execution 2559a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 2560a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson public ForkJoinTask<?> submit(Runnable task) { 2561a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson if (task == null) 2562a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson throw new NullPointerException(); 2563a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson ForkJoinTask<?> job; 2564a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson if (task instanceof ForkJoinTask<?>) // avoid re-wrap 2565a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson job = (ForkJoinTask<?>) task; 2566a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson else 256791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle job = new ForkJoinTask.AdaptedRunnableAction(task); 256891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle externalPush(job); 2569a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson return job; 2570a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2571a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 2572a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 2573a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @throws NullPointerException {@inheritDoc} 2574a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @throws RejectedExecutionException {@inheritDoc} 2575a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 2576a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson public <T> List<Future<T>> invokeAll(Collection<? extends Callable<T>> tasks) { 257791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle // In previous versions of this class, this method constructed 257891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle // a task to run ForkJoinTask.invokeAll, but now external 257991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle // invocation of multiple tasks is at least as efficient. 258091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle ArrayList<Future<T>> futures = new ArrayList<Future<T>>(tasks.size()); 258191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle 258291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle boolean done = false; 258391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle try { 258491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle for (Callable<T> t : tasks) { 258591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle ForkJoinTask<T> f = new ForkJoinTask.AdaptedCallable<T>(t); 258691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle futures.add(f); 258791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle externalPush(f); 258891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 258991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle for (int i = 0, size = futures.size(); i < size; i++) 259091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle ((ForkJoinTask<?>)futures.get(i)).quietlyJoin(); 259191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle done = true; 259291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle return futures; 259391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } finally { 259491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (!done) 259591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle for (int i = 0, size = futures.size(); i < size; i++) 259691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle futures.get(i).cancel(false); 2597a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2598a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2599a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 2600a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 2601a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * Returns the factory used for constructing new workers. 2602a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * 2603a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @return the factory used for constructing new workers 2604a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 2605a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson public ForkJoinWorkerThreadFactory getFactory() { 2606a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson return factory; 2607a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2608a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 2609a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 2610a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * Returns the handler for internal worker threads that terminate 2611a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * due to unrecoverable errors encountered while executing tasks. 2612a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * 2613a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @return the handler, or {@code null} if none 2614a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 261575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle public UncaughtExceptionHandler getUncaughtExceptionHandler() { 2616a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson return ueh; 2617a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2618a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 2619a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 2620a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * Returns the targeted parallelism level of this pool. 2621a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * 2622a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @return the targeted parallelism level of this pool 2623a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 2624a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson public int getParallelism() { 262575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle int par; 262675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle return ((par = parallelism) > 0) ? par : 1; 262791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 262891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle 262991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle /** 263091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Returns the targeted parallelism level of the common pool. 263191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * 263291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * @return the targeted parallelism level of the common pool 263391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * @since 1.8 263491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * @hide 263591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle */ 263691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle public static int getCommonPoolParallelism() { 263775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle return commonParallelism; 2638a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2639a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 2640a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 2641a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * Returns the number of worker threads that have started but not 2642a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * yet terminated. The result returned by this method may differ 2643a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * from {@link #getParallelism} when threads are created to 2644a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * maintain parallelism when others are cooperatively blocked. 2645a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * 2646a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @return the number of worker threads 2647a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 2648a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson public int getPoolSize() { 264975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle return parallelism + (short)(ctl >>> TC_SHIFT); 2650a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2651a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 2652a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 2653a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * Returns {@code true} if this pool uses local first-in-first-out 2654a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * scheduling mode for forked tasks that are never joined. 2655a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * 2656a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @return {@code true} if this pool uses async mode 2657a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 2658a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson public boolean getAsyncMode() { 265975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle return mode == FIFO_QUEUE; 2660a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2661a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 2662a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 2663a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * Returns an estimate of the number of worker threads that are 2664a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * not blocked waiting to join tasks or for other managed 2665a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * synchronization. This method may overestimate the 2666a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * number of running threads. 2667a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * 2668a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @return the number of worker threads 2669a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 2670a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson public int getRunningThreadCount() { 267191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle int rc = 0; 267291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle WorkQueue[] ws; WorkQueue w; 267391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if ((ws = workQueues) != null) { 267491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle for (int i = 1; i < ws.length; i += 2) { 267591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if ((w = ws[i]) != null && w.isApparentlyUnblocked()) 267691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle ++rc; 267791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 267891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 267991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle return rc; 2680a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2681a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 2682a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 2683a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * Returns an estimate of the number of threads that are currently 2684a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * stealing or executing tasks. This method may overestimate the 2685a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * number of active threads. 2686a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * 2687a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @return the number of active threads 2688a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 2689a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson public int getActiveThreadCount() { 269075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle int r = parallelism + (int)(ctl >> AC_SHIFT); 2691a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson return (r <= 0) ? 0 : r; // suppress momentarily negative values 2692a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2693a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 2694a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 2695a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * Returns {@code true} if all worker threads are currently idle. 2696a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * An idle worker is one that cannot obtain a task to execute 2697a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * because none are available to steal from other threads, and 2698a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * there are no pending submissions to the pool. This method is 2699a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * conservative; it might not return {@code true} immediately upon 2700a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * idleness of all threads, but will eventually become true if 2701a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * threads remain inactive. 2702a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * 2703a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @return {@code true} if all threads are currently idle 2704a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 2705a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson public boolean isQuiescent() { 270675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle return parallelism + (int)(ctl >> AC_SHIFT) <= 0; 2707a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2708a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 2709a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 2710a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * Returns an estimate of the total number of tasks stolen from 2711a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * one thread's work queue by another. The reported value 2712a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * underestimates the actual total number of steals when the pool 2713a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * is not quiescent. This value may be useful for monitoring and 2714a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * tuning fork/join programs: in general, steal counts should be 2715a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * high enough to keep threads busy, but low enough to avoid 2716a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * overhead and contention across threads. 2717a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * 2718a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @return the number of steals 2719a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 2720a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson public long getStealCount() { 272191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle long count = stealCount; 272291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle WorkQueue[] ws; WorkQueue w; 272391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if ((ws = workQueues) != null) { 272491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle for (int i = 1; i < ws.length; i += 2) { 272591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if ((w = ws[i]) != null) 272691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle count += w.nsteals; 272791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 272891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 272991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle return count; 2730a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2731a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 2732a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 2733a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * Returns an estimate of the total number of tasks currently held 2734a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * in queues by worker threads (but not including tasks submitted 2735a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * to the pool that have not begun executing). This value is only 2736a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * an approximation, obtained by iterating across all threads in 2737a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * the pool. This method may be useful for tuning task 2738a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * granularities. 2739a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * 2740a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @return the number of queued tasks 2741a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 2742a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson public long getQueuedTaskCount() { 2743a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson long count = 0; 274491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle WorkQueue[] ws; WorkQueue w; 274591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if ((ws = workQueues) != null) { 274691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle for (int i = 1; i < ws.length; i += 2) { 274791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if ((w = ws[i]) != null) 274891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle count += w.queueSize(); 274991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 2750a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2751a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson return count; 2752a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2753a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 2754a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 2755a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * Returns an estimate of the number of tasks submitted to this 2756a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * pool that have not yet begun executing. This method may take 2757a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * time proportional to the number of submissions. 2758a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * 2759a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @return the number of queued submissions 2760a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 2761a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson public int getQueuedSubmissionCount() { 276291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle int count = 0; 276391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle WorkQueue[] ws; WorkQueue w; 276491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if ((ws = workQueues) != null) { 276591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle for (int i = 0; i < ws.length; i += 2) { 276691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if ((w = ws[i]) != null) 276791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle count += w.queueSize(); 276891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 276991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 277091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle return count; 2771a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2772a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 2773a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 2774a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * Returns {@code true} if there are any tasks submitted to this 2775a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * pool that have not yet begun executing. 2776a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * 2777a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @return {@code true} if there are any queued submissions 2778a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 2779a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson public boolean hasQueuedSubmissions() { 278091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle WorkQueue[] ws; WorkQueue w; 278191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if ((ws = workQueues) != null) { 278291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle for (int i = 0; i < ws.length; i += 2) { 278391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if ((w = ws[i]) != null && !w.isEmpty()) 278491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle return true; 278591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 278691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 278791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle return false; 2788a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2789a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 2790a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 2791a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * Removes and returns the next unexecuted submission if one is 2792a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * available. This method may be useful in extensions to this 2793a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * class that re-assign work in systems with multiple pools. 2794a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * 2795a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @return the next submission, or {@code null} if none 2796a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 2797a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson protected ForkJoinTask<?> pollSubmission() { 279891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle WorkQueue[] ws; WorkQueue w; ForkJoinTask<?> t; 279991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if ((ws = workQueues) != null) { 280091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle for (int i = 0; i < ws.length; i += 2) { 280191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if ((w = ws[i]) != null && (t = w.poll()) != null) 280291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle return t; 2803a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2804a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2805a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson return null; 2806a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2807a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 2808a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 2809a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * Removes all available unexecuted submitted and forked tasks 2810a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * from scheduling queues and adds them to the given collection, 2811a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * without altering their execution status. These may include 2812a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * artificially generated or wrapped tasks. This method is 2813a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * designed to be invoked only when the pool is known to be 2814a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * quiescent. Invocations at other times may not remove all 2815a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * tasks. A failure encountered while attempting to add elements 2816a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * to collection {@code c} may result in elements being in 2817a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * neither, either or both collections when the associated 2818a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * exception is thrown. The behavior of this operation is 2819a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * undefined if the specified collection is modified while the 2820a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * operation is in progress. 2821a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * 2822a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @param c the collection to transfer elements into 2823a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @return the number of elements transferred 2824a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 2825a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson protected int drainTasksTo(Collection<? super ForkJoinTask<?>> c) { 2826a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson int count = 0; 282791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle WorkQueue[] ws; WorkQueue w; ForkJoinTask<?> t; 282891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if ((ws = workQueues) != null) { 282991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle for (int i = 0; i < ws.length; ++i) { 283091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if ((w = ws[i]) != null) { 283191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle while ((t = w.poll()) != null) { 283291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle c.add(t); 283391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle ++count; 283491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 283591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 2836a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2837a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2838a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson return count; 2839a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2840a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 2841a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 2842a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * Returns a string identifying this pool, as well as its state, 2843a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * including indications of run state, parallelism level, and 2844a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * worker and task counts. 2845a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * 2846a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @return a string identifying this pool, as well as its state 2847a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 2848a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson public String toString() { 284991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle // Use a single pass through workQueues to collect counts 285091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle long qt = 0L, qs = 0L; int rc = 0; 285191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle long st = stealCount; 2852a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson long c = ctl; 285391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle WorkQueue[] ws; WorkQueue w; 285491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if ((ws = workQueues) != null) { 285591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle for (int i = 0; i < ws.length; ++i) { 285691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if ((w = ws[i]) != null) { 285791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle int size = w.queueSize(); 285891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if ((i & 1) == 0) 285991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle qs += size; 286091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle else { 286191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle qt += size; 286291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle st += w.nsteals; 286391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (w.isApparentlyUnblocked()) 286491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle ++rc; 286591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 286691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 286791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 286891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 286975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle int pc = parallelism; 2870a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson int tc = pc + (short)(c >>> TC_SHIFT); 287191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle int ac = pc + (int)(c >> AC_SHIFT); 287291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (ac < 0) // ignore transient negative 287391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle ac = 0; 2874a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson String level; 2875a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson if ((c & STOP_BIT) != 0) 2876a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson level = (tc == 0) ? "Terminated" : "Terminating"; 2877a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson else 287891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle level = plock < 0 ? "Shutting down" : "Running"; 2879a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson return super.toString() + 2880a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson "[" + level + 2881a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson ", parallelism = " + pc + 2882a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson ", size = " + tc + 2883a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson ", active = " + ac + 2884a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson ", running = " + rc + 2885a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson ", steals = " + st + 2886a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson ", tasks = " + qt + 2887a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson ", submissions = " + qs + 2888a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson "]"; 2889a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2890a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 2891a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 289291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Possibly initiates an orderly shutdown in which previously 289391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * submitted tasks are executed, but no new tasks will be 289491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * accepted. Invocation has no effect on execution state if this 2895f73b1089c7236a7268ccce10546cc340a36f8c4bEino-Ville Talvala * is the {@code commonPool()}, and no additional effect if 289691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * already shut down. Tasks that are in the process of being 289791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * submitted concurrently during the course of this method may or 289891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * may not be rejected. 2899a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 2900a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson public void shutdown() { 2901a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson checkPermission(); 290291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle tryTerminate(false, true); 2903a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2904a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 2905a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 290691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Possibly attempts to cancel and/or stop all tasks, and reject 290791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * all subsequently submitted tasks. Invocation has no effect on 2908f73b1089c7236a7268ccce10546cc340a36f8c4bEino-Ville Talvala * execution state if this is the {@code commonPool()}, and no 290991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * additional effect if already shut down. Otherwise, tasks that 291091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * are in the process of being submitted or executed concurrently 291191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * during the course of this method may or may not be 291291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * rejected. This method cancels both existing and unexecuted 291391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * tasks, in order to permit termination in the presence of task 291491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * dependencies. So the method always returns an empty list 291591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * (unlike the case for some other Executors). 2916a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * 2917a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @return an empty list 2918a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 2919a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson public List<Runnable> shutdownNow() { 2920a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson checkPermission(); 292191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle tryTerminate(true, true); 2922a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson return Collections.emptyList(); 2923a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2924a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 2925a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 2926a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * Returns {@code true} if all tasks have completed following shut down. 2927a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * 2928a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @return {@code true} if all tasks have completed following shut down 2929a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 2930a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson public boolean isTerminated() { 2931a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson long c = ctl; 2932a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson return ((c & STOP_BIT) != 0L && 293375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle (short)(c >>> TC_SHIFT) + parallelism <= 0); 2934a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2935a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 2936a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 2937a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * Returns {@code true} if the process of termination has 2938a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * commenced but not yet completed. This method may be useful for 2939a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * debugging. A return of {@code true} reported a sufficient 2940a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * period after shutdown may indicate that submitted tasks have 294191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * ignored or suppressed interruption, or are waiting for I/O, 2942a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * causing this executor not to properly terminate. (See the 2943a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * advisory notes for class {@link ForkJoinTask} stating that 2944a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * tasks should not normally entail blocking operations. But if 2945a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * they do, they must abort them on interrupt.) 2946a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * 2947a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @return {@code true} if terminating but not yet terminated 2948a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 2949a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson public boolean isTerminating() { 2950a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson long c = ctl; 2951a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson return ((c & STOP_BIT) != 0L && 295275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle (short)(c >>> TC_SHIFT) + parallelism > 0); 2953a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2954a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 2955a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 2956a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * Returns {@code true} if this pool has been shut down. 2957a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * 2958a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @return {@code true} if this pool has been shut down 2959a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 2960a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson public boolean isShutdown() { 296191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle return plock < 0; 2962a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 2963a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 2964a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 296591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * Blocks until all tasks have completed execution after a 296691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle * shutdown request, or the timeout occurs, or the current thread 2967f73b1089c7236a7268ccce10546cc340a36f8c4bEino-Ville Talvala * is interrupted, whichever happens first. Because the {@code 2968f73b1089c7236a7268ccce10546cc340a36f8c4bEino-Ville Talvala * commonPool()} never terminates until program shutdown, when 296975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * applied to the common pool, this method is equivalent to {@link 297075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * #awaitQuiescence(long, TimeUnit)} but always returns {@code false}. 2971a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * 2972a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @param timeout the maximum time to wait 2973a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @param unit the time unit of the timeout argument 2974a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @return {@code true} if this executor terminated and 2975a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * {@code false} if the timeout elapsed before termination 2976a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @throws InterruptedException if interrupted while waiting 2977a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 2978a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson public boolean awaitTermination(long timeout, TimeUnit unit) 2979a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson throws InterruptedException { 298075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (Thread.interrupted()) 298175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle throw new InterruptedException(); 298275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (this == common) { 298375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle awaitQuiescence(timeout, unit); 298475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle return false; 298575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle } 2986a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson long nanos = unit.toNanos(timeout); 298791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (isTerminated()) 298891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle return true; 298991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (nanos <= 0L) 299091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle return false; 299191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle long deadline = System.nanoTime() + nanos; 299291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle synchronized (this) { 2993a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson for (;;) { 2994a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson if (isTerminated()) 2995a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson return true; 299691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (nanos <= 0L) 2997a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson return false; 299891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle long millis = TimeUnit.NANOSECONDS.toMillis(nanos); 299991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle wait(millis > 0L ? millis : 1L); 300091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle nanos = deadline - System.nanoTime(); 3001a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 3002a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 3003a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 3004a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 3005a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 300675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * If called by a ForkJoinTask operating in this pool, equivalent 300775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * in effect to {@link ForkJoinTask#helpQuiesce}. Otherwise, 300875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * waits and/or attempts to assist performing tasks until this 300975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * pool {@link #isQuiescent} or the indicated timeout elapses. 301075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * 301175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * @param timeout the maximum time to wait 301275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * @param unit the time unit of the timeout argument 301375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * @return {@code true} if quiescent; {@code false} if the 301475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * timeout elapsed. 301575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle */ 301675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle public boolean awaitQuiescence(long timeout, TimeUnit unit) { 301775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle long nanos = unit.toNanos(timeout); 301875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle ForkJoinWorkerThread wt; 301975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle Thread thread = Thread.currentThread(); 302075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if ((thread instanceof ForkJoinWorkerThread) && 302175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle (wt = (ForkJoinWorkerThread)thread).pool == this) { 302275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle helpQuiescePool(wt.workQueue); 302375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle return true; 302475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle } 302575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle long startTime = System.nanoTime(); 302675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle WorkQueue[] ws; 302775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle int r = 0, m; 302875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle boolean found = true; 302975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle while (!isQuiescent() && (ws = workQueues) != null && 303075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle (m = ws.length - 1) >= 0) { 303175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (!found) { 303275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if ((System.nanoTime() - startTime) > nanos) 303375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle return false; 303475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle Thread.yield(); // cannot block 303575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle } 303675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle found = false; 303775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle for (int j = (m + 1) << 2; j >= 0; --j) { 303875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle ForkJoinTask<?> t; WorkQueue q; int b; 303975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if ((q = ws[r++ & m]) != null && (b = q.base) - q.top < 0) { 304075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle found = true; 304175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if ((t = q.pollAt(b)) != null) 304275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle t.doExec(); 304375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle break; 304475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle } 304575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle } 304675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle } 304775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle return true; 304875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle } 304975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle 305075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle /** 305175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * Waits and/or attempts to assist performing tasks indefinitely 3052f73b1089c7236a7268ccce10546cc340a36f8c4bEino-Ville Talvala * until the {@code commonPool()} {@link #isQuiescent}. 305375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle */ 305475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle static void quiesceCommonPool() { 305575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle common.awaitQuiescence(Long.MAX_VALUE, TimeUnit.NANOSECONDS); 305675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle } 305775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle 305875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle /** 3059a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * Interface for extending managed parallelism for tasks running 3060a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * in {@link ForkJoinPool}s. 3061a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * 3062a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * <p>A {@code ManagedBlocker} provides two methods. Method 3063a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * {@code isReleasable} must return {@code true} if blocking is 3064a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * not necessary. Method {@code block} blocks the current thread 3065a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * if necessary (perhaps internally invoking {@code isReleasable} 3066a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * before actually blocking). These actions are performed by any 306775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * thread invoking {@link ForkJoinPool#managedBlock(ManagedBlocker)}. 306875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * The unusual methods in this API accommodate synchronizers that 306975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * may, but don't usually, block for long periods. Similarly, they 3070a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * allow more efficient internal handling of cases in which 3071a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * additional workers may be, but usually are not, needed to 3072a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * ensure sufficient parallelism. Toward this end, 3073a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * implementations of method {@code isReleasable} must be amenable 3074a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * to repeated invocation. 3075a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * 3076a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * <p>For example, here is a ManagedBlocker based on a 3077a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * ReentrantLock: 3078a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * <pre> {@code 3079a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * class ManagedLocker implements ManagedBlocker { 3080a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * final ReentrantLock lock; 3081a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * boolean hasLock = false; 3082a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * ManagedLocker(ReentrantLock lock) { this.lock = lock; } 3083a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * public boolean block() { 3084a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * if (!hasLock) 3085a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * lock.lock(); 3086a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * return true; 3087a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * } 3088a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * public boolean isReleasable() { 3089a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * return hasLock || (hasLock = lock.tryLock()); 3090a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * } 3091a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * }}</pre> 3092a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * 3093a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * <p>Here is a class that possibly blocks waiting for an 3094a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * item on a given queue: 3095a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * <pre> {@code 3096a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * class QueueTaker<E> implements ManagedBlocker { 3097a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * final BlockingQueue<E> queue; 3098a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * volatile E item = null; 3099a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * QueueTaker(BlockingQueue<E> q) { this.queue = q; } 3100a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * public boolean block() throws InterruptedException { 3101a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * if (item == null) 3102a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * item = queue.take(); 3103a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * return true; 3104a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * } 3105a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * public boolean isReleasable() { 3106a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * return item != null || (item = queue.poll()) != null; 3107a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * } 3108a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * public E getItem() { // call after pool.managedBlock completes 3109a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * return item; 3110a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * } 3111a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * }}</pre> 3112a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 3113a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson public static interface ManagedBlocker { 3114a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 3115a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * Possibly blocks the current thread, for example waiting for 3116a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * a lock or condition. 3117a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * 3118a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @return {@code true} if no additional blocking is necessary 3119a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * (i.e., if isReleasable would return true) 3120a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @throws InterruptedException if interrupted while waiting 3121a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * (the method is not required to do so, but is allowed to) 3122a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 3123a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson boolean block() throws InterruptedException; 3124a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 3125a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 3126a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * Returns {@code true} if blocking is unnecessary. 312775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * @return {@code true} if blocking is unnecessary 3128a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 3129a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson boolean isReleasable(); 3130a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 3131a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 3132a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson /** 3133a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * Blocks in accord with the given blocker. If the current thread 3134a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * is a {@link ForkJoinWorkerThread}, this method possibly 3135a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * arranges for a spare thread to be activated if necessary to 3136a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * ensure sufficient parallelism while the current thread is blocked. 3137a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * 3138a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * <p>If the caller is not a {@link ForkJoinTask}, this method is 3139a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * behaviorally equivalent to 3140a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * <pre> {@code 3141a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * while (!blocker.isReleasable()) 3142a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * if (blocker.block()) 3143a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * return; 3144a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * }</pre> 3145a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * 3146a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * If the caller is a {@code ForkJoinTask}, then the pool may 3147a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * first be expanded to ensure parallelism, and later adjusted. 3148a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * 3149a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @param blocker the blocker 3150a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson * @throws InterruptedException if blocker.block did so 3151a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson */ 3152a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson public static void managedBlock(ManagedBlocker blocker) 3153a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson throws InterruptedException { 3154a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson Thread t = Thread.currentThread(); 3155a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson if (t instanceof ForkJoinWorkerThread) { 315691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle ForkJoinPool p = ((ForkJoinWorkerThread)t).pool; 315775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle while (!blocker.isReleasable()) { 315875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (p.tryCompensate(p.ctl)) { 315991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle try { 316091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle do {} while (!blocker.isReleasable() && 316191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle !blocker.block()); 316291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } finally { 316391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle p.incrementActiveCount(); 316491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 316591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle break; 316691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 316791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 3168a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 3169a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson else { 317091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle do {} while (!blocker.isReleasable() && 317191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle !blocker.block()); 3172a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 3173a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 3174a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 3175a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson // AbstractExecutorService overrides. These rely on undocumented 3176a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson // fact that ForkJoinTask.adapt returns ForkJoinTasks that also 3177a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson // implement RunnableFuture. 3178a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 3179a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson protected <T> RunnableFuture<T> newTaskFor(Runnable runnable, T value) { 318091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle return new ForkJoinTask.AdaptedRunnable<T>(runnable, value); 3181a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 3182a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 3183a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson protected <T> RunnableFuture<T> newTaskFor(Callable<T> callable) { 318491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle return new ForkJoinTask.AdaptedCallable<T>(callable); 3185a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 3186a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 3187a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson // Unsafe mechanics 318891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle private static final sun.misc.Unsafe U; 318991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle private static final long CTL; 319091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle private static final long PARKBLOCKER; 319191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle private static final int ABASE; 3192a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson private static final int ASHIFT; 319391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle private static final long STEALCOUNT; 319491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle private static final long PLOCK; 319591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle private static final long INDEXSEED; 319675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle private static final long QBASE; 319791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle private static final long QLOCK; 3198a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 3199a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson static { 320091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle // initialize field offsets for CAS etc 3201a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson try { 320291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle U = sun.misc.Unsafe.getUnsafe(); 3203a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson Class<?> k = ForkJoinPool.class; 320491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle CTL = U.objectFieldOffset 3205a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson (k.getDeclaredField("ctl")); 320691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle STEALCOUNT = U.objectFieldOffset 3207a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson (k.getDeclaredField("stealCount")); 320891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle PLOCK = U.objectFieldOffset 320991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle (k.getDeclaredField("plock")); 321091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle INDEXSEED = U.objectFieldOffset 321191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle (k.getDeclaredField("indexSeed")); 321291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle Class<?> tk = Thread.class; 321391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle PARKBLOCKER = U.objectFieldOffset 321491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle (tk.getDeclaredField("parkBlocker")); 321591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle Class<?> wk = WorkQueue.class; 321675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle QBASE = U.objectFieldOffset 321775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle (wk.getDeclaredField("base")); 321891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle QLOCK = U.objectFieldOffset 321991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle (wk.getDeclaredField("qlock")); 322091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle Class<?> ak = ForkJoinTask[].class; 322191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle ABASE = U.arrayBaseOffset(ak); 322291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle int scale = U.arrayIndexScale(ak); 322391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if ((scale & (scale - 1)) != 0) 322491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle throw new Error("data type scale not a power of two"); 322591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle ASHIFT = 31 - Integer.numberOfLeadingZeros(scale); 3226a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } catch (Exception e) { 3227a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson throw new Error(e); 3228a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 322991770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle 323091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle submitters = new ThreadLocal<Submitter>(); 323175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle defaultForkJoinWorkerThreadFactory = 323291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle new DefaultForkJoinWorkerThreadFactory(); 323391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle modifyThreadPermission = new RuntimePermission("modifyThread"); 323491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle 323575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle common = java.security.AccessController.doPrivileged 323675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle (new java.security.PrivilegedAction<ForkJoinPool>() { 323775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle public ForkJoinPool run() { return makeCommonPool(); }}); 323875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle int par = common.parallelism; // report 1 even if threads disabled 323975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle commonParallelism = par > 0 ? par : 1; 324075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle } 324191770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle 324275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle /** 324375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * Creates and returns the common pool, respecting user settings 324475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle * specified via system properties. 324575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle */ 324675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle private static ForkJoinPool makeCommonPool() { 324775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle int parallelism = -1; 324875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle ForkJoinWorkerThreadFactory factory 324975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle = defaultForkJoinWorkerThreadFactory; 325075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle UncaughtExceptionHandler handler = null; 325175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle try { // ignore exceptions in accessing/parsing properties 325291770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle String pp = System.getProperty 325391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle ("java.util.concurrent.ForkJoinPool.common.parallelism"); 325491770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle String fp = System.getProperty 325591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle ("java.util.concurrent.ForkJoinPool.common.threadFactory"); 325675a06e56a4cc4599946e21422513e4bafa759509Calin Juravle String hp = System.getProperty 325775a06e56a4cc4599946e21422513e4bafa759509Calin Juravle ("java.util.concurrent.ForkJoinPool.common.exceptionHandler"); 325875a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (pp != null) 325975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle parallelism = Integer.parseInt(pp); 326091770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (fp != null) 326175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle factory = ((ForkJoinWorkerThreadFactory)ClassLoader. 326275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle getSystemClassLoader().loadClass(fp).newInstance()); 326391770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle if (hp != null) 326475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle handler = ((UncaughtExceptionHandler)ClassLoader. 326591770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle getSystemClassLoader().loadClass(hp).newInstance()); 326691770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } catch (Exception ignore) { 326791770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle } 326891770798d8b9280d48d30df2ed7f63b3ed9b036fCalin Juravle 326975a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (parallelism < 0 && // default 1 less than #cores 327075a06e56a4cc4599946e21422513e4bafa759509Calin Juravle (parallelism = Runtime.getRuntime().availableProcessors() - 1) < 0) 327175a06e56a4cc4599946e21422513e4bafa759509Calin Juravle parallelism = 0; 327275a06e56a4cc4599946e21422513e4bafa759509Calin Juravle if (parallelism > MAX_CAP) 327375a06e56a4cc4599946e21422513e4bafa759509Calin Juravle parallelism = MAX_CAP; 327475a06e56a4cc4599946e21422513e4bafa759509Calin Juravle return new ForkJoinPool(parallelism, factory, handler, LIFO_QUEUE, 327575a06e56a4cc4599946e21422513e4bafa759509Calin Juravle "ForkJoinPool.commonPool-worker-"); 3276a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson } 3277a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson 3278a807b4d808d2591894daf13aab179b2e9c46a2f5Jesse Wilson} 3279