Java Executors, Thread Pools & CompletableFuture

Why executors exist

Raw threads are seductive — new Thread(task).start() looks like the whole story — but they don't scale. Each OS thread costs a stack (often 512KB–1MB), a kernel scheduling entry, and bookkeeping, so spawning one per task means unbounded creation under load that exhausts memory and thrashes the scheduler. The Executor framework (Java 5) fixes this by decoupling task submission from task execution: you hand work to a pool, and a bounded set of reusable workers runs it. The two wins are reuse (pay thread cost once) and bounding (a hard cap on concurrency so a traffic spike can't take down the JVM). The rest of this guide is about choosing and tuning that pool correctly.

The Executor interface hierarchy

The framework is a layered stack of interfaces, each adding capability on top of the last:

Executor — the minimal contract, a single execute(Runnable). It just runs a task; whether that's now, pooled, or async is the implementation's choice.
ExecutorService — adds lifecycle (shutdown, awaitTermination) and result-bearing submission (submit, invokeAll, invokeAny) that hand back Futures.
ScheduledExecutorService — adds running tasks after a delay or periodically.

Executor e = Runnable::run;                          // simplest possible Executor
ExecutorService es = Executors.newFixedThreadPool(4); // lifecycle + Futures
ScheduledExecutorService ses = Executors.newScheduledThreadPool(2);

You almost always program against ExecutorService: it gives you both task results and a clean way to shut the pool down. Coding to the interface (not ThreadPoolExecutor directly in call sites) keeps you free to swap implementations later.

Factory methods and why you should distrust them

The Executors class is a factory of preconfigured pools — newFixedThreadPool(n), newCachedThreadPool(), newSingleThreadExecutor(), newScheduledThreadPool(n), newWorkStealingPool(). They're convenient, and they're also where production incidents come from, because each one is just a ThreadPoolExecutor with hidden defaults that can trigger OutOfMemoryError:

newFixedThreadPool and newSingleThreadExecutor back their queue with an unbounded LinkedBlockingQueue — if tasks arrive faster than they finish, the queue grows until the heap is gone.
newCachedThreadPool has no upper bound on threads — a burst can spawn thousands and exhaust native memory.

// Effective Java's guidance: construct ThreadPoolExecutor directly so every
// knob — queue bound, max threads, rejection behavior — is a deliberate choice.
ExecutorService pool = new ThreadPoolExecutor(
    8, 16, 60L, TimeUnit.SECONDS,
    new ArrayBlockingQueue<>(1000),            // bounded queue
    new ThreadPoolExecutor.CallerRunsPolicy()  // backpressure on overflow
);

The factories are fine for scripts and tests. For anything that takes real traffic, build the executor explicitly so the failure mode is rejection, not crash.

ThreadPoolExecutor: the six knobs and the task flow

ThreadPoolExecutor is defined entirely by six parameters: corePoolSize (threads kept alive when idle), maximumPoolSize (hard cap), keepAliveTime (how long non-core idle threads survive), the workQueue, a threadFactory, and a RejectedExecutionHandler. What trips people up is the order in which capacity is used on execute:

// core=2, queue=2, max=4 — capacity surfaces in THIS order:
new ThreadPoolExecutor(2, 4, 60, TimeUnit.SECONDS, new ArrayBlockingQueue<>(2));
// tasks 1-2 -> start 2 core threads (even if a thread is idle, prefer a new core thread)
// tasks 3-4 -> wait in the queue (capacity 2)
// tasks 5-6 -> spawn 2 extra threads up to max=4 (only once the queue is full)
// task  7   -> rejected

The non-obvious consequence: with an unbounded queue, step 3 never fires, so maximumPoolSize is ignored and the pool never grows past core — which is exactly why newFixedThreadPool only ever runs n threads. Always pass a named ThreadFactory in production too; default names like pool-1-thread-3 make thread dumps useless.

Rejection policies: what happens when the pool is full

A task is rejected when the queue is full and the pool is at maximumPoolSize (or after shutdown). The handler decides the fate of that task, and the choice is a real design decision about how your system degrades:

AbortPolicy (default) — throws RejectedExecutionException. Loud failure.
CallerRunsPolicy — runs the task on the submitting thread. Natural backpressure.
DiscardPolicy — silently drops the task.
DiscardOldestPolicy — drops the oldest queued task, retries the new one.

var pool = new ThreadPoolExecutor(2, 4, 60, TimeUnit.SECONDS,
    new ArrayBlockingQueue<>(10),
    new ThreadPoolExecutor.CallerRunsPolicy()); // submitter slows down under load

CallerRunsPolicy is the favorite for throughput-with-stability: when the pool is overwhelmed, the producer is slowed down (it does the work itself) rather than throwing or losing data. You can also implement your own handler to log, meter, or persist rejected work — useful when dropping a task silently would be a bug.

Sizing the pool: CPU-bound vs IO-bound

The right size depends entirely on what threads spend their time doing. CPU-bound work keeps the core busy, so more threads than cores only adds context-switching overhead — size around the number of cores (cores + 1 to cover occasional page faults). IO-bound work blocks on network or disk, leaving the CPU idle, so you want many more threads than cores to keep the CPU saturated while others wait.

int cores = Runtime.getRuntime().availableProcessors();
ExecutorService cpuPool = Executors.newFixedThreadPool(cores);     // CPU-bound
ExecutorService ioPool  = Executors.newFixedThreadPool(cores * 8); // IO-bound (illustrative)
// Brian Goetz formula: threads = cores * (1 + waitTime / computeTime)

Treat the formula as a starting point, not gospel — measure throughput and latency under realistic load and tune from there. And keep CPU and IO work in separate pools: mixing them means a flood of slow IO tasks can starve your CPU work of threads.

submit vs execute, Runnable vs Callable, and Future

execute(Runnable) comes from Executor, returns void, and is fire-and-forget. submit(...) comes from ExecutorService, accepts a Runnable or Callable, and hands back a Future. The difference between the two task types is the shape of the work: Runnable has void run() — no return value, no checked exceptions — while Callable<V> has V call() throws Exception, so it returns a result and may throw.

pool.execute(() -> log("fire and forget"));   // void, no handle, no result

Callable<Integer> c = () -> compute();          // returns a value, may throw checked
Future<Integer> f = pool.submit(c);
Integer result = f.get(2, TimeUnit.SECONDS);    // blocks (with timeout) for the result

A Future<V> is a handle to a result that may not exist yet: get() blocks until done, isDone() polls, cancel(true) attempts interruption. Watch the exception trap — with execute, an uncaught exception hits the thread's UncaughtExceptionHandler (you see it); with submit, the exception is captured in the Future and only re-thrown (wrapped in ExecutionException, unwrap via getCause()) when you call get(). A submitted task you never inspect can fail silently.

CompletableFuture: composing async work

A plain Future can only be polled or blocked on — there's no way to attach a continuation or combine results without tying up a thread. CompletableFuture (Java 8) implements CompletionStage and turns callback spaghetti into a fluent, non-blocking pipeline. The three composition shapes you must know: thenApply transforms a result with a plain function (T -> U); thenCompose chains a function that itself returns a future (T -> CompletableFuture<U>) and flattens it — the async "flatMap"; and thenCombine waits for two independent futures and merges them.

CompletableFuture
    .supplyAsync(() -> fetchUser(id), ioPool) // run async on YOUR pool, not the common one
    .thenApply(User::name)                    // sync transform of the result
    .thenCompose(name -> fetchOrdersAsync(name)) // chain a dependent async call (flatten)
    .exceptionally(ex -> List.of());          // fallback value on any failure upstream

For independent work, thenCombine(other, (a, b) -> ...) joins two parallel results, and allOf(cf...) acts as a barrier: it returns CompletableFuture<Void>, so you join() it and then read each future's result individually. Errors propagate down the chain wrapped in CompletionException — exceptionally recovers with a fallback, while handle((res, ex) -> ...) sees both outcomes. One gotcha: supplyAsync/runAsync use the common ForkJoinPool by default, which is shared JVM-wide and sized to your cores — pass an explicit Executor for any blocking work so you don't starve it.

Graceful shutdown

An ExecutorService keeps its (often non-daemon) threads alive until you stop it, so you must shut it down or the JVM may never exit. There are two stop methods and one wait method: shutdown() is graceful — stops accepting new tasks but lets submitted ones finish; shutdownNow() is aggressive — interrupts running tasks, drains the queue, and returns the never-started tasks; awaitTermination(timeout, unit) blocks until the pool terminates or the timeout elapses. Neither shutdown method blocks on its own — the canonical pattern combines all three:

void shutdownAndAwait(ExecutorService pool) {
    pool.shutdown();                                    // stop taking new tasks
    try {
        if (!pool.awaitTermination(60, TimeUnit.SECONDS)) {
            pool.shutdownNow();                         // force in-flight tasks
            if (!pool.awaitTermination(60, TimeUnit.SECONDS))
                log("pool did not terminate");
        }
    } catch (InterruptedException ie) {
        pool.shutdownNow();
        Thread.currentThread().interrupt();             // restore the interrupt flag
    }
}

The habits that matter: two await phases, escalate from shutdown to shutdownNow, and never swallow InterruptedException. Remember shutdownNow only requests interruption — tasks that ignore the flag keep running. Wire this into a JVM shutdown hook or your framework's lifecycle.

Virtual threads: when sizing stops mattering

Java 21's virtual threads (JEP 444) are lightweight threads scheduled by the JVM rather than mapped 1:1 to OS threads. They cost a few hundred bytes, so you can have millions, and a blocking call unmounts the virtual thread from its carrier OS thread instead of blocking it.

// one virtual thread per task — no pooling, no sizing, no queue tuning
try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
    for (var task : tasks) executor.submit(task);
} // try-with-resources auto-closes (shutdown + awaitTermination)

This upends the classic advice for IO-bound work: you no longer pool threads or agonize over sizing — create one virtual thread per task. The caveats: don't pool virtual threads, avoid pinning (long synchronized blocks or native calls keep the carrier blocked), and keep using bounded platform-thread pools for CPU-bound work, where limiting parallelism to the core count is still exactly right.

Recap

The Executor framework exists to decouple submission from execution and give you reuse plus bounding. Program against ExecutorService, but build pools with an explicit ThreadPoolExecutor so the queue bound, max threads, and rejection policy are deliberate — the factories' unbounded defaults are an OOM waiting to happen. Understand the core → queue → max → reject flow (and why an unbounded queue neuters maximumPoolSize), size pools by CPU-bound vs IO-bound, and prefer submit + Callable/Future when you need results and visible failures. Compose async work with CompletableFuture (thenApply/thenCompose/thenCombine/allOf/exceptionally) on your own executor, always shut pools down with the two-phase graceful pattern, and on Java 21+ reach for virtual threads to make IO-bound sizing a non-problem.

Java Executors, Thread Pools & CompletableFuture — A Complete Guide