Java Virtual Threads (Project Loom) — Thread-per-Request at Scale

The concurrency problem virtual threads solve

Traditional Java concurrency has a fundamental tension: OS threads are expensive (~1 MB stack, ~ms to create, OS context-switch overhead), so frameworks limit thread counts with thread pools. But blocking I/O (database queries, HTTP calls, file reads) holds a thread idle while waiting. The result: a server handling 10,000 concurrent requests with a pool of 200 threads spends most of its time context-switching and queuing, not doing work.

The workarounds — reactive programming, CompletableFuture chains, async/await — work but transform simple sequential code into complex callback trees that are hard to read, debug, and reason about.

Virtual threads (finalized in Java 21, JEP 444, Project Loom) solve this at the JVM level: you write blocking, sequential code, and the JVM manages the scheduling so that OS threads are never held idle.

Platform threads vs virtual threads

// Platform thread — OS thread behind the scenes:
Thread platform = new Thread(() -> doWork());

// Virtual thread — JVM-managed, cheap:
Thread virtual = Thread.ofVirtual().start(() -> doWork());

Carrier threads — the runtime model

The JVM keeps a small carrier thread pool — a ForkJoinPool whose size defaults to the number of available CPU cores. Virtual threads are scheduled onto carrier threads:

1. A virtual thread mounts onto a carrier thread and executes.
2. When it hits a blocking operation (I/O, sleep, LockSupport.park), the JVM unmounts the virtual thread: its stack is saved to the heap, and the carrier thread is released to run another virtual thread.
3. When the blocking operation completes, the JVM remounts the virtual thread on any available carrier thread and resumes from where it left off.

Carrier pool:    [C1]  [C2]  [C3]  [C4]    ← 4 OS threads
Virtual threads: [V1]  [V2]  ...  [V50000]  ← 50,000 concurrent tasks

V1 blocks on DB query → unmounted from C1 → C1 immediately runs V2
V1 DB result arrives  → V1 remounts on C3 → continues processing

From V1's perspective nothing unusual happened — it "blocked" on a DB call just like always. The JVM handled the parking transparently.

Thread-per-request at scale

Virtual threads make the thread-per-request model practical for high-concurrency servers. Instead of complex async APIs, each request handler is a plain synchronous method:

// Spring Boot 3.2+ — one line to enable:
// spring.threads.virtual.enabled=true

// Or manually:
try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
    for (HttpRequest req : incomingRequests) {
        executor.submit(() -> handleRequest(req)); // each request gets a virtual thread
    }
}

void handleRequest(HttpRequest req) {
    User user = db.findUser(req.userId());       // blocking JDBC — fine
    List<Order> orders = db.findOrders(user.id()); // another blocking call — fine
    sendResponse(new Response(user, orders));
}

The JDBC calls block the virtual thread, but the carrier thread is freed during the wait. 50,000 concurrent requests use only num_CPUs OS threads for CPU work, plus the I/O thread pool of the underlying networking library.

Creating virtual threads

// Thread.ofVirtual() builder:
Thread vt = Thread.ofVirtual()
                  .name("request-handler-", 0)  // numbered names
                  .start(task);

// Thread.startVirtualThread() — shorthand:
Thread vt2 = Thread.startVirtualThread(task);

// ExecutorService (recommended for servers):
try (ExecutorService exec = Executors.newVirtualThreadPerTaskExecutor()) {
    exec.submit(task1);
    exec.submit(task2);
} // blocks until all tasks finish, then closes

// ThreadFactory for framework integration:
ThreadFactory factory = Thread.ofVirtual().factory();

For Spring Boot 3.2+, Tomcat 10.1+, Jetty 12+: a single property enables virtual threads globally without code changes.

Pinning — the main gotcha

Pinning is when a virtual thread cannot be unmounted from its carrier during a blocking call. Pinned virtual threads hold an OS thread idle — exactly what virtual threads are supposed to prevent.

Pinning occurs in two situations:

1. synchronized blocks containing blocking operations

// BAD — virtual thread is pinned for the entire sleep:
synchronized (lock) {
    fetchFromDatabase(); // blocks — carrier thread held idle!
}

// GOOD — ReentrantLock parks correctly:
reentrantLock.lock();
try {
    fetchFromDatabase(); // virtual thread unmounts; carrier freed
} finally {
    reentrantLock.unlock();
}

2. Native method frames (JNI)

If a virtual thread calls a native method that blocks, the JVM cannot save its frame to the heap, so it is pinned.

Detect pinning:

java -Djdk.tracePinnedThreads=full MyApp
# Logs a stack trace every time a virtual thread becomes pinned

Java 24 is actively working to eliminate synchronized-based pinning in most cases, but until then, replace synchronized blocks that contain blocking I/O with ReentrantLock.

ThreadLocal and virtual threads

ThreadLocal works with virtual threads but has scaling issues: if each of a million virtual threads initializes a heavy ThreadLocal (a DB connection, a large buffer), memory explodes. Virtual threads are not pooled, so values don't leak between tasks — but the per-thread allocation cost remains.

ScopedValue (Java 21, JEP 446) is the virtual-thread-friendly replacement:

// ThreadLocal — one instance per thread, including virtual threads:
static final ThreadLocal<User> CURRENT_USER = new ThreadLocal<>();
CURRENT_USER.set(user);            // must remember to remove() later
// ... elsewhere:
User u = CURRENT_USER.get();
CURRENT_USER.remove();             // easy to forget → memory leak

// ScopedValue — immutable, inheritable, auto-cleaned:
static final ScopedValue<User> CURRENT_USER = ScopedValue.newInstance();

ScopedValue.where(CURRENT_USER, user).run(() -> {
    processRequest();              // CURRENT_USER.get() == user within scope
});                                // cleaned up automatically on scope exit

ScopedValue is immutable within a scope (you can't accidentally overwrite it), inherits into child threads automatically, and requires no cleanup.

Structured concurrency

Structured concurrency (Java 21 preview, JEP 453) ensures subtask lifetimes are bounded by the parent scope:

try (var scope = new StructuredTaskScope.ShutdownOnFailure()) {
    Future<User>        user   = scope.fork(() -> db.findUser(id));
    Future<List<Order>> orders = scope.fork(() -> db.findOrders(id));

    scope.join();            // waits for both forks
    scope.throwIfFailed();   // throws if either fork threw

    return new Response(user.resultNow(), orders.resultNow());
}
// At this point: both tasks are guaranteed done or cancelled

ShutdownOnFailure cancels remaining subtasks when one fails. ShutdownOnSuccess returns as soon as any succeeds — useful for parallel hedging (run the same query on two replicas, take the first response).

Without structured concurrency, a thrown exception in one fork would let the other fork run forever — a thread leak. The scope prevents this.

When NOT to use virtual threads

Virtual threads are not universally better than platform threads:

CPU-bound work — a CPU-intensive computation occupies a carrier thread for its entire run. Hundreds of CPU-bound virtual threads compete for the same small carrier pool and starve each other. Use ForkJoinPool.commonPool() or newFixedThreadPool(nCPUs).

Code relying on thread identity — some libraries use Thread.currentThread() as a map key or assume ThreadLocal values persist across tasks in a pool. Virtual threads aren't pooled, so these assumptions break.

Long synchronized/native sections — pinning turns a virtual thread into an expensive platform thread for the duration.

// CPU-bound — use platform threads:
var pool = ForkJoinPool.commonPool();
pool.submit(() -> encodeVideo(source, dest)); // needs all CPU time it can get

// I/O-bound — use virtual threads:
var exec = Executors.newVirtualThreadPerTaskExecutor();
exec.submit(() -> httpClient.get(url));       // spends most time waiting

Virtual threads vs reactive programming

Reactive frameworks (Project Reactor, RxJava) solved the same I/O-bound scaling problem by making every operation non-blocking and composing chains of transformations:

// Reactor — non-blocking, complex composition:
Mono.fromCallable(() -> db.findUser(id))
    .subscribeOn(Schedulers.boundedElastic())
    .flatMap(user -> db.findOrders(user.id()))
    .map(orders -> new Response(user, orders))
    .subscribe(response -> send(response), err -> sendError(err));

// Virtual threads — blocking, sequential, simple:
User user = db.findUser(id);           // blocks virtual thread
List<Order> orders = db.findOrders(user.id());
return new Response(user, orders);

For most I/O-bound services, virtual threads eliminate the need for reactive. Reactive still makes sense for:

• Backpressure — controlling producer/consumer throughput.
• Streaming — processing a data stream element by element.
• Existing reactive ecosystems (Vert.x, Quarkus reactive mode).

Observability

# Thread dump — includes virtual threads:
jcmd <pid> Thread.dump_to_file -format=json /tmp/threads.json

# JFR recording — virtual thread lifecycle, pinning events:
java -XX:StartFlightRecording=filename=app.jfr,settings=profile MyApp

# Detect pinning:
java -Djdk.tracePinnedThreads=full MyApp

# Native memory tracking (carrier pool):
java -XX:NativeMemoryTracking=summary MyApp

JFR emits jdk.VirtualThreadPinned events when pinning occurs. Thread dumps group virtual threads with identical stack traces — 50,000 threads all blocked on JDBC appear as one entry with a count, not 50,000 individual stacks.

Migration checklist

1. Replace the executor: newFixedThreadPool(N) → newVirtualThreadPerTaskExecutor().
2. Remove artificial pool-size tuning — thread count limits are irrelevant for I/O-bound tasks.
3. Fix pinning: audit synchronized blocks that contain blocking I/O; replace with ReentrantLock.
4. Replace heavy ThreadLocals with ScopedValue for context propagation (user, trace-id).
5. Load-test: run under realistic load and check JFR for VirtualThreadPinned events.
6. Framework support: update Spring Boot to 3.2+, Tomcat to 10.1+, Hibernate to 6.2+.

Recap

Virtual threads (Java 21) are cheap, JVM-managed threads that mount onto a small carrier thread pool and unmount during blocking I/O, freeing the carrier for other work. This enables a thread-per-request model at millions-of-threads scale using plain blocking code. Pinning — caused by synchronized blocks holding monitors during blocking calls or by JNI frames — defeats unmounting; replace with ReentrantLock. ScopedValue replaces ThreadLocal for immutable context propagation. Structured concurrency bounds subtask lifetimes to the parent scope, preventing leaks and enabling clean cancellation. Virtual threads excel at I/O-bound work; keep platform threads for CPU-bound computation. For most new services they eliminate the need for reactive programming, but reactive remains valuable for backpressure and streaming use cases.