Java JVM Memory Explained — Heap, Stack, and Everything In Between

Why JVM memory matters in interviews

Questions about JVM memory come up in almost every Java backend interview. The reason is simple: memory bugs — leaks, OOM errors, GC pauses — are some of the hardest problems to diagnose in production, and interviewers want to see that you have a mental model of where data lives and when it gets cleaned up. This article builds that model from the ground up.

The two main regions: stack and heap

The JVM divides runtime memory into two primary areas visible to application code: the stack and the heap.

The stack is a per-thread, LIFO structure. Every method call pushes a stack frame containing the method's local variables, parameters, and a work area (operand stack) for executing bytecode. When the method returns, the frame is popped and everything in it disappears immediately — no GC needed.

The heap is a single region shared by all threads. Every object created with new lands here, including arrays. The heap is managed by the garbage collector.

void demo() {
    int x = 42;                  // x lives on the stack (primitive local)
    String s = new String("hi"); // the String object lives on the heap;
                                 // s (the reference variable) lives on the stack
}
// When demo() returns, x and s are gone; the String may be GC'd

The key insight: the reference variable and the object it points to are in different places. The reference (an address-sized slot) lives wherever the variable was declared — on the stack if it's a local variable, or inside another heap object if it's a field. The object always lives on the heap.

Stack frames in detail

Each stack frame holds three things:

1. Local variable array — the method's parameters and declared local variables. Primitives are stored by value; object references are stored as pointers.
2. Operand stack — a push/pop work area the JVM uses to execute individual bytecodes (e.g., it pushes two ints, executes iadd, and the result is on top).
3. Frame data — a reference to the current class's runtime constant pool and the return address so the JVM knows where to jump when this method returns.

Because each thread has its own stack, local variables are inherently thread-safe — no other thread can see or modify them. Synchronization is only needed for shared heap data.

StackOverflowError

The stack has a fixed size, set by -Xss (default ~512 KB–1 MB depending on the JVM and OS). Deep recursion without a base case exhausts this space and triggers a StackOverflowError.

int factorial(int n) {
    return n * factorial(n - 1); // no base case → infinite recursion
}
// java.lang.StackOverflowError

The fix is almost always a missing or incorrect base case, not a larger -Xss. Reserve -Xss tuning for genuinely deep but bounded recursion (e.g., recursive descent parsers on large documents).

The generational heap

HotSpot organises the heap into generations based on the weak generational hypothesis: most objects die young. Collecting only the short-lived objects most of the time keeps GC pauses short.

┌─────────────────────────────────────────────────────┐
│                     Java Heap                       │
│  ┌──────────────────────────┐  ┌──────────────────┐ │
│  │     Young Generation     │  │   Old Generation  │ │
│  │  ┌────────┬──┬──────┐   │  │   (Tenured)       │ │
│  │  │ Eden   │S0│  S1  │   │  │                   │ │
│  │  └────────┴──┴──────┘   │  │                   │ │
│  └──────────────────────────┘  └──────────────────┘ │
└─────────────────────────────────────────────────────┘

• Eden — new objects are allocated here. It fills quickly.
• Survivor 0 / S1 — objects that survived at least one Minor GC are copied here alternately, tracking their age in bits of the mark word.
• Old Gen (Tenured) — objects that reach a configurable age threshold (-XX:MaxTenuringThreshold, default 15) are promoted here.

A Minor GC collects Young Gen only — fast, frequent, short pause. A Full GC collects the entire heap and triggers when Old Gen fills up; it is slow and its pause can affect latency SLAs.

Metaspace (replacing PermGen)

Java 8 removed PermGen and replaced it with Metaspace, which lives in native memory outside the Java heap. Metaspace stores class metadata — bytecode, method descriptors, constant pools — for every loaded class.

Because Metaspace is in native memory it grows dynamically. Without a cap it can exhaust the OS's virtual address space, so set -XX:MaxMetaspaceSize on servers where the class count is bounded.

java -XX:MaxMetaspaceSize=256m MyServer

The old OutOfMemoryError: PermGen space is gone; OutOfMemoryError: Metaspace is its equivalent. It usually means class loaders are leaking (common with hot-redeploy in application servers).

Object headers and memory overhead

Every heap object carries a hidden object header before its fields:

An Object with no fields still occupies 16 bytes. Knowing this helps when evaluating whether using millions of small objects is viable versus using primitive arrays.

Compressed oops (-XX:+UseCompressedOops, on by default when heap ≤ 32 GB) store references as 32-bit word offsets rather than 64-bit raw pointers, shrinking every reference field from 8 → 4 bytes. Crossing the 32 GB heap threshold disables compressed oops and every reference grows back to 8 bytes — heap usage often increases, not decreases.

Escape analysis and stack allocation

The JIT compiler performs escape analysis to determine whether an object reference escapes the method or thread that created it. If it doesn't escape, the JVM can:

• Allocate it on the stack — the object is freed when the frame pops, with zero GC involvement.
• Scalar-replace it — decompose it into individual primitive fields, eliminating the object entirely.

void compute() {
    Point p = new Point(1, 2); // p never escapes this method
    int sum = p.x + p.y;       // JIT may replace p with two int locals — no heap allocation
}

Escape analysis is enabled by default (-XX:+DoEscapeAnalysis). You can't force it, but keeping temporary objects local (not returning them, not storing them in fields or statics) gives the JIT the best chance to eliminate the allocation.

TLABs — why allocation is cheap

Allocating on the shared Eden naively would require a thread-safe pointer bump on every new. The JVM avoids this by giving each thread a Thread-Local Allocation Buffer (TLAB) — a private slice of Eden. Allocation within a TLAB is just a pointer bump with no synchronization.

When a TLAB fills, the thread requests a new one from Eden under a brief lock. Objects larger than a TLAB are allocated directly in Eden or promoted straight to Old Gen (humongous objects in G1).

The result: new SomeObject() is nearly as cheap as malloc in C — the cost is paid at GC time, not at allocation time.

GC roots and memory leaks

The garbage collector starts from GC roots — objects guaranteed to be live — and traces all reachable objects. Anything reachable from a root is kept alive; everything else is eligible for collection.

GC roots include:

• Local variables in active stack frames
• Static fields of loaded classes
• JNI references in native code
• Active thread objects themselves

A Java memory leak is not memory the GC can't see; it's memory the GC won't collect because a root still holds a reference to it (directly or through a chain).

static List<Object> cache = new ArrayList<>(); // static = GC root
// anything added here lives for the life of the class loader

Static collections that grow without bound are the most common source of leaks in long-running services.

Diagnosing OutOfMemoryError

When the JVM throws OutOfMemoryError: Java heap space, the heap is full even after GC. The fastest path to diagnosis:

# Enable automatic heap dump when OOM occurs:
java -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp/dump.hprof MyApp

# Manual dump of a running process:
jmap -dump:live,format=b,file=/tmp/dump.hprof <pid>

Open the dump in Eclipse MAT and run "Leak Suspects". The dominator tree shows the objects retaining the most memory; the shortest path to GC root shows why they are retained. That path almost always points directly at the bug — an unbounded static collection, a listener that was never removed, or a cache with no eviction policy.

Soft, weak, and phantom references

Java's reference types let you hold objects without preventing GC:

SoftReference<byte[]> cache = new SoftReference<>(new byte[1024 * 1024]);
byte[] data = cache.get(); // null if GC already cleared it
if (data == null) data = reload();

WeakHashMap is often misunderstood: keys are weakly referenced, so an entry disappears as soon as no strong reference to the key exists elsewhere — useful for metadata caches keyed by objects you don't own.

Native (off-heap) memory

Not all JVM memory lives on the heap. Native memory is allocated directly from the OS and is not subject to GC:

If a process's resident set size (RSS) grows well beyond -Xmx, the culprit is native memory. Use Native Memory Tracking (-XX:NativeMemoryTracking=summary) and jcmd <pid> VM.native_memory summary to break it down.

Key flags at a glance

Recap

The stack is a per-thread, frame-based region for method-local data; the heap is the shared region managed by the GC. The heap is generational (Eden → Survivor → Old Gen) to exploit the observation that most objects die young. Metaspace holds class metadata in native memory. Escape analysis and TLABs make allocation cheap. Memory leaks in Java are objects still reachable from a GC root — trace them with a heap dump and Eclipse MAT. When native memory grows suspiciously large, reach for -XX:NativeMemoryTracking.