Skip to content

Concurrency & Parallelism

What you’ll learn. The difference between concurrency and parallelism; the main execution models (threads, event loops/async, processes, actors); why shared mutable state is dangerous and how locks, atomics and memory models address it; how deadlocks arise and are avoided; what structured concurrency is and why it is now the recommended style; and the practical details of Python (the GIL, asyncio, free-threaded builds) and TypeScript/JavaScript (event loop, promises, workers). Code is labelled Python or TypeScript.

  • Concurrency is about structure: dealing with many things at once (overlapping tasks in time). A single-core machine can be concurrent by interleaving.
  • Parallelism is about execution: doing many things at the same instant on multiple cores.

Rob Pike: “Concurrency is about dealing with lots of things at once. Parallelism is about doing lots of things at once.”

Two workload types need different tools:

  • I/O-bound work (network calls, disk, databases) spends most time waiting. The goal is overlapping waits: async I/O or threads work well even on one core.
  • CPU-bound work (compression, ML, image processing) needs real parallelism: multiple processes, native threads in languages without a global lock, or SIMD/GPU.

Amdahl’s law bounds speedup: if a fraction pp of the work is parallelizable, the maximum speedup on nn processors is

S(n)=1(1−p)+p/nS(n) = \frac{1}{(1-p) + p/n}

With p=0.9p = 0.9 and n=16n = 16, S≈6.4S \approx 6.4; as n→∞n \to \infty, S→10S \to 10. Serial portions dominate, so reduce coordination and shared state. Gustafson’s law observes that larger problems often scale their parallel part, making parallelism more useful in practice.

A thread is an independently scheduled flow of control inside a process, sharing the process’s memory. The OS preempts threads at arbitrary points. Pros: simple mental model for blocking code, true parallelism on multiple cores (in most languages), good for blocking I/O. Cons: shared memory means data races; each thread costs memory (typically ~1 MB of stack reserved, though virtual) and context switches; thousands are feasible, hundreds of thousands are not (with OS threads).

A single thread runs a loop that dispatches ready callbacks; I/O is non-blocking and registered with the OS (epoll, kqueue, IOCP, io_uring). async/await is syntax that lets you write sequential-looking code that is suspended at await points and resumed later (cooperative multitasking: tasks yield voluntarily). Pros: tens of thousands of concurrent connections cheaply; no data races between awaits (code between awaits runs atomically with respect to other tasks on the same loop). Cons: any blocking call or long CPU computation stalls everything on the loop; “function coloring” (async functions can only be awaited from async contexts); race conditions are still possible across await points.

Separate address spaces, communicating via pipes, sockets, shared memory or queues. Pros: isolation, true parallelism in any language, fault containment. Cons: heavier startup and memory, serialization cost for data transfer.

An actor is an entity with private state and a mailbox; it processes one message at a time and communicates only by sending asynchronous messages (Erlang/OTP, Akka, Orleans, Elixir). No shared state means no locks inside an actor. Related: CSP (Communicating Sequential Processes; Go’s goroutines and channels): “Do not communicate by sharing memory; share memory by communicating.” Pros: easy reasoning about isolation, natural fault tolerance (“let it crash” with supervisors), scalable to distributed settings. Cons: mailboxes can overflow (backpressure needed), message ordering/protocol design is its own complexity, deadlock is still possible via circular waits on replies.

Runtime-scheduled lightweight threads multiplexed over OS threads: goroutines (Go), Java 21 virtual threads, Erlang processes, Kotlin coroutines. They let you write blocking-style code at async scale, and avoid function coloring.

A data race occurs when two threads access the same memory location concurrently, at least one writes, and there is no synchronization. A race condition is the broader class: the outcome depends on timing. The classic example:

# Python: lost update from a non-atomic increment (works on any threaded language)
import threading
counter = 0
def work():
global counter
for _ in range(1_000_000):
counter += 1 # read, add, write: three steps, interleavable
ts = [threading.Thread(target=work) for _ in range(4)]
[t.start() for t in ts]; [t.join() for t in ts]
print(counter) # frequently < 4,000,000 (behavior varies by version and build)

Interleaving: T1 reads 5, T2 reads 5, T1 writes 6, T2 writes 6: one increment lost. This is the same shape as the lost update in databases (Databases).

Strategies, in rough order of preference:

  1. Don’t share: confine data to one thread/task; pass messages.
  2. Don’t mutate: immutable data structures can be shared freely.
  3. Synchronize shared mutable state: locks, atomics, transactional memory.
  • Mutex (lock): only one holder at a time; protects a critical section. Hold it briefly; never across I/O or callbacks into unknown code.
  • Read-write lock: many readers or one writer.
  • Semaphore: a counter permitting up to nn concurrent holders (limit concurrency, e.g. connection pools).
  • Condition variable: wait until a predicate holds; always re-check in a while loop (spurious wakeups).
  • Barrier/latch: wait for N parties.
  • Atomics (compare-and-swap, fetch-add): lock-free single-variable operations built on CPU instructions.
  • Thread-safe queues (producer-consumer): the workhorse for handing work between threads.
import threading
lock = threading.Lock()
def work():
global counter
for _ in range(1_000_000):
with lock: # context manager guarantees release
counter += 1

(A lock per increment is slow; better to accumulate locally and merge once, or use per-thread counters. Reduce contention by design.)

# Producer-consumer with a bounded queue (backpressure) and sentinel shutdown
import queue, threading
q: "queue.Queue[int | None]" = queue.Queue(maxsize=100) # bounded: producers block when full
def producer():
for i in range(1000):
q.put(i) # blocks if full
q.put(None)
def consumer():
while (item := q.get()) is not None:
process(item)

Lock-free algorithms use CAS loops; they avoid deadlock and priority inversion but are notoriously hard to get right (the ABA problem, memory reclamation). Use well-tested library structures (java.util.concurrent, crossbeam, ConcurrentHashMap) rather than inventing them.

Modern CPUs and compilers reorder and cache memory operations. Without synchronization, one thread may not see another’s writes in order, or at all. A memory model is a language’s contract specifying which values a read may return. Key concept: happens-before. If operation A happens-before B, then A’s effects are visible to B. Synchronization actions (lock release then acquire, atomic store-release then load-acquire, thread start/join, channel send then receive) create happens-before edges.

Thread 1: data = 42; // (1)
ready = true; // (2)
Thread 2: while (!ready) {} // (3)
print(data); // (4) may print 0 without synchronization!

Without a happens-before edge between (2) and (3), the compiler or CPU may reorder (1) and (2), or Thread 2 may cache ready. Fix: make ready an atomic/volatile (Java) / std::atomic with release-acquire ordering (C++, Rust), or use a lock or channel.

The Java Memory Model (JSR-133, 2004) and the C++11 model standardized this. Go’s memory model guarantees that if you synchronize via channels, mutexes or sync primitives, you get happens-before; racy programs have no guarantees. Strongly ordered CPUs (x86) hide many bugs that appear on weakly ordered ones (ARM, including Apple Silicon and most servers moving to ARM): code that “worked on my machine” may break after a hardware migration. Use race detectors (Go’s -race, ThreadSanitizer in C/C++/Rust, Java’s tools) in CI.

Rule: if you use locks or channels consistently, you can mostly ignore memory models; the model matters when you write lock-free code or use relaxed atomics, which you should avoid unless profiled.

Deadlock: threads each hold a resource and wait for another held by someone else, so none proceeds. Four Coffman conditions must all hold: mutual exclusion, hold and wait, no preemption, and circular wait. Break any one.

# Classic lock-ordering deadlock
a, b = threading.Lock(), threading.Lock()
def t1():
with a:
with b: ...
def t2():
with b: # opposite order: can deadlock with t1
with a: ...

Avoidance techniques:

  • Global lock ordering: always acquire locks in a fixed order (e.g., by ID or address).
  • Hold fewer locks, for less time; avoid calling unknown code (callbacks, virtual methods) while holding a lock.
  • Timeouts and try_lock with back-off and retry.
  • Single lock or coarser granularity when performance allows.
  • Prefer message passing so there are no locks to order.
  • Detect with tools (lock-order analyzers, thread dumps; databases detect deadlocks and abort a victim).

Other hazards: livelock (threads keep reacting to each other without progress; fix with randomized backoff), starvation (a thread never gets the resource; use fair locks), priority inversion (a high-priority thread waits on a low-priority one holding a lock; the famous Mars Pathfinder incident of 1997; fix with priority inheritance), and convoying under heavy contention.

Unstructured concurrency (fire-and-forget go func(), new Thread(...), asyncio.create_task without tracking) breaks the property that makes sequential code easy to reason about: control flow enters a block and leaves it when it finishes. Leaked tasks outlive their callers, swallow exceptions, hold resources, and make cancellation unreliable. Nathaniel Smith’s 2018 essay “Notes on structured concurrency, or: Go statement considered harmful” argued that spawning should be confined to a scope that waits for all children before exiting, propagates errors, and cancels siblings on failure.

Properties of structured concurrency:

  1. Lifetimes nest: child tasks cannot outlive the scope that created them.
  2. Errors propagate to the parent; no silently lost exceptions.
  3. Cancellation propagates downward through the task tree.
  4. Resources are released deterministically.

Implementations: Trio’s nurseries (the origin), Python 3.11+ asyncio.TaskGroup, Kotlin coroutine scopes, Swift structured concurrency (async let, task groups), Java’s StructuredTaskScope (preview in several Java releases), Rust’s tokio::task::JoinSet/scoped threads (std::thread::scope), and Go’s errgroup (approximates it).

# Python 3.11+: TaskGroup, a scope that awaits children, cancels siblings on error
import asyncio
async def fetch(name: str, delay: float) -> str:
await asyncio.sleep(delay)
if name == "bad":
raise RuntimeError("boom")
return name
async def main():
try:
async with asyncio.TaskGroup() as tg:
t1 = tg.create_task(fetch("a", 0.1))
t2 = tg.create_task(fetch("bad", 0.05))
t3 = tg.create_task(fetch("c", 1.0)) # cancelled when t2 fails
except* RuntimeError as eg: # exceptions arrive as an ExceptionGroup
print("failed:", eg.exceptions)
asyncio.run(main())

Timeouts: async with asyncio.timeout(5): ... (3.11+) cancels its body on expiry. In TypeScript there is no built-in structured scope; use AbortController signals threaded through calls and Promise.all/Promise.allSettled carefully (Promise.all rejects early but does not cancel the other promises, which keep running):

// TypeScript: cancellation by AbortSignal; time-limited fan-out
async function fetchAll(urls: string[], timeoutMs: number) {
const signal = AbortSignal.timeout(timeoutMs); // aborts all requests on deadline
return Promise.all(urls.map(u => fetch(u, { signal }).then(r => r.json())));
}

Propagating deadlines (an absolute time budget) rather than per-hop timeouts across service calls is the distributed analogue (gRPC deadlines; see API Design).

CPython’s reference-counting memory management is not thread-safe, so CPython historically used a Global Interpreter Lock (GIL): only one thread executes Python bytecode at a time. Consequences:

  • CPU-bound Python threads do not run in parallel; a multithreaded CPU-bound program may be no faster, even slower, than single-threaded.
  • I/O-bound threads work fine: the GIL is released while waiting on I/O and in many C extensions (NumPy kernels, hashlib, compression), so threads overlap waiting and native computation.
  • The GIL does not make your code thread-safe: counter += 1 can still lose updates (the interpreter can switch threads between bytecodes). Individual built-in operations like list.append are effectively atomic, but compound operations are not.

Free-threaded CPython. PEP 703 (accepted 2023) makes the GIL optional. Python 3.13 (2024) shipped an experimental free-threaded build (python3.13t); Python 3.14 (October 2025) made it officially supported (PEP 779) while still not the default build, with improved single-thread performance. Ecosystem readiness (C extensions declaring thread-safety) is still maturing as of late 2026, so check that your dependencies support it before relying on it. Related: per-interpreter GILs and concurrent.interpreters (3.14) offer isolated sub-interpreters within one process. Fast-moving area; verify against current release notes.

Workload Tool
Many concurrent network calls asyncio (or threads with ThreadPoolExecutor)
Blocking libraries in async code asyncio.to_thread / loop.run_in_executor
CPU-bound pure Python multiprocessing / ProcessPoolExecutor (or free-threaded build)
Numeric work NumPy/Numba/native code releasing the GIL, or GPU
Simple pools concurrent.futures
from concurrent.futures import ThreadPoolExecutor, ProcessPoolExecutor
def fetch(url): ... # I/O-bound
def crunch(n): ... # CPU-bound
if __name__ == "__main__": # required for multiprocessing on spawn platforms
with ThreadPoolExecutor(max_workers=32) as ex:
pages = list(ex.map(fetch, urls))
with ProcessPoolExecutor() as ex:
results = list(ex.map(crunch, range(1_000_000, 1_000_008)))
import asyncio, aiohttp # third-party HTTP client
async def fetch(session, url):
async with session.get(url) as r:
return await r.text()
async def main(urls):
sem = asyncio.Semaphore(20) # limit concurrency
async with aiohttp.ClientSession() as session:
async def bounded(u):
async with sem:
return await fetch(session, u)
async with asyncio.TaskGroup() as tg:
tasks = [tg.create_task(bounded(u)) for u in urls]
return [t.result() for t in tasks]

Pitfalls specific to asyncio:

  • Blocking the loop: time.sleep, requests.get, heavy CPU, or synchronous DB drivers freeze all tasks. Use await asyncio.sleep, async libraries, or await asyncio.to_thread(blocking_fn).
  • Forgetting await: the coroutine never runs (“coroutine was never awaited” warning).
  • Orphan tasks: asyncio.create_task returns a task; the event loop keeps only a weak reference, so unreferenced tasks can be garbage-collected mid-flight. Keep references or use a TaskGroup.
  • Cancellation: CancelledError must propagate; do not swallow it with bare except:/except BaseException. Use try/finally for cleanup.
  • Race conditions persist across awaits: if key not in cache: cache[key] = await load(key) can load the same key twice; use per-key locks or store the future (single-flight).
  • Unbounded fan-out (gather of 100,000 coroutines) exhausts sockets or memory: use semaphores or worker queues.

JavaScript runs your code on a single thread with an event loop; concurrency comes from async I/O, so data races on plain objects do not occur, but logical races across await do. Order of execution: synchronous code, then the microtask queue (promise continuations), then the next macrotask (timers, I/O callbacks).

console.log("1");
setTimeout(() => console.log("4 (macrotask)"), 0);
Promise.resolve().then(() => console.log("3 (microtask)"));
console.log("2"); // prints 1, 2, 3, 4

For CPU-bound work use Worker threads (Node worker_threads, browser Web Workers), which communicate by message passing and can share memory explicitly via SharedArrayBuffer and Atomics. Node’s cluster or multiple processes scale across cores. Promise combinators: Promise.all (fail fast), allSettled (collect all), race, any. Unhandled rejections and forgotten awaits are the usual bugs; enable lint rules (no-floating-promises).

  • Prefer higher-level abstractions: thread pools, queues, concurrent.futures, async frameworks, actors, data-parallel libraries. Raw locks are the assembly language of concurrency.
  • Bound everything: queue sizes, pool sizes, concurrent requests. Unbounded queues turn overload into memory exhaustion. Apply backpressure (Reliability).
  • Size pools by workload: CPU-bound pools about the number of cores; I/O-bound pools by (latency x throughput), per Little’s Law (System Design Fundamentals).
  • Keep critical sections tiny; do I/O outside locks.
  • Make shared state immutable or confined; use copy-on-write for read-mostly data.
  • Design for cancellation and timeouts from the start.
  • Test concurrency: stress tests, race detectors, deterministic schedulers or property-based approaches (Testing); flaky tests often indicate real races.
  • Concurrency bugs are timing-dependent and rarely reproduce on demand: reasoning and design discipline outweigh debugging.
  • Concurrency is structure; parallelism is simultaneous execution; I/O-bound and CPU-bound work need different tools.
  • Shared mutable state needs synchronization; better to avoid sharing or mutating, or to communicate via messages.
  • Memory models define visibility via happens-before; locks, atomics and channels create the edges.
  • Deadlock requires all four Coffman conditions; break circular wait with lock ordering, timeouts or message passing.
  • Structured concurrency scopes task lifetimes, errors and cancellation; use TaskGroup, scopes or equivalents.
  • In CPython, the GIL prevents parallel bytecode execution (free-threaded builds change this gradually); use asyncio for I/O concurrency and processes or native code for CPU parallelism.
  • Always bound queues and concurrency, and propagate timeouts.
  1. Hands-on (Python). Reproduce the lost-update counter with threads and fix it three ways: a lock, per-thread local counters merged at the end, and a queue-based single-writer design. Time each.
  2. Hands-on (Python). Write an asyncio crawler with a semaphore limiting 10 concurrent requests, a TaskGroup and a global timeout. Verify that one failure cancels siblings.
  3. Conceptual. Two accounts transfer money to each other simultaneously using per-account locks. Show how deadlock arises and fix it with ordered locking.
  4. Conceptual. Explain why if x not in cache: cache[x] = await load(x) is racy in asyncio though there is only one thread, and propose a single-flight fix.
  5. Hands-on (TypeScript). Implement a mapLimit(items, limit, fn) helper that runs at most limit promises at a time and preserves result order.
Solutions (sketch)
  1. Local counters merged once are usually fastest: no contention. The lock is correct but slower; single-writer via queue trades latency for simplicity.
  2. A locks from then to; B does the reverse; each holds one and waits for the other. Fix: always lock the account with the smaller ID first.
  3. Between the check and the assignment there is an await where another task can run the same check; both load. Fix: store asyncio.Task/Future in the dict before awaiting, so later callers await the same future.
  4. Keep an index pointer; start limit workers that each loop: take next index, results[i] = await fn(items[i]), until exhausted; await Promise.all(workers).
  • Brian Goetz et al., Java Concurrency in Practice (2006), still the best treatment of the fundamentals.
  • Maurice Herlihy and Nir Shavit, The Art of Multiprocessor Programming (2nd ed., 2020).
  • Nathaniel J. Smith, “Notes on structured concurrency, or: Go statement considered harmful” (2018).
  • Mara Bos, Rust Atomics and Locks (2023), excellent on memory ordering.
  • Python documentation: asyncio, threading, concurrent.futures; PEP 703 and PEP 779; the “Python support for free threading” HOWTO.
  • Joe Armstrong, Programming Erlang (2nd ed., 2013), for actors and supervision.
  • C. A. R. Hoare, Communicating Sequential Processes (1985).
  • Leslie Lamport, “How to Make a Multiprocessor Computer That Correctly Executes Multiprocess Programs” (1979), the origin of sequential consistency.