Reliability, Performance & Security Basics
What you’ll learn. How to define and measure reliability with SLIs, SLOs and error budgets; the three pillars of observability and how to use them; a disciplined approach to performance work and profiling; the resilience patterns that keep partial failures from becoming outages (timeouts, retries with backoff and jitter, circuit breakers, bulkheads, backpressure, load shedding); and security fundamentals: the OWASP risks, threat modeling, and the distinction between authentication and authorization. Code is Python or TypeScript, labelled.
Reliability as a feature
Section titled “Reliability as a feature”Users care whether the system works when they need it. Perfect reliability is unaffordable and unnecessary; each additional “nine” costs roughly an order of magnitude more effort. The engineering task is to choose a target, measure against it, and spend the slack on shipping features.
SLIs, SLOs, SLAs
Section titled “SLIs, SLOs, SLAs”- SLI (Service Level Indicator): a quantitative measure of service behavior from the user’s perspective, usually a ratio of good events to valid events: the proportion of requests that succeeded; the proportion that completed under 300 ms.
- SLO (Service Level Objective): a target for an SLI over a window: “99.9% of requests succeed over a rolling 30 days.”
- SLA (Service Level Agreement): a contract with consequences (credits, penalties), usually looser than the internal SLO.
Choose SLIs that reflect user experience: availability, latency (use percentiles, p50/p95/p99, never averages alone), correctness/freshness, throughput for batch/pipelines, durability for storage. Measure as close to the user as practical (load balancer or client-side), and define “good” explicitly (a 200 that returns wrong data is not good).
Error budgets
Section titled “Error budgets”An SLO of 99.9% implies an error budget of 0.1% of events (or time) that may fail. For a 30-day window that is about 43 minutes of full downtime-equivalent. The budget turns reliability into an economic tool:
- Budget remaining: ship features, take risks, run experiments.
- Budget exhausted: freeze risky launches; invest in reliability until recovered.
This aligns product and operations: neither side argues from feeling; both consult the budget (Google SRE). Burn rate is how fast you consume the budget relative to the sustainable rate: a burn rate of 1 uses exactly the budget over the window; 14.4 would exhaust a 30-day budget in about 2 days. Alert on burn rate over multiple windows (for example, a fast window of 1 hour at burn rate 14 and a slow window of 6 hours at burn rate 6) rather than on every error blip. This produces pages that are urgent and rarely false.
# Python: compute error budget statusdef budget_status(slo: float, good: int, total: int, window_days=30, elapsed_days=10): allowed_bad = (1 - slo) * total actual_bad = total - good consumed = actual_bad / allowed_bad if allowed_bad else float("inf") # fraction of budget used expected = elapsed_days / window_days # fraction of window elapsed burn_rate = consumed / expected if expected else float("inf") return {"consumed": consumed, "burn_rate": burn_rate}
print(budget_status(0.999, good=9_995_000, total=10_000_000)) # consumed 0.5, elapsed 1/3 -> burn 1.5Availability arithmetic for dependencies: availability of a chain of hard dependencies is the product; three 99.9% dependencies in series give about 99.7%, so your service cannot be more available than its critical dependencies unless you degrade gracefully (System Design Fundamentals).
Incident response and learning
Section titled “Incident response and learning”Define severity levels, an on-call rotation, runbooks, and a clear incident commander role. Prioritize mitigation (rollback, failover, feature-flag off) over diagnosis during an incident. After incidents, write blameless postmortems: timeline, impact, contributing factors (systems and processes, not individuals), what went well, and action items with owners. Look for latent conditions: the root “cause” is rarely a single mistake, since humans are the adaptive element keeping complex systems running. Track MTTD/MTTR (mean time to detect and recover) and aim to reduce recovery time more than failure count.
Observability
Section titled “Observability”Observability is the ability to understand a system’s internal state from its external outputs, enough to answer new questions about unknown failure modes without shipping new code. Monitoring tells you that something is wrong; observability helps you ask why. The three classic signals:
- Metrics: numeric time series (counters, gauges, histograms); cheap, aggregable, good for dashboards and alerts. Prometheus is the dominant open-source system.
- Logs: discrete event records; rich context, expensive at volume. Use structured logging (JSON key-value) with correlation IDs; avoid logging secrets and PII.
- Traces: the path of a single request across services, as a tree of spans with timing. Essential in distributed systems for locating latency and errors.
OpenTelemetry (OTel) is the vendor-neutral standard (API, SDKs, wire protocol OTLP, collector) for emitting all three; it is the sensible default instrumentation choice today.
# Python: Prometheus-style RED metrics and structured logging (sketch)import time, logging, jsonfrom prometheus_client import Counter, Histogram
REQS = Counter("http_requests_total", "Requests", ["route", "status"])LAT = Histogram("http_request_seconds", "Latency", ["route"], buckets=(0.01, 0.05, 0.1, 0.25, 0.5, 1, 2.5, 5))log = logging.getLogger("app")
def handle(route, fn, request_id): start = time.perf_counter() status = 500 try: resp = fn() status = resp["status"] return resp finally: dur = time.perf_counter() - start REQS.labels(route, str(status)).inc() LAT.labels(route).observe(dur) log.info(json.dumps({"event": "request", "route": route, "status": status, "duration_ms": round(dur * 1000, 1), "request_id": request_id}))Frameworks for choosing what to measure:
- RED (for request-driven services): Rate, Errors, Duration.
- USE (for resources, Brendan Gregg): Utilization, Saturation, Errors.
- Google’s four golden signals: latency, traffic, errors, saturation.
Practices:
- Histograms over averages: percentiles from histogram buckets; averages hide tail latency. Beware that percentiles cannot be averaged across instances; aggregate buckets instead.
- Cardinality discipline: a label such as
user_idcreates millions of series and can crash a metrics store; put high-cardinality data in logs/traces. - Propagate context: trace IDs and request IDs flow through headers (W3C Trace Context
traceparent) so one request can be followed everywhere. - Alert on symptoms (user-visible SLO burn), not on every cause (CPU high); every page should be actionable; ruthlessly prune noisy alerts to avoid alert fatigue. Use dashboards for diagnosis, not alerting.
- Sampling traces (head-based or tail-based, keeping slow/error traces) controls cost.
- Add health checks distinguishing liveness (restart me) from readiness (route traffic to me).
- Use synthetic probes to detect outages before users do; use feature flags and canaries for safe rollout with automatic rollback on SLO regression.
Performance engineering
Section titled “Performance engineering”Method
Section titled “Method”- Define goals: latency percentile and throughput targets under a stated load (the SLO).
- Measure, don’t guess. Knuth: “premature optimization is the root of all evil” (the full quote continues: “yet we should not pass up our opportunities in that critical 3%”). Intuition about bottlenecks is wrong often enough to be dangerous.
- Find the bottleneck with profiling, then fix the biggest one; re-measure. The Pareto principle applies: a few hot spots dominate.
- Compare against a baseline; change one thing at a time.
- Stop when the goal is met.
Amdahl’s law (Concurrency) tells you that speeding up a part that takes 5% of the time yields at most a 5% gain.
Where time goes
Section titled “Where time goes”Most real-world slowness is not micro-level CPU cost but architectural: too many network round trips, N+1 queries, missing indexes, unbounded result sets, lock contention, serialization overhead, cold caches, synchronous calls that should be async, and chatty protocols (Databases, latency numbers in System Design Fundamentals). Algorithmic complexity ( where is possible) beats constant-factor tuning once is large.
Profiling
Section titled “Profiling”- CPU profilers: sampling profilers periodically record the call stack (low overhead, suitable for production):
py-spy,perf, Go’s pprof, Java Flight Recorder, async-profiler, Chrome DevTools/Node--cpu-prof. Instrumenting/tracing profilers (cProfile) record every call (precise, higher overhead). - Flame graphs (Brendan Gregg) visualize stack samples: width is time share, so wide frames at the top are hot spots.
- Memory profilers:
tracemalloc,memray, heap snapshots in Chrome/Node; look for leaks (monotonic growth), large allocations, GC pressure. - I/O and off-CPU profiling: where threads wait (locks, disk, network), which CPU profilers miss; critical for latency-bound services. Tracing spans give this at the service level.
- Database: slow query logs,
EXPLAIN ANALYZE,pg_stat_statements.
# Python: sampling profile of a running process, rendering a flame graph (no code change)py-spy record -o profile.svg --pid 12345 --duration 30# or profile a script with cProfile and view top functionspython -m cProfile -o out.prof myscript.py && python -c "import pstats; pstats.Stats('out.prof').sort_stats('cumtime').print_stats(15)"# Microbenchmarks: use timeit, many repetitions, realistic data; beware warm-up and noiseimport timeitprint(min(timeit.repeat("sorted(xs)", setup="import random; xs=[random.random() for _ in range(10_000)]", repeat=5, number=100)))Load and capacity testing
Section titled “Load and capacity testing”Test with realistic traffic mixes and data volumes (k6, Locust, Gatling, wrk). Beware coordinated omission: a load generator that waits for slow responses before sending the next request under-reports tail latency (Gil Tene); use open-loop (fixed arrival rate) tools or correct for it. Find the knee where latency climbs sharply: utilization near saturation makes queues explode (queueing theory: wait time grows like for utilization ), which is why you target 60-70% utilization, not 100%. Test failure modes too: what happens at 2x load, with a dependency slow, with a node lost.
Resilience patterns
Section titled “Resilience patterns”In a distributed system, failures are routine (Distributed Systems). Resilience patterns prevent a local fault from cascading. The theme: bound time, retries, concurrency and queues; isolate failures; degrade gracefully.
Timeouts
Section titled “Timeouts”Every network call needs a timeout; the default in many libraries is “forever” (or minutes). Without timeouts, threads and connections pile up behind a slow dependency and the failure spreads (resource exhaustion). Choose timeouts from observed latency (somewhat above p99 of the dependency) and keep the sum along a call chain below the caller’s own deadline; propagate deadlines downstream so work for an abandoned request is cancelled.
Retries with exponential backoff and jitter
Section titled “Retries with exponential backoff and jitter”Retry only idempotent operations (or those with idempotency keys; API Design) and only for transient errors (timeouts, 503, connection reset; not 400/404). Naive immediate retries amplify overload: if each client retries 3 times, a struggling service sees up to 4x load exactly when it can least cope (a retry storm). Mitigations:
- Exponential backoff: delay doubles each attempt.
- Jitter: randomize delays so clients do not synchronize (“full jitter”: sleep a random value in per the AWS Architecture Blog analysis).
- Retry budgets/limits: cap retries per request and, in aggregate, to a fraction of traffic (e.g. at most 10% extra load).
- Retry at one layer only; retries multiplying across layers (3 x 3 x 3 = 27) cause explosions.
- Hedged requests for tail latency (send a second request after the p95 delay, take the first response), only for idempotent reads.
# Python: retry with full jitter, a cap, and an overall deadlineimport random, time
def retry(fn, *, attempts=5, base=0.1, cap=5.0, deadline_s=10.0, retry_on=(ConnectionError, TimeoutError)): start = time.monotonic() for attempt in range(attempts): try: return fn() except retry_on: if attempt == attempts - 1 or time.monotonic() - start > deadline_s: raise time.sleep(random.uniform(0, min(cap, base * 2 ** attempt)))// TypeScript: same ideaasync function retry<T>(fn: () => Promise<T>, attempts = 5, baseMs = 100, capMs = 5000): Promise<T> { for (let i = 0; ; i++) { try { return await fn(); } catch (e) { if (i >= attempts - 1 || !isTransient(e)) throw e; const delay = Math.random() * Math.min(capMs, baseMs * 2 ** i); await new Promise(r => setTimeout(r, delay)); } }}Circuit breaker
Section titled “Circuit breaker”Popularized by Michael Nygard (Release It!). A breaker wraps calls to a dependency and tracks failures. States:
CLOSED --(failure rate > threshold)--> OPEN --(cool-down elapsed)--> HALF-OPEN ^ | calls fail fast | | | | +------------(probe succeeds)------------+<--------(probe fails)-----------+- Closed: calls pass through; failures are counted.
- Open: calls fail immediately (no load on the struggling dependency, no waiting threads) and a fallback runs (cached value, default, degraded feature).
- Half-open: after a cool-down, allow a few probe calls; success closes the circuit, failure reopens it.
# Python: minimal circuit breakerimport time
class CircuitOpen(Exception): ...
class Breaker: def __init__(self, failures=5, reset_s=30.0): self.failures, self.reset_s = failures, reset_s self.count, self.opened_at = 0, None
def call(self, fn): if self.opened_at is not None: if time.monotonic() - self.opened_at < self.reset_s: raise CircuitOpen() # half-open: allow one trial call try: result = fn() except Exception: self.count += 1 if self.count >= self.failures or self.opened_at is not None: self.opened_at = time.monotonic() raise self.count, self.opened_at = 0, None return result(Production libraries add thread safety, sliding windows, and metrics: resilience4j, Polly, pybreaker, opossum, service-mesh/Envoy outlier detection.)
Bulkheads
Section titled “Bulkheads”Isolate resources so one failing dependency cannot consume everything: separate thread pools, connection pools or semaphores per dependency or per tenant/priority class (like watertight compartments in a ship). If the recommendation service hangs, its pool fills but checkout still works.
Backpressure and load shedding
Section titled “Backpressure and load shedding”A system must be able to say “slow down” or “no”. Without backpressure, producers faster than consumers cause unbounded queues, latency growth and memory exhaustion.
- Bounded queues and blocking/failing
putwhen full; reactive streams define demand signaling; TCP flow control is backpressure at the transport level; HTTP429/503withRetry-After; gRPC flow control. - Load shedding: when saturated, reject excess work early and cheaply (before expensive processing), preferably lowest-priority first, instead of failing everything slowly. A system that serves 100% of requests at 10 s latency is effectively down; one that serves 70% at 100 ms is up for most users. Adaptive concurrency limits (e.g. gradient/AIMD algorithms, Netflix’s concurrency-limits library) adjust to observed latency.
- Rate limiting at the edge (case study).
- Queue age matters more than depth; drop requests older than their client timeout (the client has gone away).
Graceful degradation and other practices
Section titled “Graceful degradation and other practices”- Fallbacks: serve stale cache, default recommendations, read-only mode; decide per feature what is essential.
- Idempotency and dedup for safe retries.
- Redundancy across zones, with capacity to lose one (N+1; “N+2” for critical paths).
- Safe deployments: canaries, blue-green, progressive rollout, feature flags, fast rollback. Most outages follow a change.
- Configuration is code: validate and roll out gradually; config pushes cause many large outages.
- Chaos engineering: deliberately inject failures (kill instances, add latency) in controlled experiments to validate assumptions (Netflix Chaos Monkey, Gremlin, Litmus, AWS FIS).
- Dependency hygiene: know your critical dependencies and their SLOs; avoid cycles in the call graph; protect the database as the shared bottleneck.
Security basics
Section titled “Security basics”Security is a property of the whole system built into every stage, not a feature bolted on. Core principles: least privilege, defense in depth, secure by default, fail closed, minimize attack surface, don’t trust input, keep it simple (complexity is the enemy of security), and assume breach.
OWASP Top 10
Section titled “OWASP Top 10”The OWASP Top 10 is a widely used awareness list of the most critical web application risks, updated every few years (2021 edition; a newer edition was in development for 2025, so check owasp.org for the current list). The 2021 categories, with practical countermeasures:
| Risk | Essence | Typical defenses |
|---|---|---|
| A01 Broken Access Control | Users act outside intended permissions (IDOR: /orders/123 of another user) |
Enforce authorization server-side on every request; deny by default; object-level checks |
| A02 Cryptographic Failures | Weak or missing encryption of sensitive data | TLS everywhere; modern algorithms; no homegrown crypto; secure key management |
| A03 Injection | Untrusted data interpreted as code (SQL, OS command, XSS) | Parameterized queries; output encoding; input validation; ORMs used correctly |
| A04 Insecure Design | Flaws in the design itself | Threat modeling; secure design patterns; abuse cases |
| A05 Security Misconfiguration | Default creds, verbose errors, open cloud buckets | Hardened baselines; infrastructure as code; automated config scanning |
| A06 Vulnerable and Outdated Components | Known-vulnerable dependencies | Dependency scanning (SCA), SBOMs, timely updates |
| A07 Identification and Authentication Failures | Credential stuffing, weak sessions | MFA, rate limiting, secure password storage, session management |
| A08 Software and Data Integrity Failures | Untrusted updates, insecure CI/CD, deserialization | Signed artifacts, pinned dependencies, protected pipelines |
| A09 Security Logging and Monitoring Failures | Attacks go unnoticed | Audit logs, alerting, incident response plan |
| A10 Server-Side Request Forgery (SSRF) | Server fetches attacker-chosen URLs (cloud metadata endpoints) | Allow-lists, network egress controls, metadata service hardening |
Injection is the canonical example, and a one-line fix class:
# Python: SQL injection and its fixuser_input = "x' OR '1'='1"cur.execute(f"SELECT * FROM users WHERE name = '{user_input}'") # VULNERABLE: string concatenationcur.execute("SELECT * FROM users WHERE name = %s", (user_input,)) # SAFE: parameterized query// TypeScript: XSS: never inject untrusted strings into HTMLelement.innerHTML = userComment; // VULNERABLEelement.textContent = userComment; // safe: treated as text// Frameworks (React, Vue, Angular) escape by default; dangerouslySetInnerHTML / v-html bypass that.Also know: CSRF (cross-site request forgery; mitigate with SameSite cookies and CSRF tokens), XSS (output encoding, Content-Security-Policy), SSRF, path traversal, insecure deserialization, secrets in source control (use a secrets manager and scanning), supply-chain attacks (pin and verify dependencies, lockfiles, minimal base images), and security headers (Strict-Transport-Security, CSP, X-Content-Type-Options). Store passwords with a slow, salted, memory-hard hash (Argon2id, scrypt or bcrypt), never a fast hash like SHA-256 alone. Validate input at trust boundaries with allow-lists; encode output for its context.
Threat modeling
Section titled “Threat modeling”A structured way to find security problems at design time, when fixes are cheapest. Adam Shostack’s four questions: What are we working on? What can go wrong? What are we going to do about it? Did we do a good job?
Process:
- Model the system: draw a data flow diagram (actors, processes, data stores, data flows) and mark trust boundaries where data crosses privilege levels (internet to service, service to database, tenant to tenant).
- Enumerate threats per element with STRIDE: Spoofing (identity), Tampering (data), Repudiation (deny actions), Information disclosure, Denial of service, Elevation of privilege.
- Prioritize by impact and likelihood (DREAD is deprecated; use simple risk ratings, or CVSS-style severity for vulnerabilities).
- Mitigate, accept, transfer or avoid; record the decisions (an ADR works; see Engineering Practice).
- Validate: tests, reviews, pen tests, and revisit when the design changes.
Alternatives: attack trees, PASTA, LINDDUN (privacy). Keep it lightweight and regular: an hour-long whiteboard session per significant feature beats an annual 80-page document.
Authentication vs authorization
Section titled “Authentication vs authorization”- Authentication (authn): who are you? Verifying identity.
- Authorization (authz): what are you allowed to do? Decisions about permissions.
Confusing them is the root of many bugs: a valid login (authn) does not mean access to any resource (authz).
Authentication:
- Passwords: hashed with Argon2id/bcrypt/scrypt, unique salts, rate-limited, checked against breached lists. Prefer to delegate to an identity provider.
- MFA (something you know/have/are); phishing-resistant methods (passkeys/WebAuthn/FIDO2) are the modern direction because they bind credentials to the origin and defeat phishing.
- Sessions: server-side session IDs in
HttpOnly; Secure; SameSitecookies, or tokens. JWTs (JSON Web Tokens) are signed (not encrypted by default) claims; validate signature, issuer, audience and expiry; avoid putting secrets in them; revocation is hard, so use short lifetimes plus refresh tokens. Do not use JWTs merely because they are fashionable; opaque session IDs with server-side state are simpler and revocable. - OAuth 2.0 is an authorization delegation framework (letting an app act on your behalf with limited scope); OpenID Connect (OIDC) layers authentication on top (ID tokens). Use the Authorization Code flow with PKCE for browsers and mobile apps; implicit flow is deprecated. Machine-to-machine: client credentials or mutual TLS/workload identity (SPIFFE).
- SSO via SAML/OIDC for enterprises.
Authorization models:
- RBAC (role-based): users have roles (admin, editor) with permissions; simple, can suffer role explosion.
- ABAC (attribute-based): policies over attributes of subject, resource, action, environment (“doctors may read records of patients in their department during their shift”).
- ReBAC (relationship-based): permissions derived from relationships in a graph (“viewer of the folder implies viewer of documents inside”), as in Google Zanzibar and open implementations (OpenFGA, SpiceDB); natural for sharing models.
- Policy engines externalize decisions (Open Policy Agent/Rego, Cedar).
Principles: enforce server-side, on every request, at the object level (check that this user may access this order, not merely that they are logged in); deny by default; centralize policy checks rather than scattering if user.is_admin through code; log authorization decisions; follow least privilege for service accounts and tokens (scoped, short-lived).
# Python: object-level authorization (prevent IDOR); never trust IDs from the clientdef get_order(user, order_id): order = orders.get(order_id) if order is None or (order.customer_id != user.id and not user.has_role("support")): raise NotFound() # 404, not 403, to avoid revealing existence (a design choice) return orderSecrets and supply chain
Section titled “Secrets and supply chain”Never commit secrets; use a secrets manager or workload identity; rotate; scan repositories and CI logs; treat dependencies as code you run: lockfiles, vulnerability scanning, SBOMs, signed builds (Sigstore, SLSA levels), and minimal container images. Apply least privilege in CI/CD, as pipelines are high-value targets.
Pitfalls
Section titled “Pitfalls”- SLOs at “100%” or chosen without reference to user needs; alerting on causes instead of symptoms.
- Averages in latency reporting; high-cardinality metric labels.
- Optimizing without profiling; benchmarking unrealistic data.
- Retrying everything everywhere, without jitter, budgets or idempotency.
- Unbounded queues and missing timeouts.
- Treating security as a final audit rather than a design and development activity; authentication without object-level authorization.
- Hand-rolled crypto, fast password hashes, secrets in repos.
Key takeaways
Section titled “Key takeaways”- Define reliability as SLIs and SLOs from the user’s viewpoint; error budgets balance velocity and stability; alert on budget burn.
- Observability needs metrics, logs and traces tied together by correlation IDs; use OpenTelemetry; prefer percentiles to averages.
- Optimize by measuring: profile, fix the biggest bottleneck, re-measure; architectural fixes beat micro-optimizations.
- Resilience = bounded timeouts, jittered and budgeted retries on idempotent calls, circuit breakers, bulkheads, backpressure and load shedding, graceful degradation.
- Security: know the OWASP risks (access control and injection first), threat-model designs with STRIDE, apply least privilege and defense in depth.
- Authentication proves identity; authorization decides access, enforced server-side at the object level.
Exercises
Section titled “Exercises”- Hands-on (Python). Compute the error budget in minutes for 99.95% over 30 days. If an incident burned 20 minutes, what fraction remains? What burn rate would exhaust the budget in 3 days?
- Hands-on (Python). Add jitter and a retry budget to the retry function: no more than 10% of calls (tracked in a sliding window) may be retries.
- Conceptual. Service A calls B with 3 retries, B calls C with 3 retries. C is down. How many requests reach C per original request, and what do you change?
- Hands-on. Profile a slow Python function with
cProfileorpy-spy, identify the top hot spot, and verify your fix with a before/after measurement. - Conceptual. Do a mini threat model (STRIDE) for a password-reset-by-email flow. List at least five threats and mitigations.
Solutions (sketch)
- Budget = 0.0005 x 43,200 min = 21.6 min; 20 min burned leaves 1.6 min (about 7%). Exhausting 30 days’ budget in 3 days means a burn rate of 10.
- Up to (1+3) x (1+3) = 16 requests to C per original request. Retry at a single layer (the edge), add budgets and circuit breakers at B, use deadlines.
- Examples: spoofing (attacker requests reset for victim: send only to the registered address); tampering (token guessing: long random single-use tokens, short expiry, store hashed); repudiation (log reset events); information disclosure (do not reveal whether an email exists: identical response); DoS (rate limit per account and IP); elevation (token reuse after password change: invalidate sessions and tokens; require re-auth for sensitive actions); email takeover as an out-of-scope but real dependency risk.
Further reading
Section titled “Further reading”- Betsy Beyer et al., Site Reliability Engineering (2016) and The Site Reliability Workbook (2018), including the chapters on SLOs and alerting on SLOs.
- Michael Nygard, Release It! (2nd ed., 2018), the source of circuit breaker and bulkhead.
- Charity Majors, Liz Fong-Jones, George Miranda, Observability Engineering (2022).
- Brendan Gregg, Systems Performance (2nd ed., 2020).
- Marc Brooker, “Exponential Backoff and Jitter” (AWS Architecture Blog, 2015).
- Jeff Dean and Luiz Barroso, “The Tail at Scale” (CACM, 2013).
- Adam Shostack, Threat Modeling: Designing for Security (2014).
- OWASP Top 10 and OWASP Cheat Sheet Series (owasp.org); OWASP ASVS.
- Ross Anderson, Security Engineering (3rd ed., 2020, free online).
- OpenTelemetry documentation (opentelemetry.io).
- RFC 6749 (OAuth 2.0), RFC 7636 (PKCE), OpenID Connect Core specification.