1 — Software Architecture & System Design

Software Architecture & System Design for Production Systems A CONCEPTUAL & REFERENCE DECK Software Architecture & System Design for Production Systems Md Ahbab Hamid Khan https://ahbab.dev/ Companion note to the opening slide: from one HTTPrequest to patterns, data, teams and defensible decisions. FRAMING 1 One request is a better spine than a catalogue A pattern catalogue is a list of answers with the questions thrown away. It readswell and teaches little. A reader can learn the words CQRS, saga and sidecar inan afternoon, and still have no way to tell whether a given system needs any ofthem. Worse, a catalogue rewards recall. Design rewards judgement.A single request has one property a catalogue never has: a clock. Follow onerequest and every claim becomes checkable. Each hop costs time, memory, sockets,connections and money. When the trace is on the board, an argument aboutmicroservices stops being a matter of taste. It becomes a question about whichhop got slower, which one got a new failure mode, and who now carries the pager.The spine used in the talk is a plain one: a keystroke in a browser, a DNSanswer, a TLS handshake, an edge cache lookup, an ASGI server accepting thecon

Software Architecture & System Design for Production Systems Rule of thumb If the request path cannot be drawn on a whiteboard in two minutes, nobody on theteam can predict how the system fails. That is an architecture problem, not adocumentation problem. Browser keystroke DNS and TLS setup Edge CDN, WAF ASGI server Business logic Database WAL, fsync Serialise response Network back out Parse, layout, paint Page is usable 1 to 2 RTT hit or miss event loop durable write critical rendering path Figure 1. The life of one request. Solid navy is the outbound path, dashed teal isthe return path that ends at a usable page. Client: browser, mobile app, another service Edge: TLS, WAF, CDN cache, rate limits Application server: ASGI workers, event loop Business logic: rules, validation, orchestration Data layer: pool, database, cache, object store Background work: queues, workers, schedules scales with users and devices scales with points of presence scales with processes and cores scales with team size and clarity scales the hard way: state scales with queue depth request Figure 2. The six hops as a stack. The request goes down, the response comes backup, and each layer has its own unit of s

Software Architecture & System Design for Production Systems Table 1. Commonly cited orders of magnitude for latency reasoning. These arerules of thumb for budgeting, not measurements from any specific machine. Operation Typical order In L1 units What it means in practice L1 cache reference 1 ns 1 The CPU is almost never the reason a re- quest is slow. L2 cache reference 4 ns 4 Still free at request scale. Main memory reference 100 ns 102 An in process cache hit. Cheap, but not zero. SSD random read 0.1 ms 105 One disk seek costs about a hundred thou- sand memory reads. Datacentre round trip 0.5 ms 5 × 105 The price of one extra service call inside a region. Indexed Postgres query 0.2 to 2 ms 106 Parse, plan, index descent, row fetch and network, together. DNS lookup, cold 20 to 120 ms 2 × 107 Paid once per client, then cached by the resolver. TLS handshake, resumed 0 to 1 RTT 3 × 107 About 30 ms on a 30 ms link. Often hidden by keep alive. TLS handshake, full 1 to 2 RTT 6 × 107 About 60 ms on the same link, before any byte of your data. Cross continent RTT 100 to 200 ms 1.5 × 108 Physics. No amount of code removes it. Sequential scan, 1M rows 100 to 500 ms 2 × 108 The tax for a mi