Howardism · Vol. 03Plate II · No. 02
Architecture, in order.
Notes21TopicArchitectureOldest10 Apr 2026Newest23 May 2026
Model internals: encoder-free fusion, training, scaling, evals.
| Title | Summary | Date |
|---|---|---|
| The AI-Native Safe-Choice Inversion | Buying the legacy incumbent used to be "safe"; post-AI, *being* the incumbent = not AI-native; boards give buyers air cover; a counter-positioning play | |
| Building Is Cheap, Arguing Is Expensive | "In technical debate, code wins": generate three PRs vs whiteboard; prototype over design doc; reduce design docs | |
| Code as Source of Truth | Docs go stale at high coding throughput; check specs/skills into the repo; onboard via Claude; spec-drift verification | |
| Founder-Led Sales Discipline | Stay founder-led until PMF; don't offload sales to an AE *or* an agent; explicit tension with Founder As Agent Orchestrator | |
| Jagged Intelligence (Ghosts, Not Animals) | "Ghosts not animals": jagged statistical circuits, no intrinsic motivation; car-wash/strawberry failures; stay in the loop, treat as tools | |
| Narrow Wedge into a Legacy Market | Disrupt without being feature-complete: be the best for a narrow customer profile (tech cos outgrowing QuickBooks); Google-Sheets MVP; the wedge-flip lesson | |
| Outsource Your Thinking, Not Your Understanding | "You can outsource your thinking but not your understanding"; understanding as the non-delegable human bottleneck; knowledge bases as understanding-tools | |
| Product Velocity as Moat | Shipping speed as differentiator + trust signal ("you'll scale with us"); a treadmill that must convert into durable lock-in | |
| Software 3.0 | Karpathy's taxonomy: 1.0 code, 2.0 weights, 3.0 prompting; LLM as programmable interpreter; MenuGen "shouldn't exist"; neural-net-as-host-process extrapolation | |
| The Verifiability Thesis | LLMs automate what you can *verify* as computers automate what you can *specify*; RL verification rewards → jagged peaks; "verifiable + labs care"; everything eventually verifiable | |
| Problem-Solution Fit Discipline | Idea-stage thesis: three defenses against premature building (time, resources, belief friction) all eroded; AI as devil's advocate is the antidote to confirmation-bias-with-research-engine | |
| The Bitter Lesson | Sutton 2019: scaled general methods beat hand-engineered structure; recurring justification across the wiki for dissolving harnesses into models; caveat — mechanical verification and character may not migrate inward | |
| Time-Aligned Micro-Turns | The core interaction-model move: input/output as continuous streams in ~200ms interleaved chunks, no turn boundaries; streaming-sessions inference (upstreamed to SGLang), latency-tuned MoE kernels, bitwise trainer-sampler alignment | |
| TML-Interaction-Small | TML's first interaction model: 276B MoE / 12B active, audio+video+text in / text+audio out, 200ms micro-turns, async background agent; best turn-taking latency of any model; research preview May 2026 | |
| Model Spec Midtraining (MSM) | New training phase between pretrain and AFT: train base model on synthetic docs discussing the Model Spec; controls AFT generalization; cuts agentic misalignment 54%→7%; beats deliberative alignment baseline | |
| Model Spec Science | Empirical study of which Model Spec features best generalize alignment; value explanations > rules alone, specific > general "be ethical" framing; first concrete examples in Li et al. 2026 | |
| Opus 4.6 → 4.7 Changes and Multi-Agent Coding Considerations | 4.6→4.7 delta table + six hazards for multi-agent coding teams: role-based model selection, prompt re-tuning, harness invariants, per-agent context budget, unattended-fan-out safety, independent reviewer | |
| Scale-Dependent Prompt Sensitivity | Large models underperform small ones on 7.7% of standard benchmarks due to overthinking; brevity constraints recover 26pp and fully reverse hierarchy on GSM8K/MMLU-STEM | |
| When to Use Claude Opus 4.6 for Work | Decision rules for Opus 4.6 deployment: solver-not-planner, elaboration-load-bearing tasks, brevity constraints, Pareto frontier check | |
| LLM-as-Compiler Knowledge Base | Karpathy's architecture: LLM incrementally compiles raw docs into a persistent interlinked wiki, replacing RAG with a 4-phase ingest→compile→query→lint pipeline | |
| What Are AI Tools? | Overview of AI tools landscape and categories |