Software Enters a Third Era: Karpathy Maps the Road Ahead
- Nyquiste
- Jun 23
- 3 min read
Former Tesla AI chief Andrej Karpathy argues that software is undergoing its most radical shift since the dawn of computing. Speaking to students and early-career engineers, he traced three successive “software epochs” and explained why fluency across all of them will define tomorrow’s best developers.
Three Epochs of Code
Software 1.0 — Classical Code (1950 → )Programs are handwritten in languages like C++ or Python. Precision is superb, but development is slow and expertise-heavy.
Software 2.0 — Neural Networks (≈ 2012 → 2022)Here the “code” is a model’s weights. Engineers curate data, run optimizers, and let the computer write itself. GitHub stores 1.0 code; Hugging Face now serves as the GitHub of trained models.
Software 3.0 — Large Language Models (2023 → ?)LLMs turn natural language into an instruction set. English prompts, not source files, direct behavior. Karpathy calls this “programming in English.”
LLMs as Utilities—and Operating Systems
Karpathy likens today’s AI labs (OpenAI, Anthropic, Google) to power companies:
Capital-intensive “grids.” Training frontier models costs billions.
Metered access. Usage is sold per million tokens.
Intelligence brownouts. When APIs fail, work across the globe stalls.
Yet LLMs also behave like operating systems. Closed-source “Windows/Mac” rivals coexist with open-source “Linux” equivalents such as Meta’s Llama. We are roughly in the 1960s mainframe era: compute is centralized, thin clients connect over the network, and true “personal AI” is still too costly—though small-scale experiments on devices like Mac Minis hint at what is coming.
Working with “People Spirits”
LLMs exhibit quirks that feel human:
Superhuman recall for facts, hashes, and code snippets.
Hallucinations and logical blind spots—they may insist 9.11 > 9.9 or misspell “strawberry.”
No long-term memory consolidation. Everything outside the prompt window is forgotten.
Security vulnerabilities such as prompt injection.
Developers must harness strengths while sandboxing weaknesses—treating LLMs as brilliant but eccentric colleagues.
Partial-Autonomy Apps: The Real Opportunity
Karpathy warns that full “agent” hype echoes early self-driving promises: flashy demos hide years of edge-case slog. The practical sweet spot today is partial autonomy with a human firmly in the loop.
Successful apps share four traits:
Automated context management. The tool decides which files or snippets to feed the model.
Multi-model orchestration. Embedders, chat models, and diff generators cooperate behind the scenes.
Human-audit GUI. Users approve changes via clear visuals—not raw JSON.
An autonomy slider. You can request a single-line fix, a file-wide refactor, or a repo-wide overhaul depending on risk tolerance.
Karpathy cites Cursor (AI IDE) and Perplexity (search companion) as blueprints. Both accelerate the generate-verify loop: the LLM proposes, the user reviews, and the cycle repeats quickly.
Designing for Agents, Not Just Humans
Because everyone can “program in English,” documentation and interfaces must evolve:
Serve machine-readable docs in plain Markdown, not screenshot-laden pages.
Replace “Click this button” with curl commands an agent can execute.
Consider a site-level “llm.txt” that tells models what your domain does, just as robots.txt guides web crawlers.
Lightweight tools such as git-ingest or DeepWiki already transform GitHub repos into LLM-friendly dumps with a single URL tweak.
Karpathy’s Advice to New Engineers
Master all three paradigms. Decide when explicit code, trained weights, or prompt logic is best.
Build Iron-Man suits before Iron-Man robots. Focus on augmenting humans; full autonomy will be a decade-long marathon.
Rewrite everything. Legacy stacks—from Tesla’s C++ autopilot to your local CRM—will be eaten first by Software 2.0, then 3.0. The green-field work is enormous.
“Billions of people just received a brand-new computer overnight,” Karpathy concluded. “Now it’s our turn to program it.”
Comments