Going Fast

Sub-second deploy times on production software are possible, and are sometimes a reasonable goal. Deploy times longer than a second are a price, and we should make sure we're getting our money's worth for that price.

The Benefits of Fast

OODA (Observe, Orient, Decide, Act) loops are the process by which . Making your software be good requires observing it, paying attention to it, deciding to do something, and then doing that thing. Fundamentally, this is what a software team is doing. There's benefits to doing each step well , but they're limited. They all benefit from being done many, many times.

Detailed observation of the current state is useful, but you can only see the current state. After you have Acted, you get a whole new state to observe. Your quick glance at the world after you have introduced a feature - say, at an inbox full of emails saying that the feature sucks - is working from an extremely privileged position, compared to your observation of the pre-feature world, which you carefully mine for information about your (non-existent) feature. (ditto Orient)

Decision making is essentially tree crawling. You look at two branches, think very hard, and pick which branch you like better. The first benefit to speed is obvious - you can just try more branches if you're going fast. The second is that decision making is a skill, and decision making specific to your app, team and industry is a skill that you probably have very little experience with. Doing it more times will just make you better at it than spending hours poring over each decision.

Acting fast is the real cost center of fast OODA looping. It's skill intensive and risky. Like the other steps it has some benefits from repetition - skill building, better worlds to practice on. There's also the benefit of having access to what you build sooner. But really, most of how we think about Acting fast will come from how to act fast, and how to do it without ruining everything.

How to Go Fast

Let's talk about when you shouldn't or can't go fast. If you're unable to observe the world quickly, or can't understand what you're seeing, looping again gains you no benefit - you're still effectively deciding and acting from where you were before the last loop. This is where metrics come from - you want something that goes up or down, right away , clearly , as soon as you do something. Software guys love performance tuning for this reason - deciding and acting can be difficult and technical, but the goal metric, execution speed, is extremely simple. And, like leveling up in some simplistic video game, the better you're doing, the more enemies you can kill times you can run the program.

You also can't go fast when you don't have any ideas for what to do next. Straightforwardly, this is a skill issue. Think about your problem more and free yourself to have stupider ideas. Frequently you will find that you, a dummy who can't even come up with a non-stupid idea, also have made a mistake in judging that an idea is stupid. Slow decisions can also come from communication - if the team disagrees, if many sign-offs are required, if the decision isn't clearly understood by the actor.

The big one, though, is the chance of irreparable harm. If, by picking a stupid idea and executing on it very fast, you lose your company's biggest contract by deleting all their data, you quickly learn a valuable lesson but you do not come out ahead. Stripe (and probably Amazon, what doesn't come from Amazon) calls these trap door decisions. There's trap door actions, too. Quite a lot of slowness comes from the goal of not doing this, or from traditions adopted to not do this - testing, reviews, staging environments, launch processes, canary rollouts.

So, how to go fast? Avoid those things . Have metrics that don't suck. Work on projects where you will be able to tell if you are doing a good job. Talk and think about your project. Develop a mutual understanding of what you are doing and how you might do it. Build walled gardens where technical mistakes aren't catastrophic but can still be observed and evaluated.

Concretely, though?

Things that are slow about development cycles, once you've decided what to write, and how to accelerate them:

Figuring out how to write the code

Keep your codebase small. Aggressively prune and combine functionality. Use consistent typing and good naming conventions. Leave comments that express your motivation. Sacrifice efficiency to do things in the simplest way that could possibly work. Keep architecture as simple as possible - shared environment, shared types, everything in a single codebase.

Writing the Code

Use AI assistants, and lay your code out in a way that is easy for them to understand. Work with tools that you're familiar with. Make the type system work for you. Write tests in a way that won't need to be updated when the underlying implementation changes. Do the hard part first - it's the part that's most likely to force a change of plans or a realignment. Use an opinionated auto formatter that hooks into your IDE and never, ever think about formatting.

Validating the Code

By this I mean all of:

Trying it out locally
Unit testing
E2E testing

And, essentially, the goal for all of them is to make it as easy as possible to put the code in the situations it will be in in production. For standalone functions, this means clean interfaces - include as little information as possible in the arguments and return types, so they can be mocked and validated easily. For more architectural pieces like database access or web APIs, this means limiting the parts of prod that can't run on local. Something like sqlite is ideal for this - you can put a slice of the actual production database in memory for unit testing. The opposite would be something like using cloudfront to validate json payloads before they reach your app - if there's an inconsistency in the expected types, you could have no way to detect an outage before staging e2e tests.

Reviews

These can be skipped, and if they can't be skipped, they should be synchronous, at the same desk. I will tell you what hell is. Hell is sending out a review using method A, getting the comment “Why don't you use method B here? It's simpler ”, rewriting the code to method B, requesting re-review at 6pm, and realizing the reviewer is on PTO tomorrow. Double hell is on Monday, when they say “sorry to make you re-do all that, I think method A was actually cleaner.”You have now torched off days for want of fifteen minutes of sitting and looking at code together.

Build &Deployment Time

Don't build software that doesn't compile in the space of a breath
Once it compiles, scp the binary into production.

Congratulations, your deployment time is now twelve seconds. Your rollback time is six seconds (your deploy script holds onto prior versions), and most of that is spent typing the command. You can knock out production once a month and, if you notice within four minutes each time, maintain four nines of uptime.

JT You're a Madman, You Can't Keep a Production App Running Like That

Listen, I didn't ask me how to run a software business, I asked me how to go fast. I would not use all, or even most, of this advice if I was running, say, AWS S3. If I was the technical cofounder of a two-person startup with no customers and a demo meeting in an hour, I'd use all of them.

The thing to do is figure out, for any given project, where you are on that line. I think the biggest place this goes wrong is unlaunched products. There's really not very much difference between a 2-person startup and an unlaunched product. You want that thing done and visible as fast as possible, because you have no idea if you built what your (customers, PM) wanted, or if they will still want what they said they wanted in the (sales call, kickoff meeting) after they see the first draft.

I also think you could probably keep a production app running like that, with a good year of practice. But that's another thing.