Pairing with AI #3: iterative planning (or, no plan survives first contact with the code)

ℹ️ See my write-plan and implement-plan skills.

The pre-AI coding era: BDUF (big design up front) vs iterative planning

I spent the first 10 years or so of my career (going back to the 90s here, folks) in the Waterfall/BDUF era: long meetings discussing endless hypothetical scenarios, big planning documents covering months of work, missed deadlines, and deliverables that typically ended up misaligned with actual needs. The reasons for this frequent misalignment are now obvious, since our industry has matured since then:

  1. There was no feedback until we wheeled out our giant project at the end, and by then it was too late.
  2. In planning, what people tell you they want is often not what they actually want or need (they may think they know, but since there’s no software yet for them to interact with, they’re guessing, and doing their best to answer all your planning questions).

This is what spawned the Agile movement, which started to gain traction in the early 2000s, and at the time it felt revolutionary. It turned Waterfall on its head, with short development and deliverable cycles and frequent feedback (Agile has unfortunately accreted its own layers of bureaucracy since then, but that’s another story).

The AI coding era: BDUF is back, but should it be?

In our new era of AI-assisted coding, BDUF is back: we can now generate planning documents and code faster and cheaper, so why not? You can develop your plan, walk away while AI works on it for a few hours, and then deliver. If what you delivered ends up needing changes, repeat the process. This is how the popular Superpowers skill for executing plans works (and most other AI planning skills I’ve seen): “Load plan, review critically, execute all tasks, report when complete.”

There are two aspects to feedback: customer feedback, and technical feedback. Faster delivery with AI-assisted coding means you can get customer feedback sooner, even with BDUF. But critically, technical feedback is still missing. If it turns out something was poorly designed in step 1 of your 12 step plan, its effects cascade through the rest of the work, making it difficult to unwind later. And when you dutifully do your code review of the AI generated code in a giant pull request, you’re almost certainly going to miss things (this has always been true: see The Paradox of Code Reviews: Why Small PRs Get More Comments Than Large Ones).

One answer to this is to try to flush out any problems in the plan by iterating on the plan itself. Boris Tane takes this approach: “Read deeply, write a plan, annotate the plan until it’s right, then let Claude execute the whole thing without stopping.” It looks like this:

1-shot AI planning with annotation cycles

While this does help catch problems, in my experience no amount of up front planning and research is going to catch all the problems. This is especially true if you’re working in a large legacy codebase: until your new code is actually written and run, there are likely to be interactions and complications that are hard to foresee just by staring harder at the pre-existing code while planning.

The alternative: iterative planning and implementation

My approach to AI-assisted plan implementation is to stop and review at the end of each step. This means you’re reviewing much smaller change sets, so you’re less likely to miss things, and you can correct and refine the plan as you go. It looks like this:

Iterative AI planning and development

This approach is codified in my write-plan and implement-plan skills. You can follow those links to see the skills in full, but here are the “Steps” sections of each:

write-plan skill: writing steps

Steps that tell a story, each building on the one before it.

  • Each step is small, and focused on one logical and cohesive change. It may span several files.
  • Tests for new or changed behavior belong in the step that changes the behavior, not in a later step. Skip them where they do not apply, as for a Rake task, a script, or trivial config.
  • A finished step is marked ✅.

implement-plan skill: implementing steps

Run this at the end of every step that produces code to commit.

  1. Run the checks. Run the tests and the linters over the changed files. The project’s CLAUDE.md names the commands.
  2. Ask for a review. Ask the user to review the code. Name where the code departed from the plan. Wait for them to confirm that the review is finished and the code is approved. Until they confirm, leave the plan unchanged and draft no commit message.
  3. Update the plan document. Mark the step ✅. Where the implementation departed from the plan, edit the step to match what you built. Edit every later step the departure changes. The plan holds no historical record.
  4. Commit. Draft the message with the commit-messages skill, which also covers the approval the message needs before `git commit` runs. Include the updated plan document in the commit.

A key thing to note in step 2: “Name where the code departed from the plan.” I iterated on this and found that the implicit permission this gives to the agent to deviate from the plan is more effective than trying to codify a bunch of rules for when deviation is permitted. Note my experience here is with Claude Code only. I’ve found it does a good job at understanding the intent of a planned code change, and deciding when to deviate from the planned implementation details when it discovers a problem. I find it misjudges rarely with the current Opus model, since each step is tightly focused, but if it does, I’ll catch it in the review.

You mean I can’t walk away for a couple hours and come back to a fully implemented plan?

That’s right, and that’s part of the point. You have to stay engaged with the work. Many developers have understandably become despondent, as they now feel more like project managers than developers (just telling Claude what to do and feeling removed from the actual work). I actually enjoy my approach, as it feels much more like a traditional pairing session, with Claude as the driver and me as the navigator. While most of the time I don’t jump in to edit code myself, I can and sometimes do, because Claude stops at the end of each step to get my feedback before committing the code.

So give this approach a try. I think you’ll find it results in higher quality and a more enjoyable coding experience.

This is the third post in a series on coding with AI. It’s inspired by the talk I gave at RubyConf in July (video should be available soon). The previous post was Pairing with AI #2: fixing Opus 5’s unintelligible output.

Pairing with AI #2: fixing Opus 5’s unintelligible output

ℹ️ See my ASD-STE100 (Simplified Technical English) output style for Claude Code

This post was originally going to be about AI-assisted planning, but if you’re a Claude Code user, you may be experiencing the same frustration I have recently with Opus 5: it’s often incomprensible. Others have been frustrated also. Matt Pocock posted the other day: “Christ, Opus, talk like a normal person. I built this app and I have no idea what it’s saying.”

Sometimes it veers from extremely dense technical language to just bizarre. Opus served up this gem while I was doing a PR review the other day (the code under review provides alternative text suggestions):

…Then a permissions misconfiguration degrades to “you don’t get the 👀 and the summary” instead of “you don’t get the review.” The suggestions are the product; the acknowledgment is garnish, and garnish shouldn’t be able to take the meal down.

So the suggestions are the product, which I think Opus is saying is also the meal, which apparently can be ruined by a permissions misconfiguration. Oh wait, does it mean it’s actually the acknowledgements (the garnish) that are the problem? And what is the 👀 emoji referring to?… What are we talking about again? 😵‍💫

Claude’s Fable and Sonnet models don’t have this problem, but Fable is too expensive to use, and most of my current work is not “routine” so Sonnet is typically not up to the task.

You can make Opus 5 intelligible again with a custom output style.
For example, Lydia Hallie created an ELI5 output style (explain like I’m 5).

Another approach that’s getting attention is based on ASD-STE100 (Simplified Technical English):

It was originally developed in the 1980s by the European Association of Aerospace Industries (AECMA) at the request of the European airline industry, which wanted a standardized form of English for aircraft maintenance documentation that could be easily understood by non-native English-speakers. It has since been adopted in many other fields outside the aerospace, defense, and maintenance domains for its clear, consistent, and comprehensive nature.

A misunderstanding of intent in an airplane repair manual could prove fatal, so having a standard for very clear technical writing was critical. Turns out it’s great for AI assisted coding too. But I did some looking around online, and couldn’t find an example of an output style specifically based on it. So I made one: here is my ASD-STE100 (Simplified Technical English) output style for Claude Code. Copy it into your ~/.claude/output-styles directory, start a new Claude Code session, run /config and select it in the Output Styles menu.

I’ve been using it for a few days now and it’s been transformational. Opus 5 is intelligible now! 🎉 No weird analogies, no self-invented jargon, no long convoluted sentences, no hedging or obfuscation, and no rotating through multiple synonyms to refer to the same thing.

It doesn’t mean nuance is lost. It just means you get it in a series of short, clear sentences, instead of a long, obtuse one.

I used asd-ste100-skill as a starting point (it focuses only on agent output, but I want it for all output). A couple things to note:

  • I didn’t attempt to use the full STE manual as that would be quite heavy and probably too limiting (the manual has 53 writing rules and a dictionary of only about 900 approved words).
  • If you don’t use CONTEXT.md files in your projects, you can remove the “Project vocabulary” section.

This is the second post in a series on coding with AI. It’s inspired by the talk I gave at RubyConf in July (video should be available soon). The first post was Pairing with AI #1: brainstorming.

Pairing with AI #1: brainstorming

This is the first post in a series on coding with AI. It’s inspired by the talk I gave at RubyConf last week: AI-assisted coding: practical lessons from small startups to legacy codebases. Here are the presentation slides. Video should be available in a week or two.

Typically the first step in my Claude Code assisted workflow is getting a rough idea ready for planning. Not the plan itself, but brainstorming any aspects I haven’t thought through yet, or identifying possible gaps. Popular skills for this are grilling and grill-with-docs from Matt Pocock’s “AI Hero” skills and brainstorming from the Superpowers skills. I’ve worked with both and they each are suited to different needs:

RailsConf 2018 highlights

I drafted this post after RailsConf in 2018, and very belatedly realized I never published it. So I’m finally getting around to it 😉. Reviewing it now, 8 years later, I’m pleasantly surprised at how relevant most of the talks still are.

Below are my favorite talks that I attended at RailsConf 2018. Since there were multiple tracks, there were plenty of talks I didn’t have the opportunity to see. For highlights shared by others, check out RailsConf 2018 – Top 10 Favorite Talks (from Al Tenhundfeld), Our Favorite Ruby on Rails Talks from RailsConf 2018 (Planet Argon), and Five things I learned at RailsConf 2018 (Stephen Giles). There’s also a full playlist of the RailsConf 2018 talks if you might to see even more.

This was my first time in Pittsburg, so I took some time out to walk around and see some of the city, so there are more pictures below too.

RSpec: 5 rules for using let effectively

let can enhance readability when used sparingly (1, 2, or maybe 3 declarations) in any given example group, but that can quickly degrade with overuse.
RSpec Official Documentation

Motivation

I’ve been using Rspec since 2012 and in all this time I’ve never had a really clear picture of how to best use let. I don’t have this issue with any other aspect of Rspec.

Probably our most common Rubocop violation where I work is MultipleMemoizedHelpers (too many let calls). In previous consulting work, I’ve seen this in many other codebases as well.

What to do about it? On one side, you have Thoughtbot (the creators of FactoryBot) and other prominent members of the community arguing that you should never use let, because of the tangled messes they often see with let overuse in large codebases (eliminating let usage isn’t a practical option for us, given our multiple large legacy Rails applications and many teams). On the other side, you have the well regarded Better Specs site recommending let and of course the Rspec docs themselves, but their examples only cover simple cases. Then in the middle, there’s the quote at the top of this page: a somewhat cryptic comment tucked away in the Rspec code documentation, cautioning to use let sparingly.

But what does it mean to use let sparingly? What is the right strategy for reducing the number of MultipleMemoizedHelpers violations? Why should I care? After a lot of research, focusing primarily (but not solely) on advice from Rspec maintainers that I found in various corners of the internet, I formulated the 5 rules below to answer these questions.

Much of the advice in these rules is really about good habits in general with writing tests, through the lens of using let effectively. let is a tool. It’s up to you how to use it.

ℹ️ Note there’s a Claude Code skill waiting for you at the end of the post, so Claude will know how to use let effectively too.

Rails views, internationalization, special characters, and testing with Rspec

The problem

File this under small problems that take more time than they should to solve, and I couldn’t find an answer with a web search.

Let’s use a simple example. If you have text like this in your translation file (e.g. en.yml):

users:
  new:
    header: "Let's go!"

And then show it in a view template (e.g. app/views/users/new.html.erb):

<h3><%= t('.header') %></h3>

And then try to match it in an Rspec test, you’ll get an error that it couldn’t be found:

expect(response.body).to include(I18n.t('users.new.header'))

Failure/Error: expected "[...]Let&#39;s go![...]" to include "Let's go!"

Rspec with Rails 7 and System Tests

Hello world! It’s time for my first post in over 4 years.

I recently set up a new Rails 7 project with Rspec and looked online for tips, as one does. I’ve set up many Rails projects before, but not yet with Rails 7, and it’s been a while. The top result in Google for “rails 7 with rspec” is currently Adrian Valenzuela’s Setup RSpec on a fresh Rails 7 project. His post was really helpful for me shaking off the rust. So rather than writing another post that’s 80% the same, I’ll just share a few additional tips. Think of this post as a companion piece to Valenzuela’s.

Terry Toppa, 1939-2021: a remembrance of my father

My father passed away on May 10 last year, after a short and unexpected battle with cancer (aside from some back pain, he was doing fine just a few weeks earlier). I wrote his obituary the next day. There was a short graveside committal service, where I also had the opportunity to say a few words about him. I want to share those words here, and his obituary. Two days ago, March 10, he would have been 83.

“Magic: The Gathering” Standard deck brew – Jeskai Pirate Aggro

This is my first post about Magic: The Gathering, which I’ve been playing for years. If you have no idea what I’m talking about, I highly recommend this New Yorker article about the history and culture of the game or if you prefer audio, this episode of Planet Money from NPR, about how the game has managed to stay popular for over 25 years.

Ever since Rivals of Ixalan came out about a year ago, Path of Mettle has been my favorite card to try to build around. It’s a finicky card that requires your deck to be stacked with the specific types of creatures it needs, but the payoff is that, once transformed into Metzali, Tower of Triumph, it’s “a one-card, synergistic game-ender,” as Craig Krempels put it. I can’t resist trying to make a card like that work. You see Field of Ruin rarely these days, Teferi, Hero of Dominaria can’t touch it, and it can take down a Carnage Tyrant (since its ability doesn’t target). The one damage spread across the board by Path of Mettle entering the battlefield is also highly relevant in the current metagame, with a lot of one toughness creatures running around in mono-blue, white aggro, and token decks (it also hits Llanowar Elves in Sultai and Pteramander in Drakes). Of course the trick is, any competitive deck can’t rely on one card – you still need to be able to win without it, and I’ve been getting good results with this build, which I’ve been iterating on for a while.

Ad for Sugar in 1966 Issue of Time

1966 ad for sugar in Time magazine

1966 ad for sugar in Time magazine

In 1995 I photocopied this ad from a 1966 issue of Time magazine. I was in grad school doing some research on the Vietnam war, and couldn’t help but notice it. It’s almost as over the top as the old Saturday Night Live fake ad for speed. I thought I lost the photocopy years ago, but found it in a box in my basement the other day.

If you can’t make out the “Note to Mothers” at the bottom, it says:

Note to Mothers: Exhaustion may be dangerous – especially to children who haven’t learned to avoid it by pacing themselves. Exhaustion opens the door a little wider to the bugs and ailments that are always lying in wait. Sugar puts back energy fast – offsets exhaustion. Synthetic sweeteners put back nothing. Energy is the first requirement of life. Play safe with your young ones – make sure they get sugar every day.