Teaching Agents What We Know

Orchestration and workflows as compound advantages

In a recent case study, I explored what happens when you give a coding agent explicit domain knowledge about investigative journalism workflows. The results were striking - behavior shifted dramatically when the agent had access to structured guidance about the field. But the implications extend well beyond coding and journalism. As AI agents become more capable, we need to think carefully about how to encode what we know in a form they can act on.

This challenge is well understood in software engineering. Before AI entered the picture, developers had an endless menu of tools for keeping conceptual understanding aligned with implementation: version control, documentation systems, project management frameworks, code review practices, architectural decision records. The proliferation reflects a fundamental tension in building complex systems: the gap between what we intend and what we actually build.

Software teams are now leveraging these same processes to bring AI into the workflow. I walked through an example of this with my students in a recent class: Start with a functional diagram that captures the system’s conceptual architecture, translate that into an implementation plan, then generate code from the plan. Each stage creates an artifact that can be reviewed, revised, and handed off, whether to a human collaborator or an AI agent. The explicit, concrete nature of these artifacts creates shared context.

This pattern is playing out at scale in the coding agent ecosystem. Tools like Claude Code support “skills,” markdown files that encode domain-specific workflows and best practices. Joe Amditis’ journalism skills repository demonstrates what this looks like in practice: structured guidance that shapes how an agent approaches tasks like data cleaning or source verification. The agent operates within a framework that reflects accumulated professional knowledge.

Ethan Mollick has been writing about this kind of approach through the lens of management. His argument is that the skills we’ve developed for delegating to humans - setting clear expectations, providing relevant context, knowing when to check in - are precisely the skills that matter for working with AI agents. The better you are at articulating what good looks like and where the risks lie, the more effectively you can leverage these tools. For my coding agent investigation, that meant writing explicit guidance about what investigative journalism requires - provenance, transparency, and human approval of all decisions. The agent didn’t magically become an investigative journalist, but it stopped making the naive errors that would have derailed the work.

All these examples circle around the importance of our ability to externalize tacit knowledge in a form that’s useful for collaboration. This is a challenge of articulating workflows and managing orchestration, and it’s a challenge that is rapidly becoming relevant for non-engineering domains.

Not long ago, Anthropic warned against turning to agents for many cases because they were too complex and brittle. The mental model of genAI projects was the “purple sparkles” approach, where you take an existing product and sprinkle in AI.

But now, agents work. In this newer model, the LLM sits across many different capabilities, almost integrated into your computer. It takes actions that you would, it builds tools for itself, and there’s no explicit switching from one workflow to the next. Time will tell, but the trajectory of improvement may be steep enough to largely obsolete yesterday’s best practices.

To give one example, Dan Shipper’s piece on agent-native architectures lays out a compelling vision: Build from primitives to emergent capabilities to defined workflows as needed. Start simple, add structure only where it demonstrably helps. But this is a radically different approach to developing tools, and knowing where to add structure requires judgment that the field doesn’t have yet.

In my own work, we’ve been testing LLM-powered personalization approaches for email newsletters. This used to be a deterministic set of steps, some of which called an LLM. It’s increasingly moving toward an agentic approach where the system makes contextual decisions about content and formatting. But how do we decide the right level of granularity when the models are advancing so quickly? Do purpose-built workflows still make sense? How should agents exist alongside human domain expertise at varying levels of involvement? These are the questions I find myself returning to as I try to apply these tools to non-coding domains.

The field is in a period of rapid experimentation. Every week brings new orchestration frameworks, new agent architectures, new claims about what’s possible. Most won’t survive contact with real work.

But one thing is increasingly clear: Capabilities are compounding. Using AI is no longer the differentiator; orchestration, tooling, and workflow transfer are.

This means the bottleneck has shifted. The scarce resource is the ability to articulate what you know in forms that agents can act on. The organizations and individuals who figure out that translation problem will build an exponential advantage. Everyone else will be generating output that increasingly looks the same.