Matthijs van Henten

Applied Agent Orchestrator

Radical Shift Left

So with Markdown rapidly becoming the industry standard programming language, I think we aren't talking about a very important concept that should be leveraged to reduce the slop and complexity that we are creating together using LLMs.

At this point, LLMs have become so powerful that a single prompt can build a significant part of your application. Senior engineers know that even though they have a ton of experience and context, they need to chop up different pieces of context so that they can reason and fit them into their short-term memory. Language is the first compression algorithm to replace stares, song, and expressive dance or violence.

For LLMs this is, funnily enough, not much different. These forgetful senior or staff level engineering robots seem to be hyper-focused on one area and then forget to pay attention to the other all the time.

And, since LLMs aren't that much different from brilliant engineers in their onboarding week, we have ways to deal with that.

These are the same ways that we've been dealing with teams that have large cohorts of junior programmers, that are dealing with unreasonable delivery pressure, or simply just have a lower bar for quality. We introduced CI/CD, testing pipelines, test coverage targets, integration testing, etc. On the front end, we add end-to-end testing, visual diffing, component libraries.

But the one thing that has the potential to reduce the risk significantly is a radical shift to the left.

If the pipe dream is that the specification becomes the app, and those who have been around the industry know that this pipe dream has been around for a while (UML! BPMN!), then logically from that follows that the things that build the components in your flowchart must be created as small atomic building blocks.

If we accept that agents write the assembly level code for applications (Python, Typescript, or, god forbid PHP and Java), then we humans are looking for a higher level programming language, but also we're looking for deterministic artifacts that not only capture the intent and the functionality that we desire, but also lock in the inputs and outputs that we design for and shortcut the feedback cycle for this particular piece of the puzzle.

While markdown as a format, and English as a grammar, do a reasonable job at carving out intent, requirements and general direction (after rigorous review, reshaping and redefining, with ontologies taking care of managing our vocabularies), specific DSLs like Mermaid, PlantUML, Smithy and TypeSpec take care of defining contracts in an even more rigid shape.

So if we go back to the dream: "The spec becomes the app", then doing something as simple as taking OpenAPI model definitions, turning them into Mermaid ERDs, then turning them into TypeScript ORM models, API resolvers, and type checkers is the logical consequence of that dream. This is what I talk about when I talk about the radical shift to the left.

It's not a political ideology, although I am known for having socialistic sympathies. It is in fact the right way to solve building complex applications in the modern era of AI coding.

LLMs are probability engines. We can steer the probability in our favor and we can make them behave more or less predictably. But even basic SaaS products can quickly become complex enough to start resembling a Jenga tower of vibe coded bits and pieces. At the same time LLM coding has changed the economics of building frameworks, generators, template engines. Where in the past it would make sense for a team to adopt an opinionated framework and stick with it despite it not being fit for purpose or fit for the business they were in, we can now take those frameworks and figure out what the framework we needed should have been, and build one ourselves. And where building ORMs or other abstraction layers used to be the bricklaying, the soul-crushing, menial work for juniors, it has now become a single prompt with the test coverage target being the ultimate feedback loop for an LLM.

So the radical shift left now has become my primary engineering philosophy. If there are aspects of a project that can be generated by analyzing existing definitions, contracts or specifications, then these components should be generated. LLMs can build GraphQL resolvers using OpenAPI specifications on top of your REST APIs and database models, but more importantly, they can build a framework or generator library that does that without expending a single token.

LLMs can build complex ORM interfaces based off of your OpenAPI component models. But it's better to build the generator that can produce the same output over and over again at an industrial scale. LLMs can build type checkers and runtime validation libraries that take your existing definitions and turn them into deterministic code branches that you can leverage throughout your project. By doing so, you reduce the margin for error and the margin for probability to go out of bounds.

Over the past months, me, my team and my organization have proven that greenfield projects and retrofitting legacy systems can be done fast, frictionless and without creating slop by this radical shift left. I've released several open source libraries that we leverage day to day in our projects to generate code and build REST APIs at an industrial scale. At this point, the patterns and the recipes have become so solidified that standing up a new service is a matter of days and sometimes hours, including end-to-end test coverage, integration testing and demoable business use cases.

So I hope this rant will inspire you to dig deeper into shifting left - engineering wise :)

References

Some of the TypeSpec emitters and generators mentioned above (kattebak is my playground org):