Breaking down the docs testing barrier with AI

One thing I believe to be true: AI works best when it reduces barriers to entry, not when it works for us. Let me explain through a recent experiment that's changed how I approach complex technical skills.

When I first heard Manny Silva coin the term docs as tests, my immediate reaction was: "Hey, I've kind of done that before!" The methodology is simple: test your docs the same way engineers test code to ensure docs stay in sync with your product. This was something I was already actively experimenting with using GraphQL introspection and UI testing frameworks.

Since Manny started socializing docs as tests, I've watched technical writers' eyes light up with goals of applying it to their workflows. But then reality hits. The technical complexity feels overwhelming. At Docker, we test UI flows using Playwright. Some feel intimidated by Playwright, let alone Cypress or other frameworks engineers use for UI testing. And honestly? We're busy. Learning a new technical skill requires time, dedication, and interruptions to our already packed schedules.

This is where AI becomes genuinely useful. Think of AI not as a replacement here, but a learning bridge.

The barrier to entry problem

Here's what typically happens: We hear about docs testing, get inspired, open tesing framework documentation, and immediately feel lost. Most testing frameworks (with the exception of Doc Detective) don't have documentation to outline docs testing use cases. The learning curve feels steep and the setup seems foreign.

I've been here. It's the same feeling I had years ago when I was learning to code. The overwhelming sense that everyone speaks a technical language you don't understand yet.

But I've discovered AI can translate that technical language into something digestible, if you use it consciously.

A real-world experiment

Let me share an example of how this played out for me. At Docker, I needed to build out CLI command testing using Doc Detective. The goal was to create tests that validated CLI commands we provided as examples in our guides, then run these tests on an automated schedule alongside our Playwright UI tests.

While I'm pretty experienced with Cypress and Playwright, Doc Detective was new territory. I was confident I could learn it, but as the sole contributor and maintainer of our docs testing suite, I was struggling to balance learning with building.

Here's where I decided to experiment with a new AI strategy.

First, I did my homework. I read Doc Detective's documentation, went through their Get Started guide, and wrote my first test. I wanted to understand the basics before introducing AI into the equation. After feeling confident I understood the fundamentals, I dove into the reference documentation to understand how CLI command testing worked with Doc Detective.

I'll pause here to mention something crucial: this learning phase is key to actually understanding the tools you're using. You can't skip it.

Only after grasping the basics did I spin up a new Claude project. I uploaded plain text versions of specific sections of Doc Detective's documentation into the project's knowledge base. My goal wasn't to create a general Doc Detective assistant. I wanted to build something specialized for my exact use case.

I asked Claude to create a tutorial for me to walk through building my first production tests side by side. Crucially, I didn't ask it to write the test for me. I asked it to use the Doc Detective docs as a reference database to help me build something for my specific use case.

It worked spectacularly. With Claude's generated tutorial, I was able to write my first CLI command test while actually learning Doc Detective patterns and logic.

From learning to building

But I wasn't done yet. I had a lot of CLI command tests to write, and I was still working solo on this project.

After creating that first successful test, I had an idea. I fed my well-structured, working test back into my Claude project and labeled it as a "good example." Then I stripped it down with placeholders to create a template that included basic setup and cleanup steps. This became my boilerplate for future tests.

This is when things got really interesting. I used this template as my foundation and worked alongside Claude to create additional tests. My workflow evolved into something smooth: take my boilerplate, work with Claude to adapt it for new CLI commands, polish what Claude returned (because AI absolutely makes mistakes), iterate together, and finally land on a polished test.

The more I worked with my Claude Project, the better it became. It learned my patterns, my preferences, and my specific use case. This felt like a genuinely good use of AI. I was using it as a specialized expert that I'd trained for my exact workflow, with my tool's documentation as the reference point, while I maintained control over reviewing and refining the output.

The result? I learned Doc Detective, developed skills in writing, reviewing, and editing CLI tests, and worked significantly faster than I would have trying to figure it out on my own.

What this really means

This shift in how I approach learning complex technical skills reflects something bigger happening in software development. We're witnessing a fundamental change in developer workflows. Developers are moving away from the traditional pattern of "read docs, build, publish" to a new model of "ask AI, iterate, publish."

This shift isn't necessarily bad. It's just different. And for technical writers, it presents both challenges and opportunities.

The opportunity for us? AI can reduce barriers to entry, allowing even the most technically intimidated writer to create sophisticated tests and automation workflows. Skills that once required extensive time and dedication can become accessible in a matter of weeks.

But here's what's crucial to understand: the key to using AI as a specialized tutor is enhancement, not replacement. When I fed my successful test back into Claude as a "good example," I wasn't just creating efficiency. I was teaching the AI to work within my standards and patterns. When I polished and iterated on Claude's suggestions, I was building my own expertise while leveraging AI's speed.

Why this matters

This experiment taught me something important about our relationship with AI tools. The most valuable applications aren't about automation. They're about accessibility. AI can make complex technical skills approachable without making us dependent on it.

How many times have you avoided learning something because the entry point felt too steep? How many methodologies have you bookmarked but never implemented because you didn't know where to start?

AI can bridge that gap between "I want to learn this" and "I understand this well enough to apply it." But only if we use it as a learning partner, not a replacement for learning.

The conscious choice

The key word in all of this is conscious. It's easy to let AI do everything and learn nothing. It's harder, but more valuable, to let AI guide you while you do the work. The difference determines whether you end up with copy-pasted code you don't understand, or actual skills you can build upon.

This approach isn't limited to docs as tests or Doc Detective. It applies to any technical skill that feels out of reach. API testing, automation workflows, data analysis. All of these become more accessible when you have an AI collaborator that understands your specific context and standards.

The conscious choice to learn alongside AI rather than through AI makes all the difference. You accelerate your learning while maintaining ownership of the work. You build genuine expertise while leveraging AI's speed and pattern recognition.

The energy in our industry might feel defensive right now, with everyone scrambling to AI-proof themselves. But maybe the real opportunity isn't in competing with AI. It's in using it to become more capable, more curious, and more willing to tackle the technical challenges we've been avoiding.

This is AI at its best. Not doing the work for us, but making the work possible for us. Turning barriers into bridges, intimidation into curiosity. And that feels like progress.