Stephen Mizell
API by Design cover
New book API by Design

My book API by Design is now out! It's a book about finding ways to measure complexity so we can design and build better APIs.

Buy now

Posts

About My Book, API by Design

I fell into the topic for my book API by Design by accident. This is a behind-the-scenes look at how that happened and a deeper glimpse into the premise that API design should be more accessible to people who are just starting out.

The challenges of API testing

I was researching API testing years before writing this book and discovered Gary Bernhardt’s wonderful talk called Boundaries. He made a claim that it’s not feasible to test the entirety of a system from the outside in. The problem is with the math of it all. There are too many tests to write by hand to test every possible state of the software. If there are just 20 conditionals in the code—if true do this, else do that—you need 220 tests to cover all the possible combinations of those conditionals. That’s over a million tests. Most code is more complex than 20 conditionals.

This creates a problem. Developers are left to try to find bugs by testing a very small percentage of all the possible states of their application. It’s up to them to look in the places where there may be issues in hopes of catching issues early. But it’s a game of chance, writing a few hundred tests by hand hoping to find the problems in a sea of millions of possible states of their software.

So I started reading more about this issue and found how people use fuzzing, property-based testing, and generative testing to let computers do the testing for them rather than write all those tests on their own. The idea is, given a well-defined interface, computers should be able to come up with millions of tests on their own and run them as a way to find bugs before they get into production.

I wanted to do this with APIs. And the OpenAPI document seemed to be the place to start.

To do this, I wrote some code to calculate the possible states of a schema and generate examples for them. I was going to generate a bunch of tests for an API this way by testing all the valid JSON structures and common problems with them. I had a working prototype for my tiny OpenAPI document I was working with and decided to point it at a larger, more-complex OpenAPI document.

My laptop churned and churned trying to create those tests. It never finished. There were too many possible states to generate in a reasonable time without some serious optimization work.

I made a few tweaks and decided to calculate the number of possible states instead of generating all the examples. This was fast, and when I got the numbers back, I was shocked. “There’s no way for me to generate and run this many tests,” I thought. “This is way too complex.” And that was where I started thinking about complexity in API designs.

I realized these calculations gave a sense of the complexity of the API schemas and the overall API. This in itself was useful, so I kept digging.

The barrier to entry is too high

This is an assumption, but I think that the reason it takes so long to get proficient at API design is because you have to cultivate a feel for what’s complex and what isn’t. You have to know how a decision today will affect the API for months and years to come. This doesn’t leave much hope for the people starting out.

I remember well my experience starting out with APIs. I was working at a small company and needed to build something that could tie two different systems together. An API became the best choice. I started reading, getting books on the topic, finding blog posts, watching videos—the information I needed to learn was spread out all over the place. And the advice was always so divided. I’d read something giving advice for building APIs. Then I’d read something else saying to not build APIs a certain way because it was bad. Then the next thing told me that way was bad and to do it this other way. As someone who struggles with perfectionism, it was tough to have to synthesize so many conflicting approaches. The lack of metaphors We communicate primarily through metaphors, and the API industry doesn’t have many great ones. Even the acronym API is unhelpful. It doesn’t convey what it is or how we use it. When I tell people I work on APIs, they always ask what API stands for. My response is, “It stands for application programming interface, but that doesn’t really mean anything to most people.”

We use metaphors for sharing understanding about deeper topics. We talk about time as money—we save it, spend it, waste it, and invest it. In software, we talk about computers as if they were factory workers. We give them jobs, they run tasks, they do work as workers, they pick jobs off a queue. Without metaphors we’re left with the abstract, making it hard to share knowledge with others. That’s where we are with APIs.

Why I wrote this

APIs are too complex to test. We make it a challenge for newcomers to learn API design. And we lack good ways to talk about these things due to a lack of good metaphors. I wrote this book to try to address these issues.

If API design is about managing complexity, then anyone designing and building them should be able to understand the consequences of their design choices as they make them. We can’t deal with complexity after the fact—the problems compound on themselves and the complexity spreads. And we need a better language to talk about complexity in APIs in order for us to share the knowledge about dealing with it.

I’ve tried to explore these things in this book, using the metaphors of physics and entropy to talk about complexity. I hope this makes the topic more approachable. In doing so it helps everyone—if we can get everyone making better design decisions then we can have better APIs across the entire industry.

Read more →

Weeknote 21

I’ve been writing a lot these past few weeks. I’m excited to say I finished the first draft of a book I’ve been working on. It’s a shorter one, but I wanted it to be short. I hope to have a pre-order up soon for it along with more information on what it’s about. It’s about dealing with complexity in API design. On writing and editing I’ve been using my Remarkable 2 for writing.

Read more →

Weeknote 20

I’m experimenting with an idea. I bought a Mac Mini M1 early this year, but I’ve wanted to have something I could use away from my desk. I considered getting a new Mac laptop to replace my 2014 one that’s in need of a new battery. I also considered getting an iPad with a keyboard. But I decided to try out a Chromebook. There are lots of things I’m not thrilled about with a Chromebook.

Read more →

Weeknote 19

This is my sixth weeknote in a row that I’ve published without publishing a regular post. I’m OK with it. I’m writing a lot, personally and for work, so I’m keeping the habits, just not on here. I’m fortunate to be working with companies right now that put a high value on writing and ask me to spend a big part of my time doing it for them. Consuming too much I read a tweet recently that got me thinking about the amount of content I consume in a week.

Read more →

Weeknote 18

This was a quieter week compared to the last few. I did a lot of personal writing and writing for work. The driving range where I hit golf balls has been closed, so that’s been a bummer. On communication I struggle in my work life finding ways to express myself. I’ve experienced that a lot these past few weeks. My issue is that I use weak language to share my ideas and opinions.

Read more →

Weeknote 17

I had my 20-year class reunion this past weekend. They couldn’t find enough interest in having a larger party, so we had a small gathering at a restaurant with outdoor seating. It was fun, and I was able to remember everyone’s name. I’m now in the habit of using Field Notes as a way to keep notes as I’ve mentioned here before. I finished up one this week and started up a new one that’s part of their National Parks Series.

Read more →

Weeknote 16

This was my first week back to work after my time off, and wow it was tough to get back into the groove. We started homeschooling for the 2021-2022 school year this week. I’m helping out by teaching writing for both kids—my wife heroically handles the other subjects. For writing, we’re working on making up stories together by drawing pictures and then writing out what’s going on. We try to think about how it fits on a story arc.

Read more →

Weeknote 15

Yesterday July 30th was our 10-year anniversary. We celebrated by traveling to the Ocoee River in Tennessee to go white water rafting. We went rafting on the Ocoee on our honeymoon and thought it would be fun to retrace our steps a decade later. The river never disappoints. It’s dam-controlled and always has flowing water and good rapids. There are two different sections you can raft on the Ocoee—the upper and the middle.

Read more →

Weeknote 14

This was another vacation week—one more to go next week. We spent a few days at the pool, I played golf with my brother, and we cleaned out my car for our upcoming trip to whitewater rafting. To give an idea of how long it’s been since I’ve cleaned out my car, I found three toothbrushes from my trips to the dentist under the seats. My dentist gives out goody bags after a cleaning visit that include a toothbrush, a tiny tube of toothpaste, and floss.

Read more →

A Rule of Three for APIs

Leonard Richardson gave a talk back in 2016 called The Magic Arrow. He shared his experiences building a mobile application for the New York Public Library that allowed the patrons to borrow and read ebooks. He found there were characteristics of architectural designs that made them more extensible and reusable, and he summed them up under the idea of the magic arrow. When we draw architectural diagrams, we draw boxes and connect them with arrows.

Read more →