Recently I’ve been trying to help our interns with API design and I figured I should write something about it.
If I have expertise in anything, it’s probably API design. This is kind of frustrating because it is not very tangible. Like if someone is good at optimizing microcode for x86 processors or something like that, you can point to it and say look, this is what she’s good at. But API design is so much fuzzier. What even is a good API?
Not that I can’t code, or optimize, or debug — I’m pretty good at that too! In fact, I don’t think you can be a good API designer if you don’t code a lot. An API isn’t some abstract, ivory tower thing. It’s a user interface for programmers. And how are you supposed to design that user interface if you never use it?
What is a good API?
So, to get back to the question: What is a good API?
A good API is a good user interface for programmers. What does this mean?
It’s easy to understand how to use the API.
The code that uses the API is simple and straightforward.
The API helps you in writing code that is performant, correct, and bug free.
The API is pleasant to work with. (Sparks joy!)
Terms like “easy”, “simple” and “pleasant” are of course subjective. What is easy depends on what you have encountered before. Since most programmers are familiar with for-loops an API designed around that might be easier than one using other forms of iteration. But I don’t think everything about APIs is subjective. Bad APIs can feel unpleasant to work with in an almost visceral way, and they tend to share some common traits:
Functions take lots of arguments — either directly or in structs — and behave very differently depending on what arguments you pass. Some of the arguments are only used in some of the call modes.
You need multiple lines of setup before you can call the function that does the thing you want to do and multiple lines of cleanup afterward. It is not very clear exactly what setup and cleanup is needed and what “state” the system is in.
The documentation is confusing and mentions lots of special cases.
Similarly, there are signs of a good API:
Each function has a clear single purpose and few arguments. There are no confusing options that fundamentally change what the function does.
Calling a function is a one-liner. If any setup is needed, it is clear what needs to be done and hard to do it wrong.
The documentation tends to be short because it is obvious what the functions do and they don’t have a lot of special cases.
Simple things are simple, complicated things are possible.
Another important property of an API is that it is easy to implement.
This may be a bit controversial. After all, an API is an interface — why should we care what the implementation looks like? Shouldn’t we just make the best possible interface regardless of how complicated it is to implement?
I think there are two counterarguments. First, you cannot view any single software system in complete isolation. Every line of code added somewhere increases the total complexity. It is not just the cost of implementation, it also needs to be maintained, upgraded, documented, refactored, learned, etc. The more complex something is, the harder all of this becomes.
Second, the purpose of an API is to abstract the underlying implementation. But the abstraction is almost never 100 % perfect. One way or another, some properties of the implementation leak through. This can be things like:
Debugging. You call the API, but you get an error, a crash, or some other unexpected result. Now you need to go into the implementation to figure out why you got that result. (You could of course also avoid doing that and just try to debug the system as a black box, but that’s orders of magnitude harder.)
Performance. You call the API, but you get worse performance than you expected or needed. Now you have to go into the actual implementation to try to understand why the calls are so expensive.
When the implementation of the API is simple, doing this kind of deep digging is easy. There is relatively little code in the implementation and it is easy to see where something goes wrong or where time is being spent. The concepts in the implementation match up with the concepts in the API, making the transition from interface to implementation straightforward.
If the implementation is complex, has multiple layers, and different concepts than the API, this is much harder.
Let’s look at some practical tips for API design.
Know your domain and your users
When you design an API you design a way of thinking and reasoning about a problem in a structured and precise way. To be able to do that you need to understand the problem domain. For example, you can’t design an API that handles colors unless you understand RGB color theory, gamma spaces, etc.
You must also understand your users. What are their needs? What are they trying to do with your API? What’s their background knowledge? You need to be able to put yourself in their shoes — empathize with them.
The best APIs are almost educational, serving as a bridge between the users’ ideas of what they want to do and your technical knowledge of how it can be achieved.
Sometimes you can achieve this by borrowing already established domain concepts and terminology. If you can, that’s great, because your users might already know them. But often you have to invent your own completely new concepts. This is the creative part of API design, which, at least for me, is a big part of the fun.
So understanding the users and the domain is paramount, but how do you acquire that understanding? That brings me to the next point.
Minimize planning
There’s a lot of programming advice out there that says that you should plan out and fully design your system before even thinking about writing any code. If you follow this school you work in distinct phases. First, you learn about the domain, then you create the system design, and finally, you implement it.
I guess this style might work for some people, but I find it hard to learn and plan in a vacuum. I prefer to approach the design as a dialogue. I learn a little about the domain, design a single feature, implement it, then go back and learn about the next thing I want to do. As my understanding of the domain and the challenges improve, I need to go back and revise my earlier decisions.
I think this is crucial. To know how an API design works, you have to get in there, try it out, see how it feels. As you get more experience in the space, your understanding improves and this helps you make better decisions.
Consider this: At the very start of a project is when you have the least information and knowledge about the problem domain — every moment you spend working on it gives you more experience. So why should you do all the planning and decisions in the beginning, when you have the least information.
Dig shallow graves
Instead of planning everything out I pick one small task that the system needs to do and implement it. I try to pick the smallest possible task that I can think of that would still somehow be useful to the end-user. I find it valuable to implement “real features” like this instead of just building out low-level functionality because it tests how well the system integrates with everything else and gives a fairly good picture of how the API works in the “real world”.
After it is done, I take a step back and look at it to see if there are any obvious design issues that need to be fixed before moving on to the next task.
During this process, I fully expect that I will make mistakes and have to backtrack. So an important part is to make sure that if I dig myself into a hole with the design — which I’m likely to do at some point, regardless of how much planning I do — it is a pretty shallow hole that I can easily get myself out of. By taking small steps, one at a time, I never have to backtrack too far — typically the full circle of picking a feature, planning it, and implementing it is done in just a day or two.
Digging shallow graves also means trying to avoid design choices that are hard to back out of if they should prove problematic. Two situations where I’ve seen this happen are:
Basing your implementation on a third-party library that does not cover all your use cases. (Can’t do feature X, doesn’t work on platform Y, etc) If you need to change the technology your implementation is built on, you typically need a full rewrite.
