One fine summer day, three years ago, I was tasked with integrating my Rails app with a new internal micro-service. This was the first SDK I had ever made in Ruby, and the first SDK I had built where I hadn’t also implemented the service myself.

The service was using a HAL+JSON Hypermedia API, which was new to me, but seemed pretty cool. I started to write a basic SDK library that mimicked ActiveRecord where possible, to keep future SDKs (which I knew were coming) easy to create and update.

But as soon as I tried to actually start calling the service API, I ran into trouble…

  • Fields that I expected to be present on every resource were omitted when NULL.
  • It wasn’t clear which fields were required on create.
  • Some fields couldn’t be updated once set.
  • The search endpoint didn’t have the filters I needed for our admin.
  • Key names mixed snake_case, kebab-case, and camelCase in (seemingly) unpredictable ways.
  • etc.

My friendly neighborhood service developer quickly and happily fixed all these issues as they came up. A few weeks later, I finished v1 of the SDK, plugged it into our app, and called it a win.

Then a few new endpoints were added to the service. Similar issues, quickly resolved. Then the next micro-service came, for another resource type. Rinse and repeat, another few weeks of starts and stops.

I was getting my work done, but it was taking a lot longer than I (or my stakeholders) might have hoped. We tried to get better documentation of this new service as we built it, so that there were fewer unknowns. This worked up to a point, but often the service didn’t end up matching our nuanced business rules, and had to be reworked after I started building/updating the SDK for a new endpoint.

Something had to change, though being still relatively new to this process, I wasn’t sure what.

Our third micro-service was different. This time, I was invited to the API design session that our services team always did before building a new service. As I sat in the room, I got to see the proposed database diagram, API endpoints, and business rules. I got to speak into how we planned to actually call this service, and we were able to change the design to better suit our use case. When the service was built and I started the SDK, I knew exactly what I needed to do for each of the API endpoints. The implementation went much faster and much smoother.

Then we added our forth, fifth, and sixth micro-services. Each time I sat in the room while the API design was discussed, and we were able to make key changes and clarify behavior in advance. The last couple SDKs were so trivially easy to implement, I handed the work off to another Ruby developer who just used my SDK library, and called it the way they expected it to work. That was a great feeling for me as an internal library author, but it was also a testament to the process of API design which we had iterated into being simple and unsurprising.

Fast forward three years.

Another team in my company is building a Ruby SDK for a new service (sadly not using my library, but that is irrelevant for this story). I am reviewing their Pull Request on Github, and I notice some particularly awkward code related to detecting missing records. I ask about why this code is the way it is, and they say that they are basically going off the API response and guessing at a contract. The service documentation isn’t written yet, so that is somewhat understandable, but I start pushing on the service developers to get more information.

After reviewing a somewhat ambiguous design document and some half-remembered summaries of design discussions, we eventually get to the root of the decision, and are able to argue for a different approach. This particular issue was largely a failure of documentation (and attention to details that were, in fact, provided initially), but it was also an indicator that other teams are still in the isolated API design pattern our business unit was stuck in for my first couple SDK implementations.

So here is the moral of the story:

As a producer (of APIs, apps, or avocados)

talk to your consumers (aka “users” or “customers”) before you set to work.

Sounds simple right? It is easy to get this wrong, especially on larger teams, or disconnected teams within a larger organization. We might feel that it is the Product Owner’s job to talk to consumers. Or our UX Engineer. Or our Market Researcher.

This mindset sneaks up on the best of us. We get a set of detailed success criteria on our ticket. All the edge cases seem to be covered. Why can’t we just sit down to do the work?

By trusting an intermediary to understand our consumer, we lose context on the consumer’s actual needs. We lose the nuance, we miss asking the kinds of questions that only we as a producer might think to ask. And we take out an important early feedback loop. By showing the consumer what we intend to produce, we can quickly catch problems. If we wait until after we have started implementing the API (even if we rush out an MVP version) it delays that feedback, which then often means rewriting code.

As a great anonymous philosopher sarcastically tweeted:

Weeks of coding can save you hours of planning

We can definitely take this too far, having endless rounds of “discovery” meetings with our consumer.

But finding that balance is worth the effort in the long run.