Setting the Right Expectations
It is tempting to claim that security is requirement number zero as it emphasizes the importance of building secure software. I cannot deny that security is important and that is why the previous episode ensured that the security framework for the Phonebook application is top notch or at least adequate. There is one requirement even more important than security, and that is usability. When a product, any product, is not usable, it is in fact useless. And useless products are not worth the effort to engineer them. In this episode I will show how you can develop a product efficiently that is usable by design. It is based around the notion that by collaborating with your user, you will ensure that the product you develop is valuable from its first release.
This is a multi-part second episode of the ongoing series. Enjoy and applaud.
All code for this episode is as always available from my Github: repository.
In the previous episode, the first of the second season, we created a safe environment to develop our online Phonebook application. Safety was provided by the Gradle plugins that check for vulnerabilities in our dependencies, whether our dependencies are up to date and whether we created vulnerabilities in our own code. See the previous episode for the details, you can find it here:
With the knowledge that our online application will be safe to use by our users and will be safe to run by us, safe enough at least. That is a comforting thought.
This episode will discuss delivering the first version of the application to our first users. We will be its first users. This approach ensures us that the time we spend building this application spend time will be worth the effort. We will adopt a Consumer Driven Contracts approach. We will ensure that the users of our APIs will be involved in defining the API from the first line of code we write.
We will respect our consumers’ rights, and make sure that we develop something that is not only needed, but also useable.
Pencil and Paper Design
One of the more ironic facets of software development is the fact that design is best performed with a pencil and paper. That is my experience. Using digital tools are great in that they allow you to ‘undo’ your errors while drawing, but in my experience, they are ineffective when you need to design collaboratively with all your stakeholders. As of yet, I have not come across a tool that allows multiple people to draw on the same canvas in the same design. Digital tools are just too cumbersome and there is always a lag between drawing and displaying. Nothing beats a whiteboard in that respect.
Even when I create the design all myself, I tend to use my whiteboard, or as with the design for this episode, my reMarkable 2. It is a digital version of a notebook, with all the same characteristics. There are no frills and features that district me from the task at hand; Designing.
The application will be exposing all information and functionality through a RESTful API. Users of our application will access an endpoint, the API, over http and will receive a json document as a response. The application will not provide a user interface to the user. This means that a user interface will have to be developed separately, using the API.
The responses the application will return will be defined by the consumer of the API and its implementation will be developed by us. The benefit of this approach may not be obvious, an elaboration is warranted.
Let me take you back to season 1, where I explained the merits of Test-Driven Development (TDD) and Behavior Driven Development (BDD). In short, both concepts complement each other and ensure that you only develop what is specified and tested. We continue this approach and ensure that we only develop something that is useful and usable.
A nice start is to understand RESTful APIs. We will start with the http verbs
DELETE. There are more verbs of course, but we limit ourselves for now to these five. In fact, this episode will reduce the scope to only one http verb;
A RESTful API does not necessarily have to implement these verbs. It is common practice though, to stay with the http verbs when defining an API. We will stick with the http verbs and start with GET.
The first API we will develop allows the consumer of the API to retrieve a specific contact, and while we are at it, we will also provide an API to retrieve all contacts. The design shows this ‘idea’ because that is what it is right now.
The design shows that when the user accesses the URL
http://localhost:9090/contacts. All contacts stored by the Phonebook application will be returned. This is usually the simplest API that does not require a lot of discussion.
Though what is a lot more discussion-worthy, is how to retrieve a specific contact. As you can tell by looking at the Resource Model that there are three ways of retrieving a specific contact. The most common and often least user-friendly way of retrieving a specific resource, e.g., a contact, is by its unique id. A number that is usually generated in a way that makes it unique, preferably globally unique. When defining unique ids as part of your resources, you will want them to be meaningless, allowing you to use a generic algorithm or function to generate them. Unfortunately, it also means that these ids are hard to remember. Therefore, exposing this type of id in a user interface would prohibit a pleasant user experience.
Although ids are not user friendly, APIs are not necessarily intended to be accessed by users, but by other systems. This could be a user interface but could also just as easily be another system. Systems do not mind meaningless ids, meaningful names are to systems the same difference. The convention for APIs, especially RESTful APIs, is that the id-based approach is provided by default. This means that we will provide at least the following URL
http://localhost:9090/contacts/1 to access the contact with the value 1 as its id. When there are only ten ids this is fine, but then we would not need a Phonebook application, would we?
Let us also make it a bit more user friendly and allow for the retrieval of a specific contact based on the name and provide the following URL
http://localhost:9090/contacts/names/arc-e-tect This is one way of defining the endpoint to retrieve a specific contact. This is where you talk to your users and ask whether this is a convenient API.
The user, i.e., the consumer, will tell you that they do not need an API that allows retrieval by name, or they will tell you that this is in fact a very useful API. A lot more useful than the id-based retrieval of a contact. You can provide an endpoint for the API, for example the URL
http://localhost:9090/contacts?name=arc-e-tect. The two definitions for the name-based retrieval API, through the path or the parameter, are equally good. Neither is bad. This is the reason why we want to have, at this point, the discussion with our consumer and preferable more than one.
We want to discuss with them which endpoint is the most convenient endpoint. Do the future users of the API consider the URL
http://localhost:9090/contacts/names/arc-e-tect more convenient than the URL
http://localhost:9090/contacts?name=arc-e-tect or the other way around? They decide, and as the future consumers of your API it is their right, which the name-based retrieval API is going to be. This guarantees that you will develop the most usable API out of the two.
Your next question to ask your customers, the consumers of your API, is whether they have one or more phone numbers. This is of course just a ruse to get them thinking about what the name-based API should return. Likely, the first thought is to return a specific contact based on the name of the contact. But this question will result in the understanding that this API will have to result in a list of contacts with the same name. While your customers think of this, please think of this yourself for a moment.
For now, we go for the common id-based API. We will not concern ourselves with the name-based API at this time.
It is time to develop our application and implement the API we discussed and agreed upon with our customers. We will do this in a way that is going to get us feedback from our customers in the quickest way possible. We will mock our whole application, and it will not even be a mock-up. Be ready for the paradox.
Two Faces of the API Coin
An API always has two sides, like two sides of the same coin. There is the interface side, which is the contract defined by your API’s consumer. And there is the implementation of the API, the implementation of the contract. The beauty of APIs is that as the implementor of the API you get to decide how the interface is implemented, as long as you do not break the contract. It can be as efficient or inefficient as you want. You can have a thousand pigeons flying around as an implementation, as long as it respects the contract.
We want to make sure that our APIs are useful and usable. We already think they are usable because our customers defined the contract. But thinking is what you do when you do not know. So, we implement the API and allow our customers to use it, experience it and provide us with valuable feedback to understand what is good and useable and what is not so good and needs improving.
I like to use a mocking framework, MockServer, to implement the API. It is one of two frameworks I like to work with. The other is WireMock. Both are each other’s equivalents. I like MockServer better than WireMock because I got it to work faster. WireMock has a great integration with Spring Boot, the framework our Phonebook application will be based on. On the other hand, MockServer is great with Cucumber, which is the better fit for me eventually. You could say I like BDD more than Spring Boot. Whichever you use, they are both great. Yet, in this series will use MockServer.
You may recognize the class name of the application, which is the application we have developed throughout. There are a few noteworthy changes that deserve some elaboration.
Since your appetite should be raised by now, we will start with the MockServer:
private static ClientAndServer mockServer = startClientAndServer(9090);
Here we programmatically create a MockServer that will listen on port 9090. This MockServer can, when told to, act as both a client and a server. For now, we only need the server part. A MockServer instance can be created programmatically as well as through a Gradle dependency. It is also possible to spin on up in a Docker container. I prefer the programmatic approach because it keeps things nice and clean, and we can use the same approach throughout the development of our product. Please keep in mind that I am still referring to the server part of MockServer. The client is a different story; you will find out when it becomes relevant.
Privacy and GDPR
The port we provide is the http port on which our application will run, and it will be accessible on localhost. This is important because of two reasons. The first is practical, localhost is (almost) always reachable. The second reason is more important. We are working with phone numbers and names which are both unique enough to identify a single person. They are PII, Person Identifiable Information, where specific laws apply to. Because we are still in the phase of API definition and discussion with our API consumers, we stay away from this added complexity for now.
For now, keep in mind that we must treat the information stored in the Phonebook with special care and as long as we do not have that care in place, we have to use made up information that is not retraceable to an actual person but ourselves.
The code snippet shows that the information returned is defined as strings, formatted as json. It is the easiest way to create the response payload.
The payload is either a single contact or a list of contacts. We will use these definitions as the body of the responses generated by our MockServer instance.
Here you see the creation of what MockServer calls expectations. These are the responses you can expect when sending specific requests.
It is quite self-explanatory. When an http
GET request is sent to localhost on port 9090, as defined when the MockServer was created, and the request path is /contacts then a response is created that has http-status
200 (OK). There will be a header in the response indicating a response of type json, and the string
bodyContacts is the body of the response.
Similarly, a GET on endpoint /contacts/1 will create the response with a json representation of the contact with id 1. For contacts 2 and 3 the equivalent expected responses are defined.
Note that the last expectation is using a wildcard /contacts/.* meaning that every value after the base-URL will match with this expectation and an error will be the response. Since the request is to get a specific Contact based on the id, the response will have the http-status
The way MockServer works is that it matches requests with expectations in the order in which they were defined. Therefore the error expectation is defined as the final expectation. Otherwise it would match also all requests for which there is a Contact.
This annotation is not doing a lot. It starts-up the Spring Boot application following the processes around Spring applications. We could leave it out and just start the main application as if it were a standard Java application. Since I will be developing a web application based on Spring Boot, I want to start using Spring Boot as soon as possible, without adding complexity I do not yet need and therefore cannot justify. Adding a simple annotation is doing exactly that, allowing the use of Spring Boot without adding complexity.
Spring Boot is very well supported in the Gradle ecosystem. Just like it is very well supported in the Maven ecosystem in case you want to use Maven instead of Gradle.
The snippet shows the relevant Spring Boot lines in the build.gradle file. We add support for it in our Gradle environment through the plugin definitions, and dependencies to be used when building our application. By adding this particular dependency, we actually get a complete setup for web-development, based on the Spring framework. It is possible to add all dependencies by hand, but why complicate things when you can keep them simple.
At this point, we can build the application using
gradlew bootRun which will compile and start our application. The task
bootRun is added to Gradle by adding the plugins for Spring Boot. Running this task will start the application and configure the webserver as needed.
We are cheating though because the Spring Boot application is not defined yet. The
bootRun will start the Application and defer control to the Spring application class, provided it is defined in our application’s
main() function. Which it is not. Still, the
main() function is called and our MockServer is started.
You can now use your browser and go to
http://localhost:9090/contacts and you will get:
The full source code for this version of the application can be found here, in its own branch.
When you run
gradlew build on this version of the application you will notice that there are no tests executed. Further investigation shows that the project no longer has tests defined. This is on purpose, as at this point, we are still in the process of discussing with our users what the most convenient API is. Once we agree on the API, i.e., which requests are most convenient from the API consumer’s perspective and which responses are best matching what the consumer expects as a result, we will create the tests and switch to TDD and BDD practices.
At this point we get the following json representation of a contact:
And a discussion with the user will very likely result in the decision to make a small but important change. This change relates to the phone number, which is currently a single number. Phone numbers consist of four segments:
2. Area code
3. Subscriber number
4. International dial code
The fourth segment is globally, per convention, the character `+`, as different countries can define different codes to dial an international number. We therefore chose to omit the fourth segment, of course after coming to an agreement with the user of the API.
Now the json representation of the same contact will look like:
A convenient representation of a contact as agreed upon. This will be the contract of our API. When retrieving a single contact, it will be represented as shown in the snippet. When retrieving multiple contacts, these will be a list of the same representation. Furthermore, when a non-existing contact is requested, the http-status is
404 (NOT_FOUND) and a human readable error-message is part of the response body.
Contracts can be a lot more sophisticated. We probably need that in the future, but for now, this is sufficient. It is more important to start using the API then to start overthinking it.
This is also the right time to start ensuring that whatever we do, we respect the contract. Because we already have the API implemented with the MockServer implementation, we have a head-start. We will work on creating the scenarios of typical use-cases using our mock-implementation. Once we have our scenarios in place, we will develop the ‘real’ implementation of our API. This will be discussed in the second part of this extra-long second episode of season two.
Please note that the Github repository of this first part of episode two has its own branch. When cloning this branch, you will notice that the phone numbers are still single numbers instead of the segmented representation. The second part will start off with the segmented version of the phone number representations.
The text very explicitly communicates my own personal views, experiences and practices. Any similarities with the views, experiences and practices of any of my previous or current clients, customers or employers are strictly coincidental. This post is therefore my own, and I am the sole author of it and am the sole copyright holder of it.
Special thanks to my lovely wife Olcay, as well as my friend Sytse who took the time and made the effort to review my article. I am confident that the article’s quality was significantly improved by their feedback.