November 17, 2023
Testing is kind of like taxes. Nobody loves paying taxes, but there's a sort of minimum amount that's necessary in order to keep society functioning, or keep the government functioning, whatever they may be. The same with testing. Well, what kind of testing, if we've got to do some of it, should we focus on? And what kind of testing is highest ROI? What I've seen in my experience, okay, so if we cut up a system real quick here, and let's pretend that up here is the UI of the system, whatever that's built on, and then maybe that's calling or speaking with an API/logic layer, and then finally, we've got the underlying implementations of that logic down in here, that this stuff calls, and then under there, maybe somewhere further down, we've got a data store. So, I'll throw my crappy database diagram down there.
API logic and different types of testing can be brought against different facets. One that I'm not a huge fan of, and don't really recommend, is down at this layer in the individual units of a system. There's something called unit testing. The problem I find with unit testing is that most bugs don't actually occur in the individual subunits of a larger program. They actually occur at the integration points, at points where code written by different developers has to work together, and where greater complexity comes to the fore. So, in terms of testing this, I actually recommend testing here at the API layer.
The beauty of API layer testing is that it's easy to automate, and it checks everything that's going on in your back end as a holistic system. And if you identify an issue, then of course the developers have got to go down, drill in, and fix that. But API layer testing is very high ROI, and some of the easiest testing to roll out. It forces your developers to actually look at their APIs, and what those APIs are returning contract-wise, and so I have found that you get very good bang for your buck there.
Then, of course, up here at the UI, you have UI layer testing. This is what we call end-to-end tests. This is full automation of your system by having automated tests pretend that they are users, and they're hitting the UI and going all the way through and all the way back. The problem with these is that they tend to be harder to maintain; they break more easily. So, there's the ability to cover a lot of different functions underneath by working on end-to-end tests. The ROI is lower, though, than the API layer testing.
So, out of these three, if you could only do one, this is actually what I would pick. I would start there and then move up to end-to-end, and frankly, I think that if you're forcing your developers to do tons and tons of unit tests, in most business cases, in most industries, you're wasting their time.
Business Principles
Startup Costs Video Series
Pursuit of Profits
