Mocks vs Stubs - Importance in Testing

In testing, Mocks and Stubs are really two very important conceptual confusions and there are camps on both sides. Which one to use and when? IMHO, when one is building testing infrastructure, there are two concerns in that, which needs to be taken care of.

• Test behavior
• Testing artifacts

The question of mocks or stubs relate to the testing artifacts concern and nothing to do with the Test behavior. If I draw an analogy, than let's say we want to make a software based on certain requirements. And than I have to choose a language/platform to build that software. The behaviour of my software is independent of which platform I choose (Java,.Net, PHP). And the choice of a language is not going to help me a lot in lessening my effort towards thinking process that I have to put to understand the behaviour of my software. The choice of technology will have an impact on performance, maintainability, cost,actual implementation but the behavior is not going to be impacted by that.

Let's take a concrete case to understand it better. Let's say we have a class which we want to test. One of the method that we want to test is responsible for saying whether a user, when passed to it is credit worthiness or not. The user should have a pending amount of less than 2500, to remain credit worthiness. In real application, the data of user is fetched from the database. However in testing scenario, putting a real database and than populating with right set of records is in itself a task. Our testing will be slowed down a lot. Note that in this para, we are still not concerned about stubs or mocks. We are purely concerned about the test behavior. This is a harder skill to achieve and this is the more important skill to achieve. Once you have decided your testing behavior, than comes the question of using Mocks or Stubs.

I would not show here how to write stubs or mocks as there are plenty of things on it. Regarding the choice of stub or mock, the yardstick is that if the stubbed (or mocked) object do not have a lot of different variation, go for stub otherwise go for mock. Ask the question of variation at a project level. It's better to use one strategy for the whole project, the mock way or the stub way. If my user is supposed to return the same pending amount of 2500 or smaller number of variation to it, for checking the various behavior in the system stub is a good choice. However if the user object has to return a whole lot of different values in different scenarios, mock would be better. It' what you want to achieve is more important than how you want to achieve.

Bugs that should not be fixed

Of course,  we want to fix everything. It's extremely difficult to justify sometimes why we should not be fixing some bugs. There is always an angle to more security, better usability, better look and feel, a new feature which could be a killer. The points I am mentioning are more valid near the release and hold less water in the beginning of release so take your judgement.

• Usability is a never ending game. Understand your users. If they use application after going through training, then don't kill yourselves for usability. If you are making an application where

Test Driven Development

Test driven development is the new mantra of modern day development. With the modern day complex software, it has become a necessity rather than a choice. It's not that the software in earlier days were less complex but many new complex dimensions has been introduced in the modern day development.

People have different notion of Test driven development. On one extreme is the philosophy of writing of test cases before writing the code. Write a test which even does not compiles and than write the actual program so that at the end of the day, the test starts passing. On the other extreme is of course not to write any test case assuming that one is a star programmer and one can never go wrong. (Lot of developers carry this halo effect behind their head and unfortunately they only see the reflection of that halo in mirror). When the code is done than throw it across the wall to a battery of testers who become responsible for finding the flaws in the code. My personal experience has been always of striking the right balance between the two extremes. This theory of balance is very effective. Maybe a Libran inclination. Humans have a tendency to exaggerate as in Tulipomania.

So what is a good definition of Test driven development. I would say it's about having a good set of Test cases at the end of the development cycle of a phase,iteration or unit of work. It's like ACID property , where in the development process you may or may not have test cases, the logic is all in fluid state but when you are ready to complete the transaction of that phase, you should have a working set of code with adequate testing suite.

For me following are the good practices of a good Test driven development approach:

• Measure, Measure, Measure. It's my favourite and it's applicable at every facet of life. Unless one cannot measure one cannot understand the implication of action. For TDD, it's also an important thing to measure the code coverage that is how many coverage you are providing through your test case suite.
• Write test cases which are atomic in nature.In simple terms it means if one bug has been introduced in the program, it should result in exactly one test case failing. If one refactoring has been done, only one test case should fail, no more no less. In practice, this is very hard to achieve.
• Have less number of test cases. I can see you ready to jump on me. It's counter intuitive But I would strive for less number of test cases and near 100% coverage in terms of line and conditional coverage. There is no point in writing 100 of test cases which are doing the same thing. Management loves the test case matrices and more the number, better it is. But is there a point to write 50 test cases for a program which measure the difference between two time periods, even if they are like million years apart. Strive for lower foot print of code both in main logic and test cases. The lesser the code, the lesser the chances of bugs. A good coverage with least amount of test cases for time difference could be
  • Between two days, the days fall before 28th of the month.
  • Between two days, one day falls on 31st of month. Two test cases, one for starting at 31st and another for ending at 31st.
  • Between two days, one day falls on 29th of Feb. Two test cases, one for starting at 29th Feb and another ending at 29th.