THE LITTLE BLACK BOOK ON TEST DESIGN BY RIKARD EDGREN
This little book describes ways to take advantage of the multiple information sources needed for ambitious system testing.
I started investigating it deeper after I for the Xth time felt that the existing test design techniques didn’t capture the way I was working after 10 years with the same product suite. This resulted in an expanded generalization of ideas I primarily have learnt together with peers1.
I will describe a holistic merge of test strategy, test analysis, test design and test execution; activities that can be split theoretically, but are intertwined in reality. It is not about details of individual test cases; test design can also be one-liners, charters, or a mental model in your head. It is about handling complexity, developing skill, learning as you go.
I want to emphasize the results of the testing; a broad sampling space, that enables serendipity2 and aims for true
saturation3 for important areas. It is probably best suited for manual system testers that focus on any problems that are important, for people that want to find out more qualitative information than pass/fail on bare requirements. It is about test design methods that I, and many others4, have been using for a long time.
Learn from multiple sources, generate relevant test elements, synthesize testworthy test ideas, and execute with serendipity.
Software testers can follow the thoughts of social science Grounded Theory; we should use many sources of information regarding the subject (requirements, specifications, prototypes, code, bugs, error catalogs, support information, customer stories, user expectations, technology, tools, models, systems, quality objectives, quality attributes, testing techniques, test ideas, conversations with people, risks, possibilities); we should group things together, use our creativity, and create a theory consisting of a number of ideas capturing what seems important to test, together with the results.
I encourage testers to use a lot more than the requirements; to understand the need to combine things of different types, to look at the whole and the details at the same time. This piece of writing is for ambitious projects, probably spanning over several releases, and not worthwhile when resources are sparse, or when you are absolutely certain that the techniques you use are the best.
This might seem heavy, but people have phenomenal capacity.
Software testing inspiration
identify testworthy elements
synthesize test ideas
serendipitous test execution
These activities depend on the skill of the tester, a skill that is developed over time, at best with a lot of knowledge of the product and project context, and speeded by generating and sharing your own rules of thumb, your specific test design heuristics.
I see a loose scientific, theoretical base in Grounded Theory5 (Corbin/Strauss), used in social science as a qualitative, thorough analysis of a phenomenon.
The scientists examine the subject carefully, gather a lot of material, document codes and concepts in memos, combine them to categories, from which a theory is constructed.
There is no hypothesis to start with, as in the traditional natural science, and that’s quite similar to my software testing; it would be dangerous if we thought we knew all important questions at once. They acknowledge that we are dealing with a sampling problem, and that serendipity is a factor to your advantage.