Test Management is Wrong

Test Management is wrong. There. I said it.

I can't believe it took me this long to notice the obvious. If you are doing Software Development in any fashion, and are worried about how to manage your testing to develop a "quality" product, stop it.

Let's be clear about what I mean here. If you consider any software development life cycle (SDLC) process, you will find activities like the following arranged in some fashion or other:

• Requirements gathering, specification
• Software design
• Implementation and Integration
• Testing (or Validation)
• Deployment
• Lather, Rinse, Repeat (i.e. Maintain, Enhance, Fix, Mangle, Spin, and so on)

These activities aren't Waterfall or Agile or anything else, they are just activities. HOW you choose to do them will reflect your SDLC. I don't care about that right now. The part I'm picking on is the Testing bit near the middle, regardless of whether you do them in an agile, Waterfall, or some other way.

In particular, I am picking on the fallacy or myth that a good Test Management plan/process is what you need to develop and release a high Quality product.

In almost 25 years of working in the Software/IT industry, I have never worked at a company or heard of any company that had a Test Management process so solid that it lead to high quality, I don't believe there ever will be either.

Here's an analogy. Let's say I want to make some Rice Krispies treats. The ingredients are: butter, marshmallows and rice krispies. You heat 'em up, mix 'em up, flatten 'em out, cool, slice and eat. That's it.

Ya, but you know what? I have this really excellent Marshmallow Plan that will produce the Bestest treats anyone has ever tasted.

How does that sound to you? Interested to learn more about the plan? Be truthful.

If you said "Yes", I don't know if I can help you. If you said "No" there may be hope for you yet.

Why is it that Software companies are attracted to the idea that Testing is somehow equated with Quality? Testing is an activity that you may choose to do or not do and still have a great quality product. The Agile Manifesto doesn't mention testing anywhere and yet the Manifesto's signatories, thought leaders, and practitioners produce great quality stuff for people. How does that work?

Why is it only the Testing phase/activities/part that equates with Quality? Why aren't companies and managers everywhere promoting super awesome Requirements Management processes and plans at Requirements Conferences to help deliver high Quality? What about Design Management? No, no, wait, I got it. We need Deployment Management. That's it! I've solved it. Aha!

No, these are all stupid suggestions. And you know it too.

Is it because Testing/Verification activities are part of the other 'phases' or steps in software development, so surely we should be able to manage all that, right? Well, actually, no. That's the wrong approach.

Test Management systems manage and measure some of the testing activities performed on a project. Completing your testing activities tells me nothing about the overall "Quality" of the product. By definition, it is an incomplete part of the picture.

By putting your faith in Test Management plans or systems you are effectively saying "I don't know how to measure what you want (i.e. Quality), so I will measure what I can do (i.e. Test)."

Does this mean it's pointless to track the testing you've done? NO! I am not saying that. If you do something that you believe contributes to the value of the project, then please track what you are doing so that others can see what you have done. Preferably, make your progress visible in some way.

What we need to focus on is what the customer needs. Ask yourself: What problem are they trying to solve? What are we trying to deliver that is of value to them?

What we need is Value Management. There are successful (i.e. "quality") products built and released to customers that never had any independent testers on the development team. Cool. These teams get it.

I have a friend and colleague who started up a company and within hours had sold a product that didn't even have a single line of code written. Way cool! He gets it. Did he worry about Test Management? Bah! He didn't even need to pay a programmer to sell something of value to a customer.

I have mentioned it before, if you are doing any kind of testing on a software development project, you can likely place it somewhere within the Agile Testing Quadrants. Heck, you don't even have to be on an Agile team or project to see that your testing fits somewhere on the chart.

So, if you somehow create a super Test Management Plan or system that tracks all of these activities for a given project, then would you have a Quality product? That would be really cool, by the way, but no, not necessarily. And don't fool yourself into thinking that it will.

The problem is one of relationships. Human relationships.

The definition of Quality I like to use comes from Jerry Weinberg and it is based on your understanding of relationships between people. The definition of "software" that I use when teaching/training/coaching is that software reflects your understanding of the relationships between people and the systems they interact with to meet particular needs. The definition of Testing that I like to work with is the intelligent use of models and heuristics to explore the relationships between the people designing a system and those who intend to use it.

In short, if you want to understand what provides "quality" or value to your customers, look to how you are managing the human relationships. Forget Test Management. You might have better luck implementing a CRM system in your development departments as a predictor of Quality.

Testers: if you want to do a great job, focus on the testing activities that explore the systems from the perspectives of the people who matter. Never hide or bury your work (i.e. in documents, spreadsheets, test management systems, etc.). Make it visible - use dashboards, mind maps or other visual mediums because they assist with team collaboration and understanding. Keep detailed records archived somewhere if you need them, but don't worry about "managing" to those fiddly bits as an indicator of "quality" - because they're not.

Forget about the Test Plans though. Disregard Test Management systems. AVOID providing any metrics on testing coverage as an indicator of Quality... especially if your development team isn't producing equally misleading metrics about code complexity, requirements reviews, and other esoteric development activities.

Anyone (ALM Vendors take note) who sells a "Quality Management" system based upon managing test cases is lying to themselves and to you. They are doing it wrong. Don't be a victim.

You can't kludge a development process enough to make a Quality or Test Management system produce meaningful, valid indicators of value to your customers and stakeholders. It just doesn't make sense.

Approach your development teams from a human relationship perspective. Focus on how people collaborate and work together, and high quality products will emerge as a by-product. Manage the relationships with your customers, and deliver working software frequently to help them see what you can do and how you learn from past experiences. This isn't easy. It's definitely worth it though.

So. You want to deliver Quality? Test Management isn't the answer. Test Management tells you about a specific task's management. It's about as useful as Marshmallow Management in making Rice Krispies treats. That is, it may have a place in the big picture, but it's certainly not the right way to look at the problem.

Read More

Shuhari

When I work with teams to help them learn something new, I try to pay attention to a few things. Firstly, I pay attention to how people are learning, and secondly how I am teaching.

When I used to teach Physics and Chemistry in high school, one validation of 'success' often came from how the students left the classroom. Generally, teenagers often came into one of those classes the same way (at least at the start of the year): I don't want to be here, this isn't important to me, I'm not going to learn anything useful.

Okay. Gauntlet down. Let's begin.

I knew I had a good class when students left the room smiling and/or talking about the ideas covered in the lesson. The real learning happens when they talk about the subjects among themselves. We need time to absorb ideas and make them a part of us. Talking about them with others is a good first step in the learning process.

Putting ideas into practice is an important next step in the learning process. Practice makes the knowledge concrete, more permanent. Through practice we also begin to understand the limits of success under different conditions. In a classroom, practice might happen through assigned questions/exercises, experiments or projects of some sort.

Once someone groks an idea through practice, we can engage in the next level of discussion -- what next? Applicability, adaptation, extending the ideas, and so on. How can we be more successful? To paraphrase Newton, how can we stand on the shoulders of giants?

There are many learning models and ideas that apply for both learning and teaching material, and I don't mean to fill this space with them.

In recent years, my teaching (coaching, consulting) has focussed on Agile and Testing (rather than Physics and Chemistry). When I am approached for help or advice on Agile, Scrum, Exploratory Testing, or something else, I often think about the term Shuhari. (In English, pronounced: shoe - ha - ree)

Shuhari is a Japanese term from martial arts that describes the learning path to mastery. It roughly translates to "first learn, then detach, and finally transcend." From the Wikipedia page, here's the breakdown:

• shu -- "protect", "obey" - traditional wisdom - learning fundamentals, techniques, heuristics, proverbs
• ha -- "detach", "digress" - breaking with tradition - detachment from the illusions of self
• ri -- "leave", "separate" - transcendence - all moves are natural, becoming one with spirit alone without clinging to forms

Shuhari reminds me that rules and rituals are in place for beginners and that we learn to go beyond them as we mature in a particular discipline.

When I teach people about Scrum or Exploratory Testing, I often see people want to start improvising or adjusting practices right from the beginning. When you do that, you jump to "ha" but without the solid foundation or appreciation of "shu". In a martial arts class, the sensei (instructor) might smack you on the head for doing something like that. (If you're lucky.)

As we explore new ways of doing things, it's important to start at the beginning and practice the forms as described. Become comfortable with the practices. Become bored with them. Make them a part of your muscle memory so that you don't have to consciously think about them anymore. Keep practising.

*Then* one day you may ask "how about if we change this [step] a little? What do you think?" That is an excellent question. A question that drives an experiment. An experiment that drives learning and helps us to enter "ha".

Timing is the difference. Asking to vary a practice at the beginning doesn't help. Asking after you understand it makes sense.

There are different concepts and models to describe the paths to mastery. What do you think of this one?

I have three other models floating around in my head on Mastery and plan to cover them in future posts. I offered to talk about "Mastery" at the Test Coach Camp last year but there wasn't enough interest at the time. I wonder who the target audience is for this topic. People sometimes fall into careers. I actively sought mine, so these models meant something to me at the time. I reflect upon the value that each one offered and look to new insights still waiting for me.

When I think of Shuhari, I think "Practice before Change." And that reminds me of the old joke: How do you get to Carnegie Hall?

Read More