Salary Thieves

Yesterday I attended a Waterloo Agile Lean session on Story Mapping presented by Mark Levison. I had heard of Story Mapping but hadn't worked through it before. I liked the hands-on exercises approach Mark took to help us understand the process and benefits of that technique. This blog post is not about Story Mapping.

I love learning and take any opportunity I can to hear different speakers present on topics that I think may be of value. Sometimes I even attend the same talks and workshops when they are done by different speakers so that I can understand differences of style in presentation, stories, and tips/techniques for getting the ideas across to the audience/participants. Once I even attended the same topic by the same speaker 2 days in a row, and I learned something new/different the second time around! (It was a Cem Kaner talk when he was in our area a few years ago.)

Yes, I gained an appreciation of Story Mapping yesterday. More importantly, I learned a new anecdote from the speaker - one that got me thinking. At one point, Mark answered a question about managing a large backlog on a Scrum/Kanban-style (information radiator) board. The problem with really large backlogs of stuff to do (likely anything with more than 100 items in it), is that the items near the bottom become meaningless over time. You will probably never get to them because something more important always comes up.

Mark told us a quick story about Taiichi Ohno, the father of Lean Manufacturing. I looked up the story and found a reference to it here. In 1984, Ohno was interviewed about the history of development of the Toyota Production System (which later became Lean Manufacturing). At one point in the interview that discussed work standards and the early stages of doing kaizen (continuous improvements), Ohno said:

I tasked the shop floor leaders with regular kaizen of work methods and revision of the standard work, telling them "If the kanbans do not change for one month you are salary thieves."

During Mark's workshop yesterday, he mentioned the "yellowing of paper" [on the backlogs] as an indicator of "salary thieves." He was tying this point back to the question: how long has an item been sitting on your backlog? Maybe they are salary thieves.

This is an interesting point. When I work with Testing teams, I sometimes hear the right words but see the wrong practices.

For example, I often hear testers talking about Risks. That's good. They do a risk assessment and create tests for those risks but never revisit the risks. That's bad. Over time, those original risks become salary thieves. Are they still relevant? Have things changed? Are you wasting your time to put effort into managing the testing of things that are so out of date that not even the developers/programmers use them to create the code you are now testing?

What about test cases? And Regression Testing? How many of those tests check risks and ideas that are no longer relevant? How many of them are salary thieves, taking away our precious time that we could have spent finding more relevant bugs and identifying new risks to stakeholder value?

What about the tools that we use to support and manage our work? How many of them have sunset clauses tied to the creation dates of artefacts? For example, automatically identifying requirements in a backlog, test cases in a repository, or personas in a reference area that are "old" (by some definition) and therefore potential risks of losing value.

Right. I don't know any software development tools that do this right now. It seems like a perfectly reasonable feature to look for though.

When we think about the "dinky" Scrum/Kanban board with pieces of paper identifying bits of work, we realise that ageing has a very real effect on paper. We see the paper get old over time, the ink start to fade. If we have to rewrite a card or sticky note, we should be asking: what is the value of keeping it on the board at all?

With electronic tools, things are a bit different. Electrons, unfortunately, don't show signs of wear over time. Unless we specifically ask a computer to keep track of information like this, we are likely to forget about it because of all the other things we have to do and keep track of.

Physical board: 1, Electronic tools: 0.

So. What are your salary thieves?

What are some ideas you have for avoiding them?

Read More

Test Management is Wrong

Test Management is wrong. There. I said it.

I can't believe it took me this long to notice the obvious. If you are doing Software Development in any fashion, and are worried about how to manage your testing to develop a "quality" product, stop it.

Let's be clear about what I mean here. If you consider any software development life cycle (SDLC) process, you will find activities like the following arranged in some fashion or other:

• Requirements gathering, specification
• Software design
• Implementation and Integration
• Testing (or Validation)
• Deployment
• Lather, Rinse, Repeat (i.e. Maintain, Enhance, Fix, Mangle, Spin, and so on)

These activities aren't Waterfall or Agile or anything else, they are just activities. HOW you choose to do them will reflect your SDLC. I don't care about that right now. The part I'm picking on is the Testing bit near the middle, regardless of whether you do them in an agile, Waterfall, or some other way.

In particular, I am picking on the fallacy or myth that a good Test Management plan/process is what you need to develop and release a high Quality product.

In almost 25 years of working in the Software/IT industry, I have never worked at a company or heard of any company that had a Test Management process so solid that it lead to high quality, I don't believe there ever will be either.

Here's an analogy. Let's say I want to make some Rice Krispies treats. The ingredients are: butter, marshmallows and rice krispies. You heat 'em up, mix 'em up, flatten 'em out, cool, slice and eat. That's it.

Ya, but you know what? I have this really excellent Marshmallow Plan that will produce the Bestest treats anyone has ever tasted.

How does that sound to you? Interested to learn more about the plan? Be truthful.

If you said "Yes", I don't know if I can help you. If you said "No" there may be hope for you yet.

Why is it that Software companies are attracted to the idea that Testing is somehow equated with Quality? Testing is an activity that you may choose to do or not do and still have a great quality product. The Agile Manifesto doesn't mention testing anywhere and yet the Manifesto's signatories, thought leaders, and practitioners produce great quality stuff for people. How does that work?

Why is it only the Testing phase/activities/part that equates with Quality? Why aren't companies and managers everywhere promoting super awesome Requirements Management processes and plans at Requirements Conferences to help deliver high Quality? What about Design Management? No, no, wait, I got it. We need Deployment Management. That's it! I've solved it. Aha!

No, these are all stupid suggestions. And you know it too.

Is it because Testing/Verification activities are part of the other 'phases' or steps in software development, so surely we should be able to manage all that, right? Well, actually, no. That's the wrong approach.

Test Management systems manage and measure some of the testing activities performed on a project. Completing your testing activities tells me nothing about the overall "Quality" of the product. By definition, it is an incomplete part of the picture.

By putting your faith in Test Management plans or systems you are effectively saying "I don't know how to measure what you want (i.e. Quality), so I will measure what I can do (i.e. Test)."

Does this mean it's pointless to track the testing you've done? NO! I am not saying that. If you do something that you believe contributes to the value of the project, then please track what you are doing so that others can see what you have done. Preferably, make your progress visible in some way.

What we need to focus on is what the customer needs. Ask yourself: What problem are they trying to solve? What are we trying to deliver that is of value to them?

What we need is Value Management. There are successful (i.e. "quality") products built and released to customers that never had any independent testers on the development team. Cool. These teams get it.

I have a friend and colleague who started up a company and within hours had sold a product that didn't even have a single line of code written. Way cool! He gets it. Did he worry about Test Management? Bah! He didn't even need to pay a programmer to sell something of value to a customer.

I have mentioned it before, if you are doing any kind of testing on a software development project, you can likely place it somewhere within the Agile Testing Quadrants. Heck, you don't even have to be on an Agile team or project to see that your testing fits somewhere on the chart.

So, if you somehow create a super Test Management Plan or system that tracks all of these activities for a given project, then would you have a Quality product? That would be really cool, by the way, but no, not necessarily. And don't fool yourself into thinking that it will.

The problem is one of relationships. Human relationships.

The definition of Quality I like to use comes from Jerry Weinberg and it is based on your understanding of relationships between people. The definition of "software" that I use when teaching/training/coaching is that software reflects your understanding of the relationships between people and the systems they interact with to meet particular needs. The definition of Testing that I like to work with is the intelligent use of models and heuristics to explore the relationships between the people designing a system and those who intend to use it.

In short, if you want to understand what provides "quality" or value to your customers, look to how you are managing the human relationships. Forget Test Management. You might have better luck implementing a CRM system in your development departments as a predictor of Quality.

Testers: if you want to do a great job, focus on the testing activities that explore the systems from the perspectives of the people who matter. Never hide or bury your work (i.e. in documents, spreadsheets, test management systems, etc.). Make it visible - use dashboards, mind maps or other visual mediums because they assist with team collaboration and understanding. Keep detailed records archived somewhere if you need them, but don't worry about "managing" to those fiddly bits as an indicator of "quality" - because they're not.

Forget about the Test Plans though. Disregard Test Management systems. AVOID providing any metrics on testing coverage as an indicator of Quality... especially if your development team isn't producing equally misleading metrics about code complexity, requirements reviews, and other esoteric development activities.

Anyone (ALM Vendors take note) who sells a "Quality Management" system based upon managing test cases is lying to themselves and to you. They are doing it wrong. Don't be a victim.

You can't kludge a development process enough to make a Quality or Test Management system produce meaningful, valid indicators of value to your customers and stakeholders. It just doesn't make sense.

Approach your development teams from a human relationship perspective. Focus on how people collaborate and work together, and high quality products will emerge as a by-product. Manage the relationships with your customers, and deliver working software frequently to help them see what you can do and how you learn from past experiences. This isn't easy. It's definitely worth it though.

So. You want to deliver Quality? Test Management isn't the answer. Test Management tells you about a specific task's management. It's about as useful as Marshmallow Management in making Rice Krispies treats. That is, it may have a place in the big picture, but it's certainly not the right way to look at the problem.

Read More

Shuhari

When I work with teams to help them learn something new, I try to pay attention to a few things. Firstly, I pay attention to how people are learning, and secondly how I am teaching.

When I used to teach Physics and Chemistry in high school, one validation of 'success' often came from how the students left the classroom. Generally, teenagers often came into one of those classes the same way (at least at the start of the year): I don't want to be here, this isn't important to me, I'm not going to learn anything useful.

Okay. Gauntlet down. Let's begin.

I knew I had a good class when students left the room smiling and/or talking about the ideas covered in the lesson. The real learning happens when they talk about the subjects among themselves. We need time to absorb ideas and make them a part of us. Talking about them with others is a good first step in the learning process.

Putting ideas into practice is an important next step in the learning process. Practice makes the knowledge concrete, more permanent. Through practice we also begin to understand the limits of success under different conditions. In a classroom, practice might happen through assigned questions/exercises, experiments or projects of some sort.

Once someone groks an idea through practice, we can engage in the next level of discussion -- what next? Applicability, adaptation, extending the ideas, and so on. How can we be more successful? To paraphrase Newton, how can we stand on the shoulders of giants?

There are many learning models and ideas that apply for both learning and teaching material, and I don't mean to fill this space with them.

In recent years, my teaching (coaching, consulting) has focussed on Agile and Testing (rather than Physics and Chemistry). When I am approached for help or advice on Agile, Scrum, Exploratory Testing, or something else, I often think about the term Shuhari. (In English, pronounced: shoe - ha - ree)

Shuhari is a Japanese term from martial arts that describes the learning path to mastery. It roughly translates to "first learn, then detach, and finally transcend." From the Wikipedia page, here's the breakdown:

• shu -- "protect", "obey" - traditional wisdom - learning fundamentals, techniques, heuristics, proverbs
• ha -- "detach", "digress" - breaking with tradition - detachment from the illusions of self
• ri -- "leave", "separate" - transcendence - all moves are natural, becoming one with spirit alone without clinging to forms

Shuhari reminds me that rules and rituals are in place for beginners and that we learn to go beyond them as we mature in a particular discipline.

When I teach people about Scrum or Exploratory Testing, I often see people want to start improvising or adjusting practices right from the beginning. When you do that, you jump to "ha" but without the solid foundation or appreciation of "shu". In a martial arts class, the sensei (instructor) might smack you on the head for doing something like that. (If you're lucky.)

As we explore new ways of doing things, it's important to start at the beginning and practice the forms as described. Become comfortable with the practices. Become bored with them. Make them a part of your muscle memory so that you don't have to consciously think about them anymore. Keep practising.

*Then* one day you may ask "how about if we change this [step] a little? What do you think?" That is an excellent question. A question that drives an experiment. An experiment that drives learning and helps us to enter "ha".

Timing is the difference. Asking to vary a practice at the beginning doesn't help. Asking after you understand it makes sense.

There are different concepts and models to describe the paths to mastery. What do you think of this one?

I have three other models floating around in my head on Mastery and plan to cover them in future posts. I offered to talk about "Mastery" at the Test Coach Camp last year but there wasn't enough interest at the time. I wonder who the target audience is for this topic. People sometimes fall into careers. I actively sought mine, so these models meant something to me at the time. I reflect upon the value that each one offered and look to new insights still waiting for me.

When I think of Shuhari, I think "Practice before Change." And that reminds me of the old joke: How do you get to Carnegie Hall?

Read More

The Human Side of Living

As I go through life I keep noticing stories, ideas and insights into humanity and I sometimes wonder if we are meant to discover these lessons slowly or if there isn't a quicker way to learn them.

Take for example, in high school we had a really weird Religion teacher who was very Zen or meta or something, and no one got him. I mean he would use examples like "take an extension cord and plug it into itself and there you go." Huh? None of us got it. And then there would be times when he would repeatedly say things like "attack the point not the person" and that was a phrase I understood.

From him, I learned that sometimes we can meet real jerks that we can learn interesting things from. Learn to separate your feelings about what you hear and understand from the messenger. It's hard sometimes, but you can get good at this.


Later, I read a story about a samurai warrior. It's short, so here it is:

There was once a samurai who wanted to learn the difference between heaven and hell. He sought until he found a master from whom he thought he could learn. He stood before the Master and asked him what was the difference between heaven and hell. The Master took the samurai’s sword and, turning it to the flat of the blade, struck the samurai on the head. The samurai was surprised at this but chose to ignore it. He thought that the Master had failed to understand his question. He once again asked the Master about the difference between heaven and hell. Again the Master struck the samurai on the head. The samurai staggered back and puzzled over this. He approached with his question for a third time and, before he could utter a word, the Master struck him a third time. The samurai was now so enraged at this behaviour that he grabbed his sword from the Master, raised it over his head and was prepared to bring it down on the Master’s head when the Master raised one finger and the samurai paused.
"That is hell," said the Master.
The samurai was instantly so overcome by the courage of this frail old man - to have risked his life for the sake of a stranger’s question - that he fell to his knees and bowed before the Master.
"That is heaven," said the Master.

This story keeps popping into my head every now and then. What is good and evil? Is it a matter of perspective? Is it a matter of time? How is it related to compassion? Do we need to judge people/situations, or should we learn to see the good and bad in all things? We can choose what we want to make from a situation. We don't always understand the motives of others, so which stance do you initially take - heaven or hell?

On the compassion thread, I learned about HH the (14th) Dalai Lama (of Tibet) when I was in university. It wasn't part of a course, I don't remember what it was. It might have been a movie. He's a really interesting guy and has done some cool things. I pondered his thoughts on compassion and felt that he really has good insights into the human condition so some of those ideas stuck with me. (HHDL is on Twitter by the way.)

When I left school and started working full time, I discovered Jerry Weinberg. Jerry published many technical books up to that point, and started the Amplifying Your Effectiveness (AYE) conference and Problem-Solving Leadership (PSL) workshop. The workshop and conference are based upon applying the work of Virginia Satir, a family therapist, to the workplace. I find some of the models very insightful.

Skipping over many other little opportunities and lessons, I find myself thinking about a recently-published book called "The Human Side of Agile" by a colleague Gil Broza. It's a good book. I like it. It sums up a lot of lessons I learned over the years, and includes new ones I didn't know about. The title really sticks with me though.

When I am at work, I focus on doing things to help others. Help the customers get high quality software of value. Help the team members to learn, grow and become more confident in their abilities. I show patience and temper difficult situations with humour. That's my style. When the going get's tough, I get silly. Sometimes, though, I hear Jerry W's words ringing in my ears "Change your organisation or change your organisation."

I am an agent of change. I am here to help you establish a new norm, a new status quo, one that is better than you were before. I work with people to help them adapt into their new roles, and I often come across people who neither want my help nor anything to do with change.

I can understand when people are afraid of uncertainty or the unknown and I am patient enough to work with them to try and build congruence (Satir) and focus on the point not the person (high school teacher). Then there are times when certain people can very intentionally do malicious things to undermine and attack you through a show of power or superiority. I'm too old for this crap.

From Jerry, I know it is time to change my organisation when this becomes a pattern, because it is my life and I choose how I want to live and enjoy it. I don't want to be miserable at work and then bring that negativity home with me to my family that I love so much.

It's my life. I want to be happy and helping others makes me happy. I'm weird that way. I understand that not everyone gets that. I'm not here to inflict compassion and other zen mumbo-jumbo on you. I really like and appreciate the Lean and Agile values. The focus is on *people* working together to make great things that make your customers happy.

After 25 years of working in the IT sector, I can tell you that I agree with Jerry when he said that "all problems are people problems." (especially the technical ones.) When I truly came to understand that, I discovered that people are at the heart of the answer to "what is Quality?" After almost 20 years in Testing, I also discovered that test techniques are really models to test the interactions between people working on the projects. This is a bit of unique perspective and I haven't heard anyone else describe it that way, but that's how I see it and teach it.

People working with people to make other people happy. Lots of other people actually. There's nothing non-human about software development. It's all about the human side of things. And yet. Schools don't teach this. Some people choose to act in inhuman ways. How do you deal with that? Heaven or Hell?

It's your life, your choice. Change your organisation or change your organisation.

Thank you to all my teachers, past, present and future. There is still more for me to learn.

Read More

Sharing thoughts

I've been asked a few times over the past few months why I haven't blogged in a while. Funny that. I have been writing more in this past year than I think I have my whole life. It turns out that the blog posts have fallen off the list of thought-sharing media for a short while. So here's a brief note to let you know why.

About a year ago, I started keeping a daily journal as a consultant. I got the idea spark from my friend Pradeep in India when he posted an annual report a few years ago that listed a brief note for each day that year. I looked upon that report with awe and thought that might be a cool way for me to track where some of my time and days go.

The daily journal is for personal reflection and I don't share it with anyone. It's pretty boring really. Disparate facts and ideas mostly. The challenge for me was in re-developing the habit of writing a little something every day.

When I was in high school I used to write every day. Again, I never shared those thoughts with anyone. But unlike my current journal, those entries were chock full of entertaining and emotional snippets of teenage life. I suppose if I hadn't lost those journals/books, I might have enough material to create a few teen book series. ;-) As an aside, several of my high school teachers thought I would go into English and continue writing in university. There was more than one surprised look when I said I wanted to go into Science.

I stopped writing personal journal entries in University and I discovered the internets - Usenet in particular. I started writing socially on topics of interest in the alt.* user forums. This was the early 90's, so the internet was still young then. The idea of Usenet forums wasn't new to me. In fact, I was attracted to them because they reminded me of the BBS forums I used to participate on during the 80's. (Ah, stories for another day.)

After graduating and joining the workforce full-time, I discovered and joined some email discussion forums in the late 90's. Around 2000, I dived into some Yahoo groups followed by some Google groups a few years later.

Around that time I started up this Blogger account to keep track of some thoughts as I began to take an active interest in my career in Software Testing. My blog posts were infrequent as I had lost the habit of writing daily. So many things go on in my life and the work-focus of this particular blog has kept me from writing more. Now if it were a general blog on random thoughts, I would likely have written so much more.

Several years ago I jumped onto Twitter. Interesting medium that.

I find Twitter is a good medium for me personally and professionally. While I remain on a few email discussion lists, I find I don't participate in them as much as I used to. I do however read and write/share thoughts via Twitter daily.

I added a gadget/widget thingy to the blogger layout here to show you my last few tweets. Twitter changed their API last year sometime so my tweets stopped appearing here. It's taken me too long to return to this blog site and fix the HTML code to redisplay my tweets on the side panel.

I spent about an hour tonight searching, scripting and playing with code to make those tweets appear again. I felt it was important to help the casual reader of this public blog understand that while I may share the occasional long thought on this site, I am micro-blogging on Twitter almost daily.

Several months ago, I also started to capture more thoughts and fieldstones for a book I plan to write on Testing. I haven't made it public yet, and when I do, I will announce it here and on Twitter.

So, I am currently writing a daily journal, capturing thoughts for a book in progress, tweeting daily, still on a few email discussion lists, participating in half a dozen conferences each year, and still trying to find time to blog every now and then. I have a huge backlog of ideas to share here. I promise to write no less than monthly here in 2013. I will share more ideas than I have in the past.

I have also been asked if I have an email mailing list. The short answer is no, not at this time. If I add that to the writing list, something else will likely have to come off. I am open to the suggestion though. Maybe later.

Read More

What is Exploratory Testing?

What is Exploratory Testing (ET)? I am asked this every once in a while and I hear a wide range of ideas as to what it is. This is one of those topics where Wikipedia doesn't really help much.

For some, ET is just "good" testing and the reason we say "exploratory" is to distinguish it from bad testing practices. Unfortunately, bad, lazy, haphazard, thoughtless, incomplete, and incompetent testing is quite popular. I won't go into the reasons or supporting evidence for this disgraceful blight on the Software Development industry at this time. Suffice it to say, I don't want to be mixed in with that lot either, so I am happy to describe what I do as something different - something that is far more successful and rewarding when done well.

Okay, so if ET = [good] testing, what is testing then? According to Cem Kaner, "software testing is a technical investigation conducted to provide stakeholders with information about the quality of the product or service under test." This definition took me a while to absorb but the more I thought about it the more I found it to be a pretty good definition.

If you ask Elisabeth Hendrickson, she would say that "a test is an experiment designed to reveal information or answer a specific question about the software or system." See, now I really like this definition! I studied Science in university and I love the way this definition reminds me of the Scientific Method. The more I learn about testing software, the more I find similarities with doing good Science. (By the way, if you want to learn more about how to do good testing, I highly recommend you read up on the Scientific Method. So much goodness in there!)

So, is that all there is to it? Testing = Science, blah blah blah, and we're done? Um, well, no, not really. ET has its own Wikipedia page after all!


I dislike the first line description of ET on the Wikipedia page. I dislike it because it is incomplete. It says that ET is "concisely described as simultaneous learning, test design and test execution." ... AND?!? And then what?! This definition is kind of missing what happens after you do the execution part. That's really important.

Elisabeth Hendrickson offers a better description (IMHO): "Exploratory Testing is simultaneously learning about the system while designing and executing tests, using feedback from the last test to inform the next."

I like this because it closes the loop between the purpose of the test and what you do with the results. In this case, when you learn something from the test you intentionally performed, you use that to decide what you will do next. It is kind of like playing the game of 20 questions. If you play the game poorly, you ask specific questions about what you think the answer is - e.g. "is it a turnip? No. Is it a bicycle? No. Is it a ...?" There's a slight, random chance you may guess it right, but that's really unlikely. If you play the game well, each question you ask helps you narrow down the possibilities until you can make a good guess with a high probability of getting it right.

I often use this diagram to explain the relationship between exploratory testing and test cases:

When we first look at a new feature or system, we don't know very much. We design experiments (or tests) to help us learn more about it. Initially, it is like the game of 20 questions, where we try many things and look at the system in different ways to try and discover what is important to someone who matters. That is, we explore the system for qualities and risks that we believe the customers, users, or other stakeholders may care about.

Test cases are different. When you have learned something about the feature (ET session complete), you may choose to document or automate important or representative paths (i.e. test cases) through the software for future reference (e.g. "regression testing"). You don't learn anything new from these test cases, so we sometimes refer to them as scripted "checks". We may use and reuse specific test cases for many different purposes - e.g. regression testing, performance testing user profiles, sanity checks, and so on.

At some point in history, the (1) intent or purpose and (2) test design behind the testing activities were lost and some idiot propagated the idea that test cases are the important part of the testing activity. This "bad practice" has doomed most of the technological world for a few generations now.

Let me be clear about this: test cases are not important. To anyone who knows how to test, we can create and choose new representative paths at any time, and, often times, the variations between the chosen paths through a system helps us uncover new risks and potential problems. Testing requires thinking. Checking, or blindly executing test cases, does not. If executing test cases doesn't require thinking, you may as well program a computer to run them because humans are famously bad at precisely following instructions.

An important difference between exploratory testing and scripted testing is that scripted testing blinds you to everything else going on in the system while exploratory testing aims to help you see more. To use a literary example, author Paulo Coelho posted a short story on "the secret of happiness" that illustrates this point. (NOTE: please read that story before continuing here - it'll only take a few minutes. I'll wait.)

I don't know if that is the secret to happiness, but I do know that in the first run through the palace, the young man was so focussed on the task that he missed everything else - this is exactly like scripted testing. The second time, the young man took in everything but forgot about the spoon - this is random or haphazard testing. Many people think this is exploratory testing but it is NOT! Exploratory testing would be how the wise man described the secret of happiness - complete your task and take in your surroundings.

It sounds hard, doesn't it? You know what, it IS hard. Good testing requires thoughtful effort and practice. If good testing was as easy as we are often led to believe then we wouldn't have all the software problems we have today, now would we?

Okay, so if doing good testing, exploratory testing, is hard, who can do it? Good question.

From one perspective, many people do this kind of testing naturally. BUT WAIT! Many people do this style of testing naturally, the same way many people can solve rate and calculus problems intuitively in their heads whenever you try to catch a ball. The mathematics behind the motion of a ball through the air (gravitational, kinetic and frictional forces) coupled with your movement relative to the ball in order to catch it is really quite complex. Not many people would say they understand or can do the math, but most people can catch the ball. So, part of your brain knows how to do the math even if your brain doesn't tell you how it does it.

It's the same with testing. There is a method to the madness. When someone goes looking for information, it is usually in response to some question in their head. Either someone asked them the question, or they thought it up based upon some related thought. That question drives you to poke, look, observe and evaluate what you learn to answer that question. That is testing. It has important elements: the question, intentional test design, observations, and analysis of results.

Some people are good at all of these elements, some are good at some of these elements, and some suck at all of them. To the latter group of individuals I say: please step away from the keyboard, and avoid management roles. Please.

There is an interesting side note related to Agile Software Development. Practitioners and coaches of agile methods may be familiar with the Agile Testing Quadrants. You will see that "Exploratory Testing" appears in quadrant 3, so what's that all about?

Funny you should ask. It is a bit misleading.

You may think that ET in Q3 means that it is something that is only done to critique the product with some business-facing tests. Not so. Exploratory testing will be performed in any and every quadrant as long as the person doing the testing is thinking, intentionally designing their tests, and learning from the results. Last time I checked, that happens in all the quadrants.

For example, when a programmer is creating unit tests to drive the development (Q1), they are thinking about the feature and design and making choices about what to automate. There is a lot of learning going on in this process and I would very much consider this discovery process as "exploratory". However, when the unit tests are coded and running automatically with every build, these are now "checks" and no more learning is taking place. So, executing these checks that were created in an exploratory way is no longer an exploratory testing activity. Get it?

Same thing with functional tests (Q2). You start off learning and exploring but once you decide upon and document a specific set of test cases, these test cases are no longer exploratory.

Quadrant 3 is an interesting place. It is the catch-all space for the million other tests that the system users and stakeholders may be interested in. The problem here is that complete testing is impossible and there is an infinite number of perspectives one may use to examine a particular system. The human brain is uniquely qualified to process a lot of different factors really quickly, integrating and adapting to new information, and eliminating and ignoring aspects that are not a priority to the stakeholders.

Computers cannot do this. Not even close. That's why the bubble in the corner of the matrix says "Manual" - because our brains are the most efficient tools to perform this kind of testing! Of course, we make use of tools and automation to help us gather information when appropriate; we just can't let ourselves fall into the trap of thinking that computers can do this for us.

So, while exploratory testing is a means to an end in the other agile testing quadrants, it is the primary approach in this particular quadrant (Q3). Got it?

So, if you fumble your way through the other three quadrants on your agile project and you are wondering why your quality still sucks, you may need to take a serious look at finding an awesome tester with some mad exploratory testing skills. Sorry to say that this is not widely taught in schools yet, so we are still something of a rare breed.

Does this help clear a few things about Exploratory Testing? Please let me know. Cheers!

Read More

Testing is a Medium

In a few days I will be giving a presentation to the local Agile/Lean Peer 2 Peer group here in town. The group has a web site - Waterloo Agile Lean, and the announcement is also on the Communitech events page.

I noticed the posted talk descriptions are shorter than what I wrote.  The Waterloo Agile Lean page has this description:

"This session will introduce the basic foundation of Exploratory Testing and run through a live, interactive demo to demonstrate some of how it’s done. Bring your open minds and questions and maybe even an app to test. If ET is new to you, prepare to get blown away!"

The Communitech page has this description:

"Exploratory Testing is the explosive sound check that helps us see things from many directions all at once. It takes skill and practice to do well. The reward is a higher-quality, lower-risk solution that brings teams a richer understanding of the development project.
This session will introduce the basic foundation of Exploratory Testing and run through a live, interactive demo to demonstrate some of how it's done. Bring your open minds and questions and maybe even an app to test. If ET is new to you, prepare to get blown away!"

This is what I submitted:

"Testing is the medium in which solutions are developed. The value of our delivered solutions depend upon how well we understand and utilize that medium. We can fly straight like an arrow or explode outwards like a spherical sound wave.
Traditional automated TDD "checks" help us fly straight in the direction we choose. Is it the right direction though? How do we know? Are you sure?
Exploratory Testing is the explosive sound check that helps us see things from many directions all at once. It takes skill and practice to do well. The reward is a higher-quality, lower-risk solution that brings teams a richer understanding of the development project.
This session will introduce the basic foundation of Exploratory Testing and run through a live, interactive demo to demonstrate some of how it's done. Bring your open minds and questions and maybe even an app to test. If ET is new to you, prepare to get blown away!"

This blog post is *not* about the differences in session descriptions. (In fairness, I really should learn to keep it to one paragraph. I hope to get better at writing session descriptions - it'll come with practice.) Reading the last one first, I can't help think that a bit of context is missing from the two posted descriptions for the final "blown away" statement. That is, that phrase comes from the sound wave analogy and not from some arrogant expectations I have for my presentation abilities. If I had known the descriptions would be shortened, I would have at least changed that sentence.

This blog post is about the idea that I try to convey in the first sentence that is unfortunately missing from both posted session descriptions: "Testing is the medium in which solutions are developed."

Software Development is a creative process. It is the intersection of people, skills, tools and experimentation to solve a people-problem with technology. As Jerry Weinberg once said: "all problems are people-problems." Therefore all developed products or services are "people-solutions."

Perhaps the main thought with "Testing is a medium" is that you cannot (successfully) solve any problem without trying to understand what the problem is in the first place. e.g.: Who is it a problem for? Where? When? How? -- these are all *Testing* questions. When you deliver a solution, you check it with something like: "how does this meet your needs?" -- again, another question.

Software Development begins and ends with questions. Somewhere in the middle of the creative development process are more testing questions. Lots of different kinds of questions, tests and checks depending on the people, skills and risks involved in developing each particular solution. One could say that Software cannot be developed without Testing.

Have you tried? Have you ever been on a project where you didn't first ask the customer what the problem was? You didn't check to see if what you are building is working towards that design? Or you didn't ask the customer afterwards if what you delivered meets their needs? How did that work for you?

If one cannot hope to hit the target without checking many different things, why is Testing often given so little attention or recognition on development projects? Too few individuals try to develop expertise in the Testing field to elevate their contributions to the development effort.

Anyone may ask a question. That doesn't make you an "expert" in asking questions. My 10-year-old son uses scissors to cut things, but that doesn't mean I want to let him cut my hair! Everyone I meet feels they know what Testing is. Okay, then why do so many projects fail?

So, what does it take to become really good at Testing? It appears to be somewhat important for the success of most software projects. Maybe it's time to take a good look at Software Development through the medium of Testing. How might you look at things differently then? What skills or knowledge would you want to learn more about? Who do you think should be involved?

I won't be talking about any of this stuff on Tuesday though. After all, it's not in the session description. =)

Read More