Okay, to recap, Session-Based Test Management (SBTM) is a test framework to help you manage and measure your Exploratory Testing (ET) effort. There are 4 basic elements that make this work: (1) Charter or mission (the purpose that drives the current testing effort), (2) Time-boxed periods (the 'sessions'), (3) Reviewable result, and (4) Debrief. There are many different ways that you might implement or apply these elements in your team or testing projects.
Let's take a look at tracking the testing effort from strictly a Project Management perspective. Years ago, when I first became a test manager, I was introduced to the idea of the 60% 'productive' work day as a factor to consider when estimating effort applied to project schedules. That is, in a typical 8-hour workday you don't really get 8 complete, full hours of work from someone. I don't believe it's mentally possible to get that. The brain needs a break, as does the body, and there are many natural distractions in the workplace (meetings, email, breaks, support calls, stability of the code or environments, and so on), so the reality is that the number of productive working hours for each employee is actually something less than the total number of hours they're physically present in the workplace.
That 'productivity' factor changes with each person, their role and responsibilities, the number of projects in the queue, and so on. Applying some statistical averaging to my past experiences, I find that 60% seems about right for a tester dedicated to a single project. I have worked with some teams that have been more productive and some much less.
So what does this look like? If we consider an 8-hour day, 60% is 4.8 hours. I'm going to toss in an extra 15 minute break or distraction and say that it works out to about 4.5 hours of productive work from a focussed employee in a typical 8-hour day. Again, it depends on the person and the tasks that they're performing, so this is just an averaging factor.
4.5 hours is an interesting number. In the SBTM framework, the "Time box" length for the test session revolves around a normalised value. The original SBTM presentation ("How to Measure Ad Hoc Testing") suggests that 1 "Normal" session = 90 minutes (+/- 15 minutes). Your "normal" time box session may be 90 minutes, it may be less. Customise it to suit your team's needs and fit.
I'll use 90 minutes for now because it's the default recommended value to have a focussed test effort to get some solid testing done. 90 minutes. That's 1.5 hours. So 3 * Normal = 4.5 hours.
Therefore, from a project management perspective, 3 Normal sessions is an average tester's target productivity level for completed test sessions in an average 8 hour day.
It doesn't seem like a lot, three 90-minute sessions, but you'd be surprised at the number of distractions that happen throughout the day and development sprints/cycles that can prevent a tester from completing 3 sessions in a day.
Recently, I've been working with a tester on an Agile team who has been facing a high distraction level. As a result of all the interruptions for quick checks, reviews, design meetings, and so on, he has been struggling to complete 3 sessions in a day.
I should mention that I *AM NOT* asking him or anyone else on my team to complete 3 sessions in a day. I am not concerned with the numbers or the math of tracking the sessions completed, especially because the development team environment is very integrated and highly collaborative.
What I am concerned about is trying to find ways to block off his time so that he can get some solid testing done, uninterrupted. One of the powerful aspects of the time-boxed session is to reduce or minimise the distractions so that your brain can maintain the focus on the testing problems at hand. The more the distractions and interruptions, the more likely that something important or interesting will be missed.
Enter MS Outlook. When I look at my Outlook calendar it is riddled with meeting requests throughout the week. I have recurring meetings, impromptu meetings, lunch and learns, scheduled debriefs, sprint planning, retrospectives, demos and so on. I'm the Test Manager/Lead, so that's expected. When I look at my tester's calendar, it is fairly blank/open, save for the daily stand-ups and a few other weekly team meetings. It seems counter-intuitive that someone with an open calendar, so much available 'free' time, should be unable to block off time-boxed periods dedicated to testing specific features and charters.
So I've started to schedule his test sessions in MS Outlook by scheduling Appointments. At the start of the day, we'll sit down and come up with 3 important charters that we'd like to cover in the day. We block off one session/appointment in the morning and two in the afternoon. The "Subject" is a summary of the charter and priority. If the session involves another Dev or team member, they can be included in the appointment, which is now a 'meeting request'.
Outlook is a convenient tool since it (a) is available on everyone's desktop, (b) has built-in reminders, and (c) allows the tester to *see* all the free time available between scheduled test sessions. It's a way for him to try and regain control of the *uninterrupted* test sessions.
SBTM isn't intended to be a time-tracking tool. It's a productivity-enhancing tool. By keeping focussed on the charter, you give your mind time to think about the important things -- to learn about the system and the software, to apply the appropriate tools and techniques, to make the best observations you can in the time you have available. Performing good Exploratory Testing is a delicate and complex thought process.
I see my job as doing whatever I can to help provide the best environment possible for good testing to succeed. I never would have thought that Outlook would factor into that. Who knew?
Let's take a look at tracking the testing effort from strictly a Project Management perspective. Years ago, when I first became a test manager, I was introduced to the idea of the 60% 'productive' work day as a factor to consider when estimating effort applied to project schedules. That is, in a typical 8-hour workday you don't really get 8 complete, full hours of work from someone. I don't believe it's mentally possible to get that. The brain needs a break, as does the body, and there are many natural distractions in the workplace (meetings, email, breaks, support calls, stability of the code or environments, and so on), so the reality is that the number of productive working hours for each employee is actually something less than the total number of hours they're physically present in the workplace.
That 'productivity' factor changes with each person, their role and responsibilities, the number of projects in the queue, and so on. Applying some statistical averaging to my past experiences, I find that 60% seems about right for a tester dedicated to a single project. I have worked with some teams that have been more productive and some much less.
So what does this look like? If we consider an 8-hour day, 60% is 4.8 hours. I'm going to toss in an extra 15 minute break or distraction and say that it works out to about 4.5 hours of productive work from a focussed employee in a typical 8-hour day. Again, it depends on the person and the tasks that they're performing, so this is just an averaging factor.
4.5 hours is an interesting number. In the SBTM framework, the "Time box" length for the test session revolves around a normalised value. The original SBTM presentation ("How to Measure Ad Hoc Testing") suggests that 1 "Normal" session = 90 minutes (+/- 15 minutes). Your "normal" time box session may be 90 minutes, it may be less. Customise it to suit your team's needs and fit.
I'll use 90 minutes for now because it's the default recommended value to have a focussed test effort to get some solid testing done. 90 minutes. That's 1.5 hours. So 3 * Normal = 4.5 hours.
Therefore, from a project management perspective, 3 Normal sessions is an average tester's target productivity level for completed test sessions in an average 8 hour day.
It doesn't seem like a lot, three 90-minute sessions, but you'd be surprised at the number of distractions that happen throughout the day and development sprints/cycles that can prevent a tester from completing 3 sessions in a day.
Recently, I've been working with a tester on an Agile team who has been facing a high distraction level. As a result of all the interruptions for quick checks, reviews, design meetings, and so on, he has been struggling to complete 3 sessions in a day.
I should mention that I *AM NOT* asking him or anyone else on my team to complete 3 sessions in a day. I am not concerned with the numbers or the math of tracking the sessions completed, especially because the development team environment is very integrated and highly collaborative.
What I am concerned about is trying to find ways to block off his time so that he can get some solid testing done, uninterrupted. One of the powerful aspects of the time-boxed session is to reduce or minimise the distractions so that your brain can maintain the focus on the testing problems at hand. The more the distractions and interruptions, the more likely that something important or interesting will be missed.
Enter MS Outlook. When I look at my Outlook calendar it is riddled with meeting requests throughout the week. I have recurring meetings, impromptu meetings, lunch and learns, scheduled debriefs, sprint planning, retrospectives, demos and so on. I'm the Test Manager/Lead, so that's expected. When I look at my tester's calendar, it is fairly blank/open, save for the daily stand-ups and a few other weekly team meetings. It seems counter-intuitive that someone with an open calendar, so much available 'free' time, should be unable to block off time-boxed periods dedicated to testing specific features and charters.
So I've started to schedule his test sessions in MS Outlook by scheduling Appointments. At the start of the day, we'll sit down and come up with 3 important charters that we'd like to cover in the day. We block off one session/appointment in the morning and two in the afternoon. The "Subject" is a summary of the charter and priority. If the session involves another Dev or team member, they can be included in the appointment, which is now a 'meeting request'.
Outlook is a convenient tool since it (a) is available on everyone's desktop, (b) has built-in reminders, and (c) allows the tester to *see* all the free time available between scheduled test sessions. It's a way for him to try and regain control of the *uninterrupted* test sessions.
SBTM isn't intended to be a time-tracking tool. It's a productivity-enhancing tool. By keeping focussed on the charter, you give your mind time to think about the important things -- to learn about the system and the software, to apply the appropriate tools and techniques, to make the best observations you can in the time you have available. Performing good Exploratory Testing is a delicate and complex thought process.
I see my job as doing whatever I can to help provide the best environment possible for good testing to succeed. I never would have thought that Outlook would factor into that. Who knew?
