Thoughts on the StarEast 2011 conference

I first attended StarEast in 1999. I remember the day-long tutorial I attended (by Rick Craig), and two track sessions - one by Cem Kaner on hiring testers, and one by James Whittaker on "Exploiting a Broken Design Process." I know I attended other sessions but I don't have active memories of them any more. I do remember the experience of attending the conference - one of surprise and excitement. Surprise at seeing so many other people in the testing community with similar questions and problems as myself, and excitement at the speakers with lots of great information and advice to give.

Fast forward to 2011 - I returned to StarEast, this time as a speaker. I suppose I didn't need to wait 12 years to return as a speaker. I didn't intentionally ignore the conference. I think I've been busy with other things and it just didn't come up - until last Fall when I received an invitation in my inbox to submit a proposal. I'm really glad I went.

Some things were familiar - the beautiful hotel, the Florida sunshine, the amazingly fresh orange juice, and the basic conference format. One thing that was different for me this time around was the number of people/speakers that I new who were also speaking at the conference. After having attended and spoken at several other conferences over the years, I guess I have gotten to know many of the popular speakers.

I was happy to see many more people speaking that I have never heard about before.  That tells me that the community is still growing after all this time and that there are still many more people sharing their knowledge to help enlighten future generations of testing leaders. That's awesome!


I was particularly surprised at the calibre of the Keynote speakers. I was genuinely inspired by every Keynote that I attended (I think I only missed one). (You can find the 2011 program online with details of the speakers and talks but I don't believe they were video recorded.) The Keynote presentations I attended were by: Andy Kaufman, Naomi Karten, and Julie Gardiner. The majority of my #StarEast tweets were from these keynotes.  Unfortunately, I missed the Keynote by Gojko Adzic (I heard it was good too), and I'm not including the "lightning" talks here although I did attend those and some were very good.

I was excited to hear that Lisa Crispin and Janet Gregory, authors of Agile Testing, were attending the conference and am happy to have had the opportunity to meet them and speak with them! =)  Lisa even attended my track session and wrote up a nice summary about it on the Software Testing Club site.

I won't really say much about my session here. It went okay I guess. I haven't received the session feedback evaluations yet, but I can say that I was really happy when several people came up to me afterwards in the hallway to thank me for my talk. They said that they really enjoyed it. My favourite comment came from Nawwar, my track chair -- i.e. the person who introduces the speaker during the session. He went from asking me "who are you?" just before the session to "Wow. That was the best session I've seen during this whole conference!"  Nice. Thanks. =)

I had fun giving the talk - "Real-time Test Design for Exploratory Testing". Test Design is a topic that I am really passionate about and I can talk about for hours. Okay, days. ;-)  I think the main thing that I wasn't terribly crazy about my talk was the format. When I talk about Test Design, it is usually when I am teaching it. So I found it hard to just talk about it and not have an interactive session with the attendees to give them a chance to practice it. Don't get me wrong - I had some interaction in my session, but not the sit-in-front-of-a-computer-and-try-things-out kind of way.  If I give this talk again, I will try to find some way to make it more interactive. (I may need more glow-in-the-dark straws.)

Back to the conference.  I was pleasantly surprised to bump into a former employee - a tester on a team that I managed about a decade ago. Wow! Still in Testing after all this time. AND attending a conference too! Double rainbows! We've hooked another one! ;-)

I've worked with many testers over the last decade and I hear from so few of them. I am always happy to meet them again and know that they are still doing and learning about Testing. I am happy to know when I help testers get better at what they do - I get ecstatic when I find out that they continue to see Testing as a Profession and participate in conferences and networking events!

StarEast had a few more surprises for me. At the Vendor Expo I bumped into Rick Craig - the tutorial speaker from when I first attended StarEast in '99. I introduced myself and told him how I remembered him. He was both pleased and suddenly felt older. Ha ha. We had a good chat.

Continuing my stroll through the Expo, the vendors were vendors. I still think some of the tools are the wrong ones for testers (like, totally and completely without value) but I did see some interesting ones that might have potential in certain circumstances.

As an independent consultant, most vendors weren't interested in me. I don't represent some big company with deep pockets to shell out on their bloatware documentation tracking systems. Of all the vendors, only one stopped to talk to me and really understand what it was that I had difficulty with their tools. He was fascinated by my knowledge of Agile practices, how testers fit on Scrum teams, and why these tools don't help. He asked me for my card and said that he was interested in contacting me to see if I can help provide some feedback for a future generation of tools that might work better. I gave him my card. I'll be curious to see if he contacts me. =)  I love to help - just ask!

My final pleasant surprise, and ultimately the best reason for attending the StarEast conference, came between sessions and during the meal breaks. I made an effort to sit with people I didn't know and engage them in conversations. Mostly I was curious to know what they did and what brought them to the conference. I was blown away by the passion that many of these attendees have for Testing. Wow. Some described to me the problems they face at work, the inequalities from other development team members (both in status and in salary), and the barriers preventing them from doing more. And yet, here they were at the conference to "learn from the best", to find ways to provide more value, make life better for them, their teams and their organisations.

They were dripping with hope and enthusiasm and I must say that I was overwhelmed with joy on more than one occasion following a conversation like this. If I had nothing else nice to say about the StarEast conference and received no other benefit from attending, witnessing the passion of the attendees was enough to fill my heart with passion and drive to get out there and do more, teach more!

I have been exposed to a lot of negativity over the last decade in several of the companies I have worked at. It wears you down after a while. You begin to lose faith in yourself and in the [testing/quality/value] mission. Attending StarEast changed that for me and recharged my spirit in a way that I totally wasn't expecting.

I was asked if I planned to speak at StarEast again next year. I don't know. I hadn't planned on it. I might not in 2012. We'll see about the following year or maybe I will speak at one of the other SQE conferences - like Better Software, Agile Development Practices or maybe even StarWest (since I haven't been to the Pacific coast yet).

There is so much to do in this area, in my "local" part of the world. It seems I am gaining recognition everywhere but here and I need to do something about that. StarEast helped recharge me and I hope I enlightened or entertained some attendees of my track session on Test Design and Exploratory Testing. Time will tell if I had a lasting impression on anyone. I hope I did. That wasn't why I went, though, and I got so much more out of it than just the opportunity to share some of what I have learned over the years.

I'm glad I went.

Read More

Reflection on my Testing workshop at the KWSQA Targeting Quality Conference

At this year's KWSQA Targeting Quality conference I gave a half-day workshop titled "Exploratory Testing Basics".  I originally proposed that title since I thought it followed nicely from the shorter workshop I gave at the QAI TesTrek conference in Toronto last Fall. I thought to myself - I'd like to redo the exercises again, change up a few things and it should be a piece of cake.

As the Winter months progressed into Spring, I began to worry about my workshop idea more and more.  You see, the exercise I gave at the QAI conference, while fun and appropriate, only really covered one aspect of Exploratory Testing - a broader framework. Perhaps that isn't enough?  What is enough, then? What makes up the "basics" of ET?

You see, when I teach ET, it's usually one-on-one and I spend 2-3 days just to cover the basics.  It takes me a few more days of pair testing and debriefing/coaching to help the new tester put everything into practice. It really is quite complex and a lot of ideas and models may seem abstract until you try them out and adjust with good feedback.


One of the hardest parts, I feel, is trying to teach certain techniques when the tester doesn't see a need for it.  For example, Pairwise Analysis.  I was introduced to Pairwise Analysis as "Functional Analysis" about 12 years ago and I got it right away. I had done more complex mathematics in university, so it wasn't the math that was the hitch for me - it was knowing when it might be useful and then applying it.

The first project I tried it on was when I was asked to perform Installation Testing for a desktop application. If you have ever done this kind of thing, you will know that there are *many* features and variations that all conspire to convince you that it is a daunting task that may consume every waking moment for weeks on end if you ever want to try and cover all of the possible combinations of systems, hardware, software, feature selections, and so on.  Enter Functional/Pairwise Analysis. I did the math; came up with a set number of test scenarios; performed them; and reported my findings in record time -- only a few days instead of the customary 2 weeks that it had taken on previous releases.

That was really cool! I had a new tool in my Tester's Tool belt and I couldn't wait to try it out again.

Years passed and several failed attempts later to teach other testers this cool technique, I finally stumbled upon the idea of Just-In-Time teaching. That is, rather than try to teach a tester all the techniques and models that I have learned over two decades and cram them into a few days, I will wait until they are presented with a problem and introduce the appropriate technique then.

There are two important take-aways for me with this JIT approach. First, it is really effective and the tester gets it - great! Second, it may take a long time before a tester is presented with the situations where certain techniques apply - not so great.

Present day.

So, what do I cover in a 3-hour workshop that I would consider 'good enough' to cover the basics of ET?

The answer, of course, is that proper workshop coverage will be the gap in knowledge from where someone is to where you want them to be. Unfortunately, the knowledge/experience starting point for each individual attending my session will be vastly different, their needs for this information will be different, and unless they have specific concerns some of the ideas very likely won't stick.

There are many unknowns in that equation. So how can I plan an outline to cover this unknown gap for unknown purpose(s) in a short amount of time? Stress.

What did I decide to do? I didn't plan an outline. I took a page out of Jurgen Appelo's book and I had the attendees self-organise and decide.

I handed out index cards and asked each person to create a user story card with their goal for the session. We stuck them to a wall with the heading 'Backlog' on it. On another part of the wall I had a Task board with the headings 'To Do', 'In Progress' and 'Done'.

I asked all the attendees to decide as a group, select the top 4 cards that they wanted to cover this session, and place them in the 'To Do' column. They didn't believe me at first when I asked them to do this and kept asking me to decide. It was really cool to watch the transformation happen and have them take ownership for deciding on the workshop goals.

I read each card, picked one, moved it into the 'In Progress' column and began.

So what did I actually cover? The attendees grouped most of the cards together into one big group and the one they picked said that they wanted to "gain an understanding of ET techniques."

Several years ago, I worked in a Financial Services organisation and was faced with a few audits by Banks where they needed to understand what testing artefacts we produced and why they didn't match their traditional Test documentation expectations.

Knowing that I couldn't just show them test session notes, test guides and other Exploratory Testing artefacts because they wouldn't understand what they were looking at, I created a presentation that had 4 parts:

1. The Challenges in Testing
2. What is Agile Development?
3. An Overview of our Systems Testing Approach
4. Examples of Testing Artefacts we Create on Software Projects

The auditors wanted to see the last part, but I explained that I needed to cover the first three before they could understand what they saw.

In the "ET Basics" workshop at the KWSQA conference, I covered the important aspects of the first three sections above (I have expanded upon the original presentation over the years), and supplemented that with some additional exercises to cover a critical aspect of Exploratory Testing - Test Design.

Did I meet the objectives? I think so. I covered the underlying principles and ideas that form the basis of good testing, "exploratory" or otherwise. The next step would be to look at some specific models and structures and have them perform exploratory testing.

While performing ET may be an exciting and enlightening activity, it is also a complex and challenging one that requires careful debrief.  In other words, we will need more time.

Will I give this same presentation again? Not sure. I might if that's what the attendees want to see. I'll let them decide.  If they want to practice something instead, or focus on managing such a testing activity, then we will do exercises for that instead.

I had fun in this workshop and learned some great things from an exercise I tried for the first time. I also thought the training video I showed was an appropriate fit for both a Friday afternoon and the topic covered. ;-) I can't wait to build upon what I've learned from this workshop experience and offer more in the future!

Cheers! Paul.

Read More

Smart Metering - Smart Home

With a lot of sports action in the past weeks (read cricket, formula one, moto gp) I haven’t been able to run the last mile of my smart meter marathon. Finally this would be the last post in the entire smart meter series where we would take a brief look at how smart homes are envisioned to look like.

The vision for a Smart Home is not just about installing a smart meter and linking it upto the supplier. Following diagram depicts a vision of where we want to get to with a smart home:

Every Smart meter home in the grid will have a,

i. Home Area Networks (HANs)

Smart meter can communicate with the household devices and other smart appliances over the HAN via wireless technologies

ii. Home Display Unit (HDU)

It’s a display unit, communicating with the Smart meter over the HAN, giving consumption information

iii. Wide Area Networks (WANs)

Consumption information alongwith other metrics can be viewed over the internet on computers, hand held’s etc

So a smart home will be like a mesh of smart appliances, each interacting with one another improving consumer experience.

Please do send me your comments about the entire smart meter series…Ciao…

Read More

Smart Metering – The Setup

Some more buzz on smart metering........tried my hands with the aging MS Paint but gotta admit it still does come handy.....Well have my best work with paint inline along with some smart metering gyan…

Following diagram represents how a smart meter network is set-up and the actual infrastructure layout will be more or less similar to this set-up:

Here we have taken an example of a layout where power line communication (PLC) is used as the means of communication between the smart meter and the supplier infrastructure. The actual implementation of a smart meter program can adopt any modes of communication like GPRS, Broadband Lines etc. Let us now have a brief look at some of the components in the smart metering infrastructure:

i. Smart Meter Home

Every housing unit with an installed smart meter will be termed as a smart meter home. We will have a brief overview of the components in a smart meter home later in the document.

ii. Communication Centre

At the back of the smart metering infrastructure is a robust, scalable communication centre, also referred to as a head-end system, which supports two way communication i.e. data reception and data transmission. Design of such a communication center will be strategic as it has to ensure backward compatibility until the market transitions onto the new smart devices.

iii. Meter Data Management System (MDMS)

Apart from the conventional non-interval data, the MDMS has to be enabled to store interval consumption. MDMS also need to improve on the data integrity and privacy policies in wake of the huge amount of usage data been stored in the system.

iv. Stakeholders Data Access Only

Ease of data access is one of highest selling points of the smart metering program however authentication at all access points has to be ensured to ensure confidentiality and identification before allowing data exchange between authorized parties only.

v. Customer Experience

Using the existing cellular and broadband networks, suppliers can provide accurate and timely information to the customers enabling them to better manage their energy consumptio

Read More