ESB Toolkit 2.1 installation on BizTalk Server 2010

Pre-requisite for ESB 2.1

·         Microsoft Windows 2008 Server SP1/ SP2

·         .NET framework 3.5 SP1

·         IIS 7.0

·         Microsoft SQL Server 2008

·         Microsoft BizTalk Server 2009/2010

·         Visual Studio 2008 SP1/ 2010

·         Microsoft Chart Controls for .NET Framework 3.5 (This would be used by ESB Portal)

(http://www.microsoft.com/download/en/details.aspx?id=14422)

·         Microsoft Enterprise Library 4.1

Note: If your machine has Microsoft Enterprise Library 5.0 installed it would be advisable to un-install 5.0 and install Microsoft Enterprise Library 4.1 (http://www.microsoft.com/download/en/details.aspx?id=6228) as the ESB solution targets 4.1 version.

     1)    Install Microsoft ESB Toolkit 2.1 (http://www.microsoft.com/download/en/details.aspx?id=11847)

This will install the tools and samples.

     2)    Navigate to Start à All Programs à Microsoft BizTalk Server 2010 ESB Toolkit

Click Install Itinerary designer extension to register the itinerary designer extension in Visual Studio 2010

     3)    Verify whether the installation was successful by;

Open Visual Studio 2010 and check whether you see an option to create an ESB itinerary

     4)    Open BizTalk admin console and;

·         Import Microsoft.Practices.ESB.CORE64.msi into the Applications node (remember to select Overwrite resources). Continue the installer to install the assemblies into GAC.

If you get an error like;

There is a problem with this Window Installer package. A program required for this install to complete could not be run. Contact your support personnel or package vendor.

This implies there is a problem GACing the assemblies. At this point all assemblies except Microsoft.Practices.ESB.ExceptionHandling.PipelineComponents.dll are in the GAC.

o    To GAC the above, navigate to the folder where the installer had dumped all the assemblies which were to be GACed. Default location is "C:\Program Files (x86)\Generated by BizTalk".

o    Navigate to C:\Program Files (x86)\Generated by BizTalk\Microsoft.Practices.ESB\<GUID Folder>\ and extract ITEM~14. This will have Microsoft.Practices.ESB.ExceptionHandling.PipelineComponents.dll which can be dragged to the GAC.

·         Clear "C:\Program Files (x86)\Generated by BizTalk" folder

·         Import Microsoft.Practices.ESB.ExceptionHandling64 into the Applications node (remember to select Overwrite resources). Continue the installer to install the assemblies into GAC.

If you get an error like;

There is a problem with this Window Installer package. A program required for this install to complete could not be run. Contact your support personnel or package vendor.

Ignore this error as all required assemblies have been GACed.

·         Clear "C:\Program Files (x86)\Generated by BizTalk" folder

·         Import Microsoft.Practices.ESB.JMS into the Applications node (remember to select Overwrite resources). Continue the installer to install the assemblies into GAC.

If you get an error like;

There is a problem with this Window Installer package. A program required for this install to complete could not be run. Contact your support personnel or package vendor.

This implies there is a problem GACing the assemblies. At this point all assemblies except Microsoft.Practices.ESB.JMS.PipelineComponents.dll are in the GAC.

o    To GAC the above, navigate to the folder where the installer had dumped all the assemblies which were to be GACed. Default location is "C:\Program Files (x86)\Generated by BizTalk".

o    Navigate to C:\Program Files (x86)\Generated by BizTalk\ Microsoft.Practices.JMS\<GUID Folder>\ and extract ITEM~0. This will have Microsoft.Practices.ESB.JMS.PipelineComponents.dll which can be dragged to the GAC.

·         Clear "C:\Program Files (x86)\Generated by BizTalk" folder

·         For safety place the pipeline components assemblies i.e.

o    Microsoft.Practices.ESB.JMS.PipelineComponents

o    Microsoft.Practices.ESB.ExceptionHandling.PipelineComponents

; Into "C:\Program Files\Microsoft BizTalk Server 2010\Pipeline Components"

     5)    Navigate to "C:\Program Files\Microsoft BizTalk ESB Toolkit 2.1" and extract ESBSource.zip file. This would create the ESBSource solutions folder with a Keys subfolder having the assembly key files and Source subfolder with the actual ESB solution

     6)    We need to create an assembly key file to be used by the ESB solution. To do that open Visual Studio command prompt and type in;

sn -k "C:\Program File\Microsoft BizTalk ESB Toolkit 2.1\ESBSource\Keys\Microsoft.Practices.ESB.snk"

     7)    Navigate to "C:\Program Files\Microsoft BizTalk ESB Toolkit 2.1\ESBSource\Source\Samples\Management Portal\ESB.Portal"

Open ESB.Portal solution and re-build the solution.

Do NOT proceed unless the solution builds successfully. You may have to fix some references to get the solution in a compiled state.

     8)    Next we have to change the script execution policy as we are going to run some scripts to install the portal solution.

Navigate to Start à All Programs à Accessories à Windows PowerShell and click on Windows PowerShell.

·         At the prompt type get-executionpolicy which will display the current execution policy.

·         If it says Restricted we have to change it to Unrestricted as follows:

o    At the prompt type in set-executionpolicy Unrestricted

Since the portal solution scripts may target any version of Windows PowerShell we have to set the execution policies for both the x64 and x86 versions. So,

 Navigate to Start à All Programs à Accessories à Windows PowerShell and click on Windows PowerShell (x86)

·         At the prompt type in get-executionpolicy which will display the current execution policy.

·         If it says Restricted we have to change it to Unrestricted as follows:

o    At the prompt type in set-executionpolicy Unrestricted

      9)    Navigate to "C:\Program Files\Microsoft BizTalk ESB Toolkit 2.1\ESBSource\Source\Samples\Management Portal\Install\Scripts" and double click on Management_Install.cmd

This would install the portal and create all the required virtual directories in the IIS

     10)  Open IIS and make sure ESB.Portal virtual directory is created under Sites\Default Web Site.

Ensure that it is pointing to the correct folder i.e. "C:\Program Files\Microsoft BizTalk ESB Toolkit 2.1\ESBSource\Source\Samples\Management Portal\ESB.Portal"

     11)  Select ESB.Portal node and in the page on the right hand side double click Authentication.

Ensure that Windows Authentication is Enabled.

     12)  Navigate to "C:\Program Files\Microsoft BizTalk ESB Toolkit 2.0\Bin\" and double click on ESBConfigurationTool.exe.

Note: ESBConfigurationTool works in a linear way i.e. Exception Management, Database is configured first followed by Exception Management, Exception Web Services followed by ESB Core Components, Itinerary Database followed by ESB Core Components, Core Web Services and finally the Configuration. Also each of the features have to be configured separately i.e. Hit "Apply Configuration" after each step rather than trying to configure all features in one go.

·         Open ESB Configuration à Exception Management à Database

o    Check Enable Exception Management Database and specify the Database server name

o    Click on Apply Configuration

·         Open ESB Configuration à Exception Management à Exception Web Services

o    Check Enable Exception Services

o    Provide the user name, password details and Click on Apply Configuration

·         Open ESB Configuration à ESB Core Components à Itinerary Database

o    Check Itinerary Database and specify the Database server name

o    Click on Apply Configuration

·         Open ESB Configuration à ESB Core Components à Core Web Services

o    Check Enable Core Services

o    Provide the user name, password details and Click on Apply Configuration

·         Open ESB Configuration à Configuration

o    Make sure File Configuration Source is selected. (This option will save all the settings to ESB.config file)

     13)  Add all portal user names to BizTalk Isolated Host Users, BizTalk Application Users, SSO Administrator and SSO Affiliate Administrator groups.

     14)  Deploy the portal bam tables. Open a command prompt and navigate to "C:\Program Files\Microsoft BizTalk Server 2010\Tracking". Run the following commands to deploy the bam tables:

·         bm.exe deploy-all -DefinitionFile:"C:\Program Files\Microsoft BizTalk ESB Toolkit 2.1\Bam\Microsoft.BizTalk.ESB.BAM.Itinerary.xml"

·         bm.exe deploy-all -DefinitionFile:"C:\Program Files\Microsoft BizTalk ESB Toolkit 2.1\Bam\Microsoft.BizTalk.ESB.BAM.Exceptions.xml"

If you get an error like;

Deploying View... ERROR: The BAM deployment failed. The BAM Star Schema database has not been configured. Run bm.exe setup-databases to configure the database.

It was because the BAM Star Schema and analysis services have not yet been configured.

o    Open Run and type in Configuration or alternatively open the BizTalk Server Configuration

o    Click on Bam Tools in the BizTalk Server Configuration window.

o    Check Enable Analysis Service for BAM aggregations  and specify an appropriate Server Name for Bam Analysis Database  (optional) and BAM Star Schema Database

o    Click on Apply Configuration to create the databases.

o    Before running the bm deploy command again we have to grant ourselves access to the newly created BAM Star Schema.

Open SQL Management Studio à Database Instance à Security à Logins and double click on <Your Installation ID>.

In the Login properties window select User Mapping node and check on BAMStarSchema.

In the Database role membership pane for BAMStarSchema check public, db_datareader, db_datawriter and Click OK

o    Run bm.exe deploy-all -DefinitionFile:"C:\Program Files\Microsoft BizTalk ESB Toolkit 2.1\Bam\Microsoft.BizTalk.ESB.BAM.Exceptions.xml" again

     15)  Exception management portal relies on ExceptionService.svc WCF service. IIS 7.0 installation does not have a script map for .svc files and hence they have to be registered explicitly as follows:

Open a command prompt and navigate to "C:\Windows\Microsoft.NET\Framework\v3.0\Windows Communication Foundation"

Run the following command: ServiceModelReg.exe -r –y

     16)  Restore the execution policy to Restricted i.e.

Navigate to Start à All Programs à Accessories à Windows PowerShell and click on Windows PowerShell

Run set-executionpolicy Restricted
Navigate to Start à All Programs à Accessories à Windows PowerShell and click on Windows PowerShell  (x86)

Run set-executionpolicy Restricted

     17)  Restart IIS and SQL Server.

     18)  Verify whether ESB 2.1 installation-configuration was successful by opening  a browser with the following url:

http://localhost/ESB.Portal/Reports/Reports.aspx

Other Errors

      1)    On opening the ESB Management portal if you get an error message pointing to the event viewer and the event viewer message says "assembly Microsoft.Practices.ESB.Portal.Global not found", this means that the ESB portal solution build has been un-successful i.e. Step (11) has to be re-visited

      2)    On opening the ESB Management portal you get an error message pointing to the event viewer and the event viewer message says "could not load type System.ServiceModel.Activation.HttpModule", this means that because of the order of the software installation on your machine i.e. IIS installed after .Net framework the .net version has not been registered with IIS.

To fix this problem;

Open a command prompt, navigate to "C:\Windows\Microsoft.NET\Framework\v4.0.30319 (or the version installed on your machine)" and run the following command

aspnet_regiis.exe –iru

      3)    If there are still any problems with the installation try the following links
      http://dgoins.wordpress.com/2010/05/01/esb-toolkit-management-portal-installation-notes-from-the-field-2/

http://msdn.microsoft.com/en-us/library/ee236738(BTS.10).aspx

Read More

Pre-requisite for BAM Portal

One of the important pre-requisites before trying to configure BAM portal on your machine is the installation of IIS 6 extensions. Following steps should be followed before configuring the BAM portal via the BizTalk configuration tool.

·         Right click My computer  and click Manage

·         Expand Roles and select Web Server (IIS) node

·         Scroll through Role Services and check status of IIS 6 Management Compatibility role service

·         If status is Installed then close the windows

·         If status is Not Installed then right click Web Server (IIS) node and select Add Role Services

·         Select IIS 6 Management Compatibility node and complete the installation

Read More

Hobbies and Interests

Several years ago I wrote an article summarising some of the key points I keep in mind while interviewing candidates for a test team. The article is called "Hiring Software Testers in an Information Age" and is available as a PDF on my main site. The article was originally targetted to recruiters who kept asking me for advice on hiring software testers and they would always be surprised at the level of detail that I went through in describing what it takes to hire a good person for a testing position.

Conversations with recruiters over coffee would always start the same. I would say something like: if you are just trying to find a warm body to fill a position, then you don't need to hear what I have to say.  If you want to hire someone who thinks and has a good chance of fitting in with the culture of the team and organisation to provide value, then it is a complex problem that requires insights into what the position actually involves.

There are about a dozen different checkpoints that I go through when considering and interviewing candidates, and the paper I wrote touched on some of the major points but not all of them.  Actually, I even removed some of them from the article as early drafts had too much information.  My intention was to get some of the important points across without writing a book.

Recently, a colleague and friend, Michael Mahlberg tweeted the following:

RT @NolanBushnell: At Atari we hired based on hobbies and not grades in school. We ended up with the best engineering group in the world.

I liked that comment and followed up with a supporting tweet:

On Hiring: if a résumé or cover letter doesn't describe Hobbies or other Interests, I usually skip it.

This sparked some conversation on twitter and I want to elaborate on my comment here.


First off, it is important to know that I place myself in the Context-Driven School of Testing. The principles of this 'school' puts the focus on the people working together to help deliver the right solution.  One of the things I have noticed over the last 22 years in the software industry is that the best testers are the ones who care about testing. They have fun with it. They believe they are working to improve things for the customer. They have a drive and motivation that lets them ignore or put up with a lot of crap that is dumped on them by unhealthy organisations and ignorant individuals.

Unfortunately, that kind of motivation, passion or drive doesn't come across on a standard résumé. I can spot it straight away when I am talking with someone, but how do you communicate it in a standard functional résumé?  In general, I don't see it in the "technical skills" or job description sections that focus on accomplishments and other task-oriented details. The majority of the time I notice evidence or hints of passion and motivation in the cover letter, if anywhere at all.

So, when I am hiring for a test team, a team that I want to integrate well with the rest of development and the organisation, a team that I want to focus on building human relationships as well as exercise systems and scientific thinking in their quality investigations, a team that I want to encourage fun and respect for their hard work in providing valuable information to help make timely decisions, where do I start when I am looking at a stack of résumés and job applications?

People who submit cover letters go to the top of the pile. People who include "Hobbies and Interests" in their résumés are next. People who don't submit a cover letter and don't tell me anything other than a bunch of dry technical information and job details are put at the bottom of the list and often fall right off the pile.

Having a cover letter or a "Hobbies and Interests" section doesn't guarantee an interview but your chances of having me give you a quick call are higher. So, what's the deal with this "Hobbies and Interests" section anyway?

Johanna Rothman wrote an awesome book called "Hiring the Best Knowledge Workers, Techies & Nerds." If you don't have a copy yet, and you are in the business of hiring technical people, I highly recommend you get a copy of this book.  Johanna is an awesome person and a terrific writer. She writes a few blogs and I learn a lot from them.

Johanna wrote a good blog post back in 2004 titled "Tips for Reviewing Resumes". One of the points she wrote in that post says: "Hobbies or other personal information. This stuff isn’t relevant to the job and should not be part of how you select candidates."

I partly agree with this sentence. I really don't care about personal information in an application. By "personal information" I mean things like marital status, age, sex, or anything else that the government might use to classify you in their census demographics charts.

What about hobbies and other interests? Things like: playing music (e.g. piano or guitar), likes to read, play video games, do magic, go cycling, martial arts, improv, and so on.

This stuff I find both interesting and relevant! Why? Creativity, passion and (skill) transference.

I am looking to hire people -- intelligent, thinking, caring, fun people and not drones. I honestly don't know whether the stuff I'm reading on your application is true, embellished or fabricated. I want to get an idea of the whole picture of who you are. I will test you for your technical ability during the interview, so I'm not worried about that part.  Finding someone who has the right mindset and will fit in with the rest of the team is harder to grasp from technical details and job accomplishments alone.

Based on how I currently understand, apply and teach Software Testing, I like to find candidates who exercise both their creative and analytical parts of their minds -- i.e. people who are right and left-brained (to use a dated, flawed model of the human mind). When I read through the candidate's hobbies and interests, I build a list of assumptions that I can check in a phone call or in-person interview.

Assumptions might be things like:

• if I see someone who likes to do creative things like play music, sing, knit, et cetera, these are right-brained/creative activities. This is a good sign that the candidate might be good at Exploratory Testing as I do it.


• if I see someone who participates in theatre, improv or role-playing games, then they may be good at user profiling and test design.


• if someone likes to read, this may tell me that they are learners and continue to feed their imaginations. This is also a good sign for Exploratory Testing, and I am happy to ask them about the kinds of books they read.


• sometimes their interests may align very well with the interests of the organisation and the product or solution in development. Things like playing video games or participating in sports are really good hobbies to have if the hiring company makes games or develops solutions for the Sports industry. 

There are too many examples for me to list here. The point is that sometimes the Hobbies and Interests section can provide me with insights that I won't get from the Work Experience, Technical Skills or Education sections of a résumé alone. I look for evidence of creativity, motivation, passion, balance, feeding imagination, professional activities, community involvement, and so on. Sometimes, the skills they develop in their hobbies are directly transferable to the testing tasks required -- e.g. learning, observation, relationship-building, organisation, analysis, design, problem-solving, and so on. Sometimes they aren't - it depends on the individual.

These are all assumptions, I know, so I treat them like assumptions. I make note of them and ask about them in phone calls and interviews.

If I like the candidate and hire them, I use their hobbies/interests as examples when coaching them on testing theory, models and practices. Since the hobby is familiar to them, I find this is a really powerful teaching technique. I also encourage them to come up with their own testing analogies using knowledge and experiences they are familiar with. It raises their self-confidence and helps them to remember abstract ideas in their own terms.

The likelihood of me finding a Testing Superstar by skimming through résumés is pretty slim. The likelihood of me finding a candidate who I can help become a Testing Superstar is much higher if I can learn more about them in their job application; learn more about what makes them a unique, creative, passionate individual who cares about others, doing excellent work, providing value and enjoying life.

But that's just me. All interviewers are different. Your mileage may vary.

When in doubt, I suggest candidates be themselves. If you are penalised in a hiring process for it, you don't want to work there anyway. You are allowed to have a life. Work is just a part of it.

Read More

Quality Center Must Die

It is not a matter of "if" -- it is a matter of "when" HP's Quality Center software will die.  And you, my dear readers will help make that happen.

"How?" you may ask? Simple. There are two things you should do: (1) think, and (2) don't put up with crap that gets in the way of delivering value to the customer and interacting intelligently with other human beings.

But I am getting ahead of myself. Let's rewind the story a bit...

Several months ago I was hired by my client to help train one of the test teams on agile and exploratory testing methods. The department has followed a mostly Waterfall development model until now and wants to move in the Agile direction. (A smart choice for them, if you ask me.) Why am I still there after all this time? That's a good question.

After attending the Problem Solving Leadership course last year, and after attending a few AYE conferences, I changed my instructional style to be more the kind of consultant that empowers the client with whatever they need to help themselves learn and grow.  It's a bit of a slower pace, but the results are more positive and long-lasting.

I am a part of a "pilot" agile/scrum team and am working closely with one of the testers (I will call him "Patient Zero") to coach him on good testing practices to complement the agile development processes. I have done this several times now at different clients, so this is nothing new to me. One of the unexpected surprises that cropped up this time was that this development team is not an end-to-end delivery team, so when they are "done" their work, the code moves into a Waterfall Release process and it all kind of falls apart. There are still some kinks to be solved here and I am happy to see some really bright, caring people trying to solve these problems. So that's okay.


Patient Zero and I are part of a larger test team, and the rest of the test team all work on Waterfall-style projects, use Waterfall-compatible tools, and they generally don't get how we work. :) Unfortunately, one of the tools mandated for our team's use is HP's Quality Center (HPQC).  I hadn't seen that tool in about a decade and it looked very similar to how I last remember it.

To my agile coach/practitioner friends I should clarify that at no time during our sprint development work does anyone ever touch HPQC! However, once the code is deployed/falls into the Waterfall Release process, regression test cases are created in HPQC and it is used for defect tracking. It is mandated, and so shall it be done. I can live with that. It's just a tool at this point and the impact to our ability to deliver a good solution is eliminated by the fact that we don't touch it until after we are "done". (Communication and collaboration FTW!)

Two days ago.

Our whole test team took part in a 2-day HPQC training workshop on something HP calls "Business Process Testing" or BPT. Being naturally curious to learn something new, I wanted to know what BPT was and how it fits into the bigger testing picture. Here we go.

We were given a handout with some "test scenarios" to be used for training. The test scenarios fell into this pattern:

• Scenario name/title
• Requirement description
• Test Situation (I am staring at this right now and I still don't know what this means)
• Role (kind of system user this requirement/situation applies to)
• Steps

That's okay information. The "Steps" are what you might typically expect to see if you have been testing for a while. Here is an example for working with a sample web app:

1. Login to the system with (a certain type of user)
2. Navigate to some module in the app
3. Click "Create" button from the tool-bar
4. Enter mandatory field values and save
5. Search for the information you created in the previous step
6. Logout of the system

And there were 6 of these scenarios.

I read through these tests and then I tried to follow them using the system. I quickly encountered a half-dozen bugs - some with the system, some with the test scenarios/cases, and some were open questions that I would follow-up with the Product Owner for requirement clarification.

But, woah-woah-woah-hey.. wait a minute. We only need to worry about *these* documented test scenarios! I struggled hard to keep my mouth shut about the value of time and the many different kinds of tests I would happily engage in at this point if I could leave the tool alone. But, I left that to my "inner voice" and we were now one hour into the first day's training.

At this point, we were given an overview of the HPQC modules and told to (1) enter the Requirements, (2) create BPT test scenarios, and (3) "componentize" the test scenarios into groupings of related steps. This last part required some explanation since this was new to me, and once we got it, we set to work on the task.  Since we could see (1) and (2) on the handout sheets in front of us, we went straight to work on part (3). That was kind of fun working in small group of 4 looking at these scenarios and trying to come up with solutions.

And then someone went and spoiled the fun.  We were "told" that we weren't supposed to do part (3) until after we had entered the information for (1) and (2) into the tool.  I was all like "really?" and then a team lead came by and repeated the exact same thing.  This may be summed up as: Enter data into the tool first, and think later.

I was kind of shocked with this comment and attitude and in retrospect it kind of foreshadowed the rest of the training experience -- i.e. Tool and process first; think later; maybe.  Okay. I'll play along and see where this goes.

During this exercise, I began to grasp this idea of "components" as HP uses them. I think they are like Page Objects - chunks of code (or in this case, test steps) that perform a certain function, promote reuse, and reduce duplication. Although it was never described in this way, I believe the HP "BPT" module is a proprietary Do-It-Yourself DSL. Aha! I have experience with those. I get that stuff.

So, once I got it, I started to explain the concept of YAGNI to my group team members.  That is, let's not overthink or over-engineer these components. Let's build/write them based on the needs of the requirements in front of us. We will modify the components in the future as new requirements appear.  This idea was well received and we quickly came to an efficient solution for our small group.

When we took up the exercise, I found I had to explain the YAGNI concept to the instructor/trainer (and rest of the class/team) as he proposed that we try to abstract out these components to allow for compatibility with other features and system elements. What a waste of time! We cannot know what we will need beyond our immediate needs so that kind of abstraction is a pointless exercise that leads to more headaches than you need.

Eventually, I started to get that the point of writing the test scenarios using BPT was that these components form a basic vocabulary that may then be automated at some point in the future -- yes, BPT integrates with HP's QTP automation tools.  Now, I'm all in favour of consistency, clarity, reuse, and automating tests that humans should never have to do more than once (and if there is value in re-running the test), so I struggled to understand why it was never explained to us this way.  As long as I kept the DSL/automation model in my head, I understood what we were doing and can see the potential benefits of it.

I saw many testers struggle with the exercises and models we were presented with. End of day one.

Day two began with a quick recap and then we were introduced to the bureaucracy that is Waterfall and HPQC. That is, there are review processes and workflows to cover each requirement, BPT and component. Welcome to Wasteland. (I mean the Lean Development concept of "waste" here, although other interpretations of "wasteland" may be just as valid.) Easily a third of the 2-day training was spent on the processes surrounding the management and review of the various HPQC objects. sigh.

We then moved onto the concept of "parameters" for the components we created yesterday.  Okay, I get method parameters when I am scripting with Ruby, so this was no sweat.  Given the length of time I spent parametrizing my components compared to everyone else in the class, I think I may be one of the few who really got it.

I learned a few more things about the main instructor. One was that he didn't know how to identify web page elements using commonly available browser tools. Umm, aren't these tools supposed to interact with web pages? This never came up before? You have never wondered how to find the name/id for an element on a web page? really?

The other was that he had a really bad sense of humour.  He made a reference to a "QA" joke that I shall not repeat here.  Needless to say that I found it insulting, offensive, it made my skin crawl and my blood boil.  Many unpleasant feelings and ideas arose in me and it took all my willpower and strength not to react to the blatant stupidity of insulting the profession of the students in your class and the market that the HPQC tool represents.

The final blow for the day came to me when we tried to "execute" these BPT test scenarios using the HPQC tool. THEN I discover that these "parameters" can have 2 kinds of values - fixed/hard-coded and run-time. Anyone who has done intelligent automation knows NOT to hard-code values in their scripts. Data-driven is way better, and as an exploratory tester, I may not know what value I will choose until moments before.

Here's where the HPQC tool gets stupid..er.  Regardless which parameter type you choose, they may ONLY be defined BEFORE the test execution! You cannot leave empty parameters so that when you get to a particular step, the tester can decide what value to input and feed it back into the tool.

What does this mean?  As a tester, let's say you want to create a new user profile in an app of some kind. The test parameters for things like Name, Email, Address, Country, and so on, must all be defined and set before the test is executed. You cannot decide while you are actually testing!

Why does this matter to me? It matters because it means that the humans who execute these BPT tests manually have no choice in what values they may input. Testing techniques like Equivalence Classes and BVA that help guide our choices to pursue interesting paths are completely cut-off! It turns out that HPQC treats humans worse than the automated counterparts. In discussion with an automation "expert" at the end of the class I learned that at least you can code some variability into the QTP automation scripts.  This is not possible with the same test scenarios executed manually by human beings.

So. Much. Wrong.

So after my first ever QC training session, here are some of my take-aways:

• It took me 2 days to "script" 6 test scenarios in this tool and they were rotten test cases to begin with! I suspect that outside of the training environment, it will actually take longer to complete since you won't have a "reviewer" sitting next to you waiting for you to finish your piece.
• And they weren't even automated! Who knows how long it would take to tweak the "components" to make them work with a particular automation strategy.
• HP QC will never be a useful tool for any agile or rapid development efforts
• (HP best practice) Put the tool, data and review processes first, before you think. Maybe always instead of it too.
• BPT is a DSL framework for test scripting
• BPT component parameters cannot be customised during test execution. They may only be set/defined before you start testing. => No thinking allowed while testing.
• The instructor didn't appear to be knowledgeable on anything outside of the tool itself. This includes how we might actually want to use the tool. No, no, no. And I quote: "Testers must change how they work to use the tool in the way it was designed."
• As long as there are people pushing these kinds of horrible tools that suck the life and intelligence out of people, inject mountains of wasteful activities that provide no value to the customers or end users, and continue to create barriers between testers and their developer counterparts, I will always have job security in helping organisations recover from these cancers.

At the end of the day, there were several people interested in my idea of randomising tests. The automation expert in the class insisted that it couldn't be done with automation, so I called up my "Unscripted Automation" presentation slides that I gave a few years ago. He said that what I proposed was not a "best practice" and that everyone, the whole industry, was using the tools in the way that he described how they should be used.

My response was to simply say "they are all wrong."

Read More

Visual Studio: Remote Debugging

Given the task of remotely debugging UAT code from my workstation, I read so many articles around steps to get it working. Though all of them contributed in getting it working, there was not a single silver bullet post which nailed the solution for me. The approach that worked for me was a concoction of tips and tricks from multiple posts. Let me chart out the steps I treaded to get there, hoping that for some it works as a one stop shop. Please note my IDE is VS 2005 Team Edition.


1) For remote debugging, a remote debugging monitor service msvsmon should be running on the remote machine. For 64bit processes, the service executable is available at <install path>\Microsoft Visual Studio 8\Common7\IDE\Remote Debugger\x64.
2) Double click the executable on the remote machine which opens a window with a timestamp and a message “Msvsmon started a new server named <login name>@<remote machine name>. Waiting for new connections.” i.e. the service has been started. Please note that the <login name> would be the user with which you have logged onto the machine and maybe qualified with the name of the domain on which you are currently logged onto.
3) Build the solution on the host machine and place the assemblies along-with the .pdb files onto the remote machine file system. GAC these assemblies from the remote file system into the remote assembly cache.
4) Restart the appropriate service/ process so that it starts picking up our newly gaced assemblies. In my case it was the BizTalk service which I had to bounce.
5) On the host machine IDE menu, open Debug --> Attach to Process. Set Transport as “Default” and Qualifier as <login name>@<remote machine name> i.e. details shown in the msvsmon window. Please do not omit the domain name. You can simply copy paste the <login name> from the msvsmon window.
Hit [Refresh] to see processes hosted on the remote machine and attach to the required process.
6) In Step (5) if you get an error like,
         Unable to connect to the Microsoft Visual Studio Remote Debugging Monitor named <login name>@<remote  
         machine name>'.  A security package specific error occurred.
Check whether,
         The <login name> is an admin on the remote machine or added as a remote user on the remote machine.


 If yes then use the following work-around:
a) On the remote machine create a user “RemoteDebug” (any name would do) and add the user as a local administrator
b) On the host machine, also  create a user “RemoteDebug” and add the user as a local administrator
c) On the remote machine run msvsmon service under RemoteDebug user by doing a “run as”
d) On the host machine IDE open Debug à Attach to Process. With Transport set as “Default” set Qualifier as RemoteDebug@<remote machine name>. Hit [Refresh] which will display all the processes hosted on the remote machine
e) Go back to the remote machine and exit the msvsmon service. Start the service again, but this time simply double click the executable and not do a run as. This will run the service under your login name.
f) Go to the host machine IDE and open Debug à Attach to Process. With Transport set as “Default” set Qualifier as <login name>@<remote machine name>. Hit [Refresh] and this should now display all the processes hosted on the remote machine.
Please note that if you try to debug by attaching the process in step (d), the debugging will not work as the IDE is run under your login name and not under RemoteDebug


Do not hesitate to drop me an e-mail with any particular observations you have had whilst trying to setup a remote debug environment. Enjoy!

Read More

Thoughts on the StarEast 2011 conference

I first attended StarEast in 1999. I remember the day-long tutorial I attended (by Rick Craig), and two track sessions - one by Cem Kaner on hiring testers, and one by James Whittaker on "Exploiting a Broken Design Process." I know I attended other sessions but I don't have active memories of them any more. I do remember the experience of attending the conference - one of surprise and excitement. Surprise at seeing so many other people in the testing community with similar questions and problems as myself, and excitement at the speakers with lots of great information and advice to give.

Fast forward to 2011 - I returned to StarEast, this time as a speaker. I suppose I didn't need to wait 12 years to return as a speaker. I didn't intentionally ignore the conference. I think I've been busy with other things and it just didn't come up - until last Fall when I received an invitation in my inbox to submit a proposal. I'm really glad I went.

Some things were familiar - the beautiful hotel, the Florida sunshine, the amazingly fresh orange juice, and the basic conference format. One thing that was different for me this time around was the number of people/speakers that I new who were also speaking at the conference. After having attended and spoken at several other conferences over the years, I guess I have gotten to know many of the popular speakers.

I was happy to see many more people speaking that I have never heard about before.  That tells me that the community is still growing after all this time and that there are still many more people sharing their knowledge to help enlighten future generations of testing leaders. That's awesome!


I was particularly surprised at the calibre of the Keynote speakers. I was genuinely inspired by every Keynote that I attended (I think I only missed one). (You can find the 2011 program online with details of the speakers and talks but I don't believe they were video recorded.) The Keynote presentations I attended were by: Andy Kaufman, Naomi Karten, and Julie Gardiner. The majority of my #StarEast tweets were from these keynotes.  Unfortunately, I missed the Keynote by Gojko Adzic (I heard it was good too), and I'm not including the "lightning" talks here although I did attend those and some were very good.

I was excited to hear that Lisa Crispin and Janet Gregory, authors of Agile Testing, were attending the conference and am happy to have had the opportunity to meet them and speak with them! =)  Lisa even attended my track session and wrote up a nice summary about it on the Software Testing Club site.

I won't really say much about my session here. It went okay I guess. I haven't received the session feedback evaluations yet, but I can say that I was really happy when several people came up to me afterwards in the hallway to thank me for my talk. They said that they really enjoyed it. My favourite comment came from Nawwar, my track chair -- i.e. the person who introduces the speaker during the session. He went from asking me "who are you?" just before the session to "Wow. That was the best session I've seen during this whole conference!"  Nice. Thanks. =)

I had fun giving the talk - "Real-time Test Design for Exploratory Testing". Test Design is a topic that I am really passionate about and I can talk about for hours. Okay, days. ;-)  I think the main thing that I wasn't terribly crazy about my talk was the format. When I talk about Test Design, it is usually when I am teaching it. So I found it hard to just talk about it and not have an interactive session with the attendees to give them a chance to practice it. Don't get me wrong - I had some interaction in my session, but not the sit-in-front-of-a-computer-and-try-things-out kind of way.  If I give this talk again, I will try to find some way to make it more interactive. (I may need more glow-in-the-dark straws.)

Back to the conference.  I was pleasantly surprised to bump into a former employee - a tester on a team that I managed about a decade ago. Wow! Still in Testing after all this time. AND attending a conference too! Double rainbows! We've hooked another one! ;-)

I've worked with many testers over the last decade and I hear from so few of them. I am always happy to meet them again and know that they are still doing and learning about Testing. I am happy to know when I help testers get better at what they do - I get ecstatic when I find out that they continue to see Testing as a Profession and participate in conferences and networking events!

StarEast had a few more surprises for me. At the Vendor Expo I bumped into Rick Craig - the tutorial speaker from when I first attended StarEast in '99. I introduced myself and told him how I remembered him. He was both pleased and suddenly felt older. Ha ha. We had a good chat.

Continuing my stroll through the Expo, the vendors were vendors. I still think some of the tools are the wrong ones for testers (like, totally and completely without value) but I did see some interesting ones that might have potential in certain circumstances.

As an independent consultant, most vendors weren't interested in me. I don't represent some big company with deep pockets to shell out on their bloatware documentation tracking systems. Of all the vendors, only one stopped to talk to me and really understand what it was that I had difficulty with their tools. He was fascinated by my knowledge of Agile practices, how testers fit on Scrum teams, and why these tools don't help. He asked me for my card and said that he was interested in contacting me to see if I can help provide some feedback for a future generation of tools that might work better. I gave him my card. I'll be curious to see if he contacts me. =)  I love to help - just ask!

My final pleasant surprise, and ultimately the best reason for attending the StarEast conference, came between sessions and during the meal breaks. I made an effort to sit with people I didn't know and engage them in conversations. Mostly I was curious to know what they did and what brought them to the conference. I was blown away by the passion that many of these attendees have for Testing. Wow. Some described to me the problems they face at work, the inequalities from other development team members (both in status and in salary), and the barriers preventing them from doing more. And yet, here they were at the conference to "learn from the best", to find ways to provide more value, make life better for them, their teams and their organisations.

They were dripping with hope and enthusiasm and I must say that I was overwhelmed with joy on more than one occasion following a conversation like this. If I had nothing else nice to say about the StarEast conference and received no other benefit from attending, witnessing the passion of the attendees was enough to fill my heart with passion and drive to get out there and do more, teach more!

I have been exposed to a lot of negativity over the last decade in several of the companies I have worked at. It wears you down after a while. You begin to lose faith in yourself and in the [testing/quality/value] mission. Attending StarEast changed that for me and recharged my spirit in a way that I totally wasn't expecting.

I was asked if I planned to speak at StarEast again next year. I don't know. I hadn't planned on it. I might not in 2012. We'll see about the following year or maybe I will speak at one of the other SQE conferences - like Better Software, Agile Development Practices or maybe even StarWest (since I haven't been to the Pacific coast yet).

There is so much to do in this area, in my "local" part of the world. It seems I am gaining recognition everywhere but here and I need to do something about that. StarEast helped recharge me and I hope I enlightened or entertained some attendees of my track session on Test Design and Exploratory Testing. Time will tell if I had a lasting impression on anyone. I hope I did. That wasn't why I went, though, and I got so much more out of it than just the opportunity to share some of what I have learned over the years.

I'm glad I went.

Read More

Reflection on my Testing workshop at the KWSQA Targeting Quality Conference

At this year's KWSQA Targeting Quality conference I gave a half-day workshop titled "Exploratory Testing Basics".  I originally proposed that title since I thought it followed nicely from the shorter workshop I gave at the QAI TesTrek conference in Toronto last Fall. I thought to myself - I'd like to redo the exercises again, change up a few things and it should be a piece of cake.

As the Winter months progressed into Spring, I began to worry about my workshop idea more and more.  You see, the exercise I gave at the QAI conference, while fun and appropriate, only really covered one aspect of Exploratory Testing - a broader framework. Perhaps that isn't enough?  What is enough, then? What makes up the "basics" of ET?

You see, when I teach ET, it's usually one-on-one and I spend 2-3 days just to cover the basics.  It takes me a few more days of pair testing and debriefing/coaching to help the new tester put everything into practice. It really is quite complex and a lot of ideas and models may seem abstract until you try them out and adjust with good feedback.


One of the hardest parts, I feel, is trying to teach certain techniques when the tester doesn't see a need for it.  For example, Pairwise Analysis.  I was introduced to Pairwise Analysis as "Functional Analysis" about 12 years ago and I got it right away. I had done more complex mathematics in university, so it wasn't the math that was the hitch for me - it was knowing when it might be useful and then applying it.

The first project I tried it on was when I was asked to perform Installation Testing for a desktop application. If you have ever done this kind of thing, you will know that there are *many* features and variations that all conspire to convince you that it is a daunting task that may consume every waking moment for weeks on end if you ever want to try and cover all of the possible combinations of systems, hardware, software, feature selections, and so on.  Enter Functional/Pairwise Analysis. I did the math; came up with a set number of test scenarios; performed them; and reported my findings in record time -- only a few days instead of the customary 2 weeks that it had taken on previous releases.

That was really cool! I had a new tool in my Tester's Tool belt and I couldn't wait to try it out again.

Years passed and several failed attempts later to teach other testers this cool technique, I finally stumbled upon the idea of Just-In-Time teaching. That is, rather than try to teach a tester all the techniques and models that I have learned over two decades and cram them into a few days, I will wait until they are presented with a problem and introduce the appropriate technique then.

There are two important take-aways for me with this JIT approach. First, it is really effective and the tester gets it - great! Second, it may take a long time before a tester is presented with the situations where certain techniques apply - not so great.

Present day.

So, what do I cover in a 3-hour workshop that I would consider 'good enough' to cover the basics of ET?

The answer, of course, is that proper workshop coverage will be the gap in knowledge from where someone is to where you want them to be. Unfortunately, the knowledge/experience starting point for each individual attending my session will be vastly different, their needs for this information will be different, and unless they have specific concerns some of the ideas very likely won't stick.

There are many unknowns in that equation. So how can I plan an outline to cover this unknown gap for unknown purpose(s) in a short amount of time? Stress.

What did I decide to do? I didn't plan an outline. I took a page out of Jurgen Appelo's book and I had the attendees self-organise and decide.

I handed out index cards and asked each person to create a user story card with their goal for the session. We stuck them to a wall with the heading 'Backlog' on it. On another part of the wall I had a Task board with the headings 'To Do', 'In Progress' and 'Done'.

I asked all the attendees to decide as a group, select the top 4 cards that they wanted to cover this session, and place them in the 'To Do' column. They didn't believe me at first when I asked them to do this and kept asking me to decide. It was really cool to watch the transformation happen and have them take ownership for deciding on the workshop goals.

I read each card, picked one, moved it into the 'In Progress' column and began.

So what did I actually cover? The attendees grouped most of the cards together into one big group and the one they picked said that they wanted to "gain an understanding of ET techniques."

Several years ago, I worked in a Financial Services organisation and was faced with a few audits by Banks where they needed to understand what testing artefacts we produced and why they didn't match their traditional Test documentation expectations.

Knowing that I couldn't just show them test session notes, test guides and other Exploratory Testing artefacts because they wouldn't understand what they were looking at, I created a presentation that had 4 parts:

1. The Challenges in Testing
2. What is Agile Development?
3. An Overview of our Systems Testing Approach
4. Examples of Testing Artefacts we Create on Software Projects

The auditors wanted to see the last part, but I explained that I needed to cover the first three before they could understand what they saw.

In the "ET Basics" workshop at the KWSQA conference, I covered the important aspects of the first three sections above (I have expanded upon the original presentation over the years), and supplemented that with some additional exercises to cover a critical aspect of Exploratory Testing - Test Design.

Did I meet the objectives? I think so. I covered the underlying principles and ideas that form the basis of good testing, "exploratory" or otherwise. The next step would be to look at some specific models and structures and have them perform exploratory testing.

While performing ET may be an exciting and enlightening activity, it is also a complex and challenging one that requires careful debrief.  In other words, we will need more time.

Will I give this same presentation again? Not sure. I might if that's what the attendees want to see. I'll let them decide.  If they want to practice something instead, or focus on managing such a testing activity, then we will do exercises for that instead.

I had fun in this workshop and learned some great things from an exercise I tried for the first time. I also thought the training video I showed was an appropriate fit for both a Friday afternoon and the topic covered. ;-) I can't wait to build upon what I've learned from this workshop experience and offer more in the future!

Cheers! Paul.

Read More