The Future(s) of Software Testing

This topic keeps coming up in various discussions so here are some thoughts on what I think a future may hold for us one day.  What does it mean to predict the future? Does it mean it is inevitable? If it is something appealing, maybe it is something we can work towards.

What does it mean to talk about the Future of Software Testing without talking about "Quality" (whatever that is)? I believe that Testing is a means to helping you attain the desired quality but that on its own it is not an indicator of what the delivered quality will be. I think it is fair to speculate on the practice of Testing in isolation of the idea of Quality. Just like it is fair to speculate on the kind of vehicles you use for travel without saying it is a clear indicator of the quality of the journey.

When it comes to predicting the future, how far ahead do we look? It used to bother me that H.G. Wells chose to go 800,000 years in the future in his famous work, The Time Machine. Some authors go 100 years or even just 5 or 10 to tell their tale. I will not be writing about what will happen in 5 years time and I really hope it doesn't take 100 years. I don't know how long some of these events will take to manifest. I do have some idea of 'markers' that we may see along the way though.

When it comes to Software Testing, two thoughts jump immediately to mind: 1) Software Testing is an inseparable part of the creative Software/Solution Development process; and 2) there will be many different possible futures depending on how you do testing today.  Put another way, there is no one right way to solve a problem, and creating software is a complex problem-solving activity performed in shifting organisations of human interaction so there are many ways to do it. In my opinion, the technical 'fiddly bits' like code, tests and user documentation pale in comparison to the difficulty of those two primary challenges: (1) solving the real problem, and (2) working with others to get the job done.

When we ask what is the future of software testing, we are really asking what is the future of software development. So what will Testing look like in the future? Well, what does Testing look like today? What does Development look like today?


Scenario 1: No (internal formal) Testing
For some, testing is only done by the end users. This may be a single individual, a small group of people or a larger population depending on the application. For example, if you have a highly-specialised application, the best testers may be the domain experts themselves (e.g. researchers, scientists, and so on). Aside from some quick checks made during development, no formal testing is performed prior to release of the software to the end user(s) for evaluation.

In some cases, non-critical software applications that people can live without if they fail are sometimes also released without any special internal testing phases. I have found that this often happens when a company is in a new niche market and they have no competitors yet. These applications may be prototypes or ways to try to figure out user/market value. If the apps stop working, people can continue on without them.

I don't see the future of this kind of software development changing much. No (internal) Testers are affected in the present or future of this kind of development shop. I believe with the advances in development technologies, we will see the quality of deliverables improve over time although the activity of determining suitability or fitness for purpose will always remain. That is, specialised software will always require the approval of the expert customers. They are the testers here.

Scenario 2: Programmer-driven-testing
I believe that "testers" are "developers" by definition, so rather than saying "developer-testing" (which would be confusing) I will say the code jockeys are the ones who own and perform the testing here. This is different from the scenario above in that formal test frameworks are in place and testing happens prior to release to the customer or users.  There are no separate testers or test teams within the company to take a second look at the system to make sure it "looks right" (big air quotes here). Many Agile Development shops operate this way, especially smaller ones.

Again, as there are no separate testers to speak of here, the future here won't seem very surprising. I expect things will pretty much look the same as they do today - only the tools will become more sophisticated. (more about that below, in the next scenario)

Scenario 3: Functional System Testing
Unfortunately, this makes up a large part of the software development companies out there with QA/Test teams - the "traditional" software testing. It depends on the context of course, however my experience has been that this activity is largely a waste of time and money and is performed more for show and to satisfy lawyers (i.e. copious test paperwork as "evidence" of due diligence) than it is to actually raise software quality.

Anyone today whose job is to create mountainous test documentation that slows the creative development process, creates division and mistrust between project team members, and only serves to check that what has already been built has been "built as specified" is completely wasting everyone's time and money.  I believe the term here is WOMBAT - waste of money, brains and time.

The future here is easy to predict. This tester role will completely disappear in the future as it is "wasteful" (in "Lean" development terms) and provides no additional value to the development process or solution quality. Sorry kiddos - adapt or die.

What will trigger this future? Simple. Responsible development practices and smarter development tools. When you take a look at the value that these test cases (i.e. usually very narrow-focussed functional "checks") provide, they are generally standard, straightforward things like:

1. Is the feature element (button, field, widget, text, whatever) that you said is supposed to be there actually there?

• Test-Driven Development (TDD) is a development practice (available today!) that completely eliminates the need to have separate testers do this kind of thing manually. There is simply no good reason for this testing activity to continue in a manual fashion into the future. It shouldn't even be done manually today!
• (TDD-style) Automated functional "checks" directly linked to the code are way more efficient to maintain. They also facilitate good quality deliverables that don't degrade unexpectedly over time.

2. Do input fields allow unexpected inputs?

• I believe that (future) advanced development tools (programming languages, compilers, etc.) will automatically include model-based testing (MBT) subroutines that will automatically scan for such trivial aspects of developed code.
• There is really nothing magical in this kind of testing activity. Yes, it finds a lot of really good bugs. The only perceived "magic" here comes from inexperienced programmers who don't know how to develop better solutions.

Generally speaking, I don't know when programmers stopped taking responsibility for such basic testing and checking activities as part of their coding tasks. If you go back to the 1960's and 70's, there were no separate testers - programmers did it all. This is easy stuff. I really believe that if programmers had continued to "own" this kind of testing, it would have been part of the development tools by now.

We took one step forward in the 60's and 70's and then two steps backwards in the 90's and 2000's. It's no wonder that the "agile movement" is trying to move programmers back in the right direction. When this quality/value ownership in development becomes more widespread, I will be happy to report that we will have achieved 1960's-level of development craftsmanship. Again. sigh.

Scenario 4: Business Analysts, Personas and Suitability Testing
Testing in some companies happens with BA's or other specialists who act on the customer's behalf to check that the application or system developed (SUT) meets the expected fitness for purpose. That is, does the SUT meet the business needs (rules, SOP's, statutes, industry standards, and so on) of the customers or users? These types of internal testers generally have some kind of industry or domain experience or knowledge.

In testing jargon, this is more "validation" kind of testing or "did we build the right thing?" Sometimes this is done with the help of "typical user profiles" called "personas." In the absence of formal requirements or tests, one can ask the question "what would user X do in this situation?" This kind of testing has more of a basis in business and psychology and doesn't presently lend itself to automation very well.

I believe that in the future, these kinds of tests will be automated as well, using intelligent systems and algorithms that can calculate the percentage probability of a developed solution falling within the desired user parameters.

I believe that the Eureqa system provides us with a glimpse of what intelligent computer systems can do today. With advancements in hardware technology, I believe it will be practical for humans one day to interact with computers via natural-language voice control and have the computers do these kinds of checks for us. The "comparing system" or "oracle" will group business rules together into a model, run through the SUT and heuristically compare the data output with the desired business model. At this point it is simple mathematics to let you know that the SUT meets approximately 72% of the desired model, include confidence limits, and tell you which rules the SUT fails to meet.

We're not there yet. If this describes your job, I'm pretty sure your job is safe for the next 5-10 years (if you are good at what you do). Development of this kind of "oracle" really depends on a lot of things, including how quickly we get through scenario 3 above.

Scenario 5: Life-Critical Systems
This is more than simply a variation of scenario 4 above. These kinds of systems are things like medical devices, nuclear/energy management systems, aerospace and deep-sea technologies, and so on. Basically, any system that if it fails someone will very likely die.

When people's lives are on the line, I believe the responsible action will be to always have people involved in the evaluation and assessment of the SUT.

Yes, I believe that advanced development tools and technologies such as what I mentioned in scenarios 3 & 4 will greatly improve the foundational quality of all systems developed - including life-critical systems. However, the role of the responsible development organisation here will be to have good, smart, skilled people own the field trials in a way that determines "fitness for use" at a level well beyond what I have already described above.

If my word processor is unavailable, I may get annoyed but I will find another way to write a message. If a pacemaker stops running after two weeks of use, you can be sure that the patient involved cares about this problem in a really big way. As will his or her family.

Scenario 6: Black-Box System Testing, Para-Functional Testing, Exploratory Testing
This is a superset of scenarios 3 & 4 and complementary to scenario 5.  In this role, the tester takes a look at more than just the functionality of the system and is trained to ask questions about the SUT and user expectations that sometimes challenges the current design of the developed solution.

A good Exploratory Tester identifies assumptions and asks open questions about them. For example, questions about the user experience, flow of data, security and privacy, internationalisation, reliability and many other facets that are often skipped or ignored in "traditional" software test teams.

I do this style of testing, and I know many good people who also do it around the world. Unfortunately, I also know that we represent a small percentage of all the test teams out there.

I believe that this testing role fills an important niche in the software development "creative problem-solving" activity that is currently lacking in many companies. That is, we go back to the important question: are we solving the right problem for everyone who matters?

I see two possible futures here. First, if people are still involved in the development process, I believe that this testing role will be split in two parts. The hands-on testing part will be automated using the advanced development tools I already described above. We can simply add new MBT subroutines to the tools to account for new personas, perspectives and potential problem types. The creative, investigative people-interaction part will still need a skilled person to go around, talk to people and ask the right questions. This will lead to creating the right tests for the tools to help us answer.

In the second possible future, if people are not involved in the software development process, this whole testing activity will be automated in a fashion similar to what I described in scenario 4 above.

Scenario 7: Specialised tests - e.g. Performance, Usability
This is a variation of scenario 6 above.  A specialised test is one that answers a specific class of questions. My experience in Performance Testing tells me that how I do this kind of testing is different from other kinds of testing. At the end of the day, there are a set of rules and models that apply to do this kind of testing properly. We will be able to program or teach these rules to an "oracle" system at some point in the future.

Human research and expertise will go into the problem-solving models that we will program into these computer systems.  Depending on the complexity of the development project, the oracle system may be able to ask the appropriate questions and execute the tests without further prompting. In some cases, I expect the initial questions will come from a person and the computer system will be able to perform the test and report the results as required.


To sum up, I see the future of software development as being much more plug-and-play than anything we have today. Software testing activities will be largely automated with the possibility that humans may still be involved in asking important questions that lead to suitability or fitness for purpose. In time, I think that learning computer systems will be able to anticipate those kinds of questions and they will free us up to do different, more interesting creative work.

Don't worry, testers. Your jobs are safe as long as the programmers' jobs are safe - provided you are contributing value to the development activities. I believe our "developer" roles will disappear in close tandem. Software development and engineering is still a relatively young field/industry. We still have a lot of growing up to do.

Read More

GAC those assemblies

              With a lot of emphasis on unit testing during development, developers sometimes spend time in bursts for deploying assemblies to GAC. The spent time though insignificant in bursts becomes considerable when considered cumulatively as a percentage of the total deployment time during development.

Following are a couple of ways to deploy assemblies to GAC on the fly and save that little percentage:

      (1)   Right Click and Deploy

      How cool would it be to have an option of right clicking an assembly and deploying it to GAC. Really cool hey!

To do this Open your registry and add a key “HKEY_CLASSES_ROOT\dllfile\shell\Add to GAC\command” and set its value to "D:\\Program Files (x86)\\Microsoft Visual Studio 8\\SDK\\v2.0\\Bin\\gacutil.exe" /i  "%1".

Refresh and voila you get an option on right click to add the assemblies to GAC

      (2)   Post Build Event  

      I have been a big fan of post build events and have been using it to do move files around.

One of my favourites is obviously the one that GACs the assemblies.

Include the following script in the post build event to GAC your assemblies:

echo "Installing assembly to GAC"

"$(DevEnvDir)..\..\SDK\v2.0\Bin\GacUtil.exe" /i "$(TargetPath)" /f

Enjoy!!

Read More

Outlook Macro: Move read messages from inbox to folders

Being handed over a blackberry from my employer recently to monitor support e-mails, what I found out was that any e-mails that were picked by my rules and routed to some folders did not show up in my BB inbox. There are some settings that you can use on the handheld or the BB desktop manager to start syncing those folders as well. However since the BB was admin owned by my employer I could do neither of them.

So the plan was to disable these rules, let all mails get routed to my inbox and then have rules to route them to folders once I have read the rules or at-least it shows on BB. Unfortunately MS Outlook does not support rules which would run once you have read e-mails. They only get evaluated against mails coming from the exchange server. So I need an outlook macro script which would kick in every say 30 mins and move all messages from my inbox meeting the rules criteria to specific folders. After spending a few unsuccessful hours on the web trying to land an out-of-the box script I decided to write my own. It turned out to be easy, fun and educating. As a do-gooder I would like to share the macro script with you to save you a few hours…

Put the following script in ThisOutlookSession of your macro:

Private Sub Application_Startup()

Call Timers.ActivateTimer(30)  ‘This sets timer to 30 mins, It can be set to any value

End Sub

Private Sub Application_Quit()

If TimerID <> 0 Then Call Timers.DeactivateTimer ‘Please do not exclude this step, this will kill the timer we created

End Sub

Put the following script in the module of your macro:

Option Explicit

Declare PtrSafe Function SetTimer Lib "user32" (ByVal hwnd As Long, ByVal nIDEvent As Long, ByVal uElapse As Long, ByVal lpTimerfunc As LongLong) As Long

Declare PtrSafe Function KillTimer Lib "user32" (ByVal hwnd As Long, ByVal nIDEvent As Long) As Long

Private TimerID As Long 'Timer object

Public Sub ArchiveIt()

Dim ns As Outlook.NameSpace

Set ns = Application.GetNamespace("MAPI")

'Open inbox folder

Dim folder As Outlook.folder

Set folder = ns.GetDefaultFolder(olFolderInbox)

Dim destinationFolder As Outlook.folder

'Move all mails from xyz@abc.com to Inbox/Production Support/xyz

Set destinationFolder = folder.Folders("Production Support").Folders("xyz")

MoveMailsByFilter destinationFolder, folder, "[SenderEmailAddress] = 'xyz@abc.com'"

End Sub

Private Sub MoveMailsByFilter(ByVal destinationFolderHandle As Outlook.folder, ByVal inboxFolderHandle As Outlook.folder, ByVal filterText As String)

Dim filteredList As Outlook.Items

Set filteredList = inboxFolderHandle.Items.Restrict(filterText)

For i = filteredList.Count To 1 Step -1

filteredList.Item(i).Move destinationFolderHandle

Next i

End Sub

Public Sub ActivateTimer(ByVal nMinutes As Long)

nMinutes = nMinutes * 1000 * 60 'Convert time to milliseconds.

If TimerID <> 0 Then Call DeactivateTimer 'Remove any existing timers.

TimerID = SetTimer(0, 0, nMinutes, AddressOf TriggerTimer)

If TimerID = 0 Then

MsgBox "Distributor failed. Contact administrator"

End If

End Sub

Public Sub DeactivateTimer()

Dim lSuccess As Long

lSuccess = KillTimer(0, TimerID)

End Sub

Private Sub TriggerTimer(ByVal hwnd As Long, ByVal uMsg As Long, ByVal idevent As Long, ByVal Systime As Long)

Distributor.ArchiveIt

End Sub

You can play around with the different properties exposed on filteredList.Item like filteredList.Item(i).Unread = false (to pick only read messages) to fine tune your archiving criteria. Enjoy!

Read More

ESB Toolkit 2.1 installation on BizTalk Server 2010

Pre-requisite for ESB 2.1

·         Microsoft Windows 2008 Server SP1/ SP2

·         .NET framework 3.5 SP1

·         IIS 7.0

·         Microsoft SQL Server 2008

·         Microsoft BizTalk Server 2009/2010

·         Visual Studio 2008 SP1/ 2010

·         Microsoft Chart Controls for .NET Framework 3.5 (This would be used by ESB Portal)

(http://www.microsoft.com/download/en/details.aspx?id=14422)

·         Microsoft Enterprise Library 4.1

Note: If your machine has Microsoft Enterprise Library 5.0 installed it would be advisable to un-install 5.0 and install Microsoft Enterprise Library 4.1 (http://www.microsoft.com/download/en/details.aspx?id=6228) as the ESB solution targets 4.1 version.

     1)    Install Microsoft ESB Toolkit 2.1 (http://www.microsoft.com/download/en/details.aspx?id=11847)

This will install the tools and samples.

     2)    Navigate to Start à All Programs à Microsoft BizTalk Server 2010 ESB Toolkit

Click Install Itinerary designer extension to register the itinerary designer extension in Visual Studio 2010

     3)    Verify whether the installation was successful by;

Open Visual Studio 2010 and check whether you see an option to create an ESB itinerary

     4)    Open BizTalk admin console and;

·         Import Microsoft.Practices.ESB.CORE64.msi into the Applications node (remember to select Overwrite resources). Continue the installer to install the assemblies into GAC.

If you get an error like;

There is a problem with this Window Installer package. A program required for this install to complete could not be run. Contact your support personnel or package vendor.

This implies there is a problem GACing the assemblies. At this point all assemblies except Microsoft.Practices.ESB.ExceptionHandling.PipelineComponents.dll are in the GAC.

o    To GAC the above, navigate to the folder where the installer had dumped all the assemblies which were to be GACed. Default location is "C:\Program Files (x86)\Generated by BizTalk".

o    Navigate to C:\Program Files (x86)\Generated by BizTalk\Microsoft.Practices.ESB\<GUID Folder>\ and extract ITEM~14. This will have Microsoft.Practices.ESB.ExceptionHandling.PipelineComponents.dll which can be dragged to the GAC.

·         Clear "C:\Program Files (x86)\Generated by BizTalk" folder

·         Import Microsoft.Practices.ESB.ExceptionHandling64 into the Applications node (remember to select Overwrite resources). Continue the installer to install the assemblies into GAC.

If you get an error like;

There is a problem with this Window Installer package. A program required for this install to complete could not be run. Contact your support personnel or package vendor.

Ignore this error as all required assemblies have been GACed.

·         Clear "C:\Program Files (x86)\Generated by BizTalk" folder

·         Import Microsoft.Practices.ESB.JMS into the Applications node (remember to select Overwrite resources). Continue the installer to install the assemblies into GAC.

If you get an error like;

There is a problem with this Window Installer package. A program required for this install to complete could not be run. Contact your support personnel or package vendor.

This implies there is a problem GACing the assemblies. At this point all assemblies except Microsoft.Practices.ESB.JMS.PipelineComponents.dll are in the GAC.

o    To GAC the above, navigate to the folder where the installer had dumped all the assemblies which were to be GACed. Default location is "C:\Program Files (x86)\Generated by BizTalk".

o    Navigate to C:\Program Files (x86)\Generated by BizTalk\ Microsoft.Practices.JMS\<GUID Folder>\ and extract ITEM~0. This will have Microsoft.Practices.ESB.JMS.PipelineComponents.dll which can be dragged to the GAC.

·         Clear "C:\Program Files (x86)\Generated by BizTalk" folder

·         For safety place the pipeline components assemblies i.e.

o    Microsoft.Practices.ESB.JMS.PipelineComponents

o    Microsoft.Practices.ESB.ExceptionHandling.PipelineComponents

; Into "C:\Program Files\Microsoft BizTalk Server 2010\Pipeline Components"

     5)    Navigate to "C:\Program Files\Microsoft BizTalk ESB Toolkit 2.1" and extract ESBSource.zip file. This would create the ESBSource solutions folder with a Keys subfolder having the assembly key files and Source subfolder with the actual ESB solution

     6)    We need to create an assembly key file to be used by the ESB solution. To do that open Visual Studio command prompt and type in;

sn -k "C:\Program File\Microsoft BizTalk ESB Toolkit 2.1\ESBSource\Keys\Microsoft.Practices.ESB.snk"

     7)    Navigate to "C:\Program Files\Microsoft BizTalk ESB Toolkit 2.1\ESBSource\Source\Samples\Management Portal\ESB.Portal"

Open ESB.Portal solution and re-build the solution.

Do NOT proceed unless the solution builds successfully. You may have to fix some references to get the solution in a compiled state.

     8)    Next we have to change the script execution policy as we are going to run some scripts to install the portal solution.

Navigate to Start à All Programs à Accessories à Windows PowerShell and click on Windows PowerShell.

·         At the prompt type get-executionpolicy which will display the current execution policy.

·         If it says Restricted we have to change it to Unrestricted as follows:

o    At the prompt type in set-executionpolicy Unrestricted

Since the portal solution scripts may target any version of Windows PowerShell we have to set the execution policies for both the x64 and x86 versions. So,

 Navigate to Start à All Programs à Accessories à Windows PowerShell and click on Windows PowerShell (x86)

·         At the prompt type in get-executionpolicy which will display the current execution policy.

·         If it says Restricted we have to change it to Unrestricted as follows:

o    At the prompt type in set-executionpolicy Unrestricted

      9)    Navigate to "C:\Program Files\Microsoft BizTalk ESB Toolkit 2.1\ESBSource\Source\Samples\Management Portal\Install\Scripts" and double click on Management_Install.cmd

This would install the portal and create all the required virtual directories in the IIS

     10)  Open IIS and make sure ESB.Portal virtual directory is created under Sites\Default Web Site.

Ensure that it is pointing to the correct folder i.e. "C:\Program Files\Microsoft BizTalk ESB Toolkit 2.1\ESBSource\Source\Samples\Management Portal\ESB.Portal"

     11)  Select ESB.Portal node and in the page on the right hand side double click Authentication.

Ensure that Windows Authentication is Enabled.

     12)  Navigate to "C:\Program Files\Microsoft BizTalk ESB Toolkit 2.0\Bin\" and double click on ESBConfigurationTool.exe.

Note: ESBConfigurationTool works in a linear way i.e. Exception Management, Database is configured first followed by Exception Management, Exception Web Services followed by ESB Core Components, Itinerary Database followed by ESB Core Components, Core Web Services and finally the Configuration. Also each of the features have to be configured separately i.e. Hit "Apply Configuration" after each step rather than trying to configure all features in one go.

·         Open ESB Configuration à Exception Management à Database

o    Check Enable Exception Management Database and specify the Database server name

o    Click on Apply Configuration

·         Open ESB Configuration à Exception Management à Exception Web Services

o    Check Enable Exception Services

o    Provide the user name, password details and Click on Apply Configuration

·         Open ESB Configuration à ESB Core Components à Itinerary Database

o    Check Itinerary Database and specify the Database server name

o    Click on Apply Configuration

·         Open ESB Configuration à ESB Core Components à Core Web Services

o    Check Enable Core Services

o    Provide the user name, password details and Click on Apply Configuration

·         Open ESB Configuration à Configuration

o    Make sure File Configuration Source is selected. (This option will save all the settings to ESB.config file)

     13)  Add all portal user names to BizTalk Isolated Host Users, BizTalk Application Users, SSO Administrator and SSO Affiliate Administrator groups.

     14)  Deploy the portal bam tables. Open a command prompt and navigate to "C:\Program Files\Microsoft BizTalk Server 2010\Tracking". Run the following commands to deploy the bam tables:

·         bm.exe deploy-all -DefinitionFile:"C:\Program Files\Microsoft BizTalk ESB Toolkit 2.1\Bam\Microsoft.BizTalk.ESB.BAM.Itinerary.xml"

·         bm.exe deploy-all -DefinitionFile:"C:\Program Files\Microsoft BizTalk ESB Toolkit 2.1\Bam\Microsoft.BizTalk.ESB.BAM.Exceptions.xml"

If you get an error like;

Deploying View... ERROR: The BAM deployment failed. The BAM Star Schema database has not been configured. Run bm.exe setup-databases to configure the database.

It was because the BAM Star Schema and analysis services have not yet been configured.

o    Open Run and type in Configuration or alternatively open the BizTalk Server Configuration

o    Click on Bam Tools in the BizTalk Server Configuration window.

o    Check Enable Analysis Service for BAM aggregations  and specify an appropriate Server Name for Bam Analysis Database  (optional) and BAM Star Schema Database

o    Click on Apply Configuration to create the databases.

o    Before running the bm deploy command again we have to grant ourselves access to the newly created BAM Star Schema.

Open SQL Management Studio à Database Instance à Security à Logins and double click on <Your Installation ID>.

In the Login properties window select User Mapping node and check on BAMStarSchema.

In the Database role membership pane for BAMStarSchema check public, db_datareader, db_datawriter and Click OK

o    Run bm.exe deploy-all -DefinitionFile:"C:\Program Files\Microsoft BizTalk ESB Toolkit 2.1\Bam\Microsoft.BizTalk.ESB.BAM.Exceptions.xml" again

     15)  Exception management portal relies on ExceptionService.svc WCF service. IIS 7.0 installation does not have a script map for .svc files and hence they have to be registered explicitly as follows:

Open a command prompt and navigate to "C:\Windows\Microsoft.NET\Framework\v3.0\Windows Communication Foundation"

Run the following command: ServiceModelReg.exe -r –y

     16)  Restore the execution policy to Restricted i.e.

Navigate to Start à All Programs à Accessories à Windows PowerShell and click on Windows PowerShell

Run set-executionpolicy Restricted
Navigate to Start à All Programs à Accessories à Windows PowerShell and click on Windows PowerShell  (x86)

Run set-executionpolicy Restricted

     17)  Restart IIS and SQL Server.

     18)  Verify whether ESB 2.1 installation-configuration was successful by opening  a browser with the following url:

http://localhost/ESB.Portal/Reports/Reports.aspx

Other Errors

      1)    On opening the ESB Management portal if you get an error message pointing to the event viewer and the event viewer message says "assembly Microsoft.Practices.ESB.Portal.Global not found", this means that the ESB portal solution build has been un-successful i.e. Step (11) has to be re-visited

      2)    On opening the ESB Management portal you get an error message pointing to the event viewer and the event viewer message says "could not load type System.ServiceModel.Activation.HttpModule", this means that because of the order of the software installation on your machine i.e. IIS installed after .Net framework the .net version has not been registered with IIS.

To fix this problem;

Open a command prompt, navigate to "C:\Windows\Microsoft.NET\Framework\v4.0.30319 (or the version installed on your machine)" and run the following command

aspnet_regiis.exe –iru

      3)    If there are still any problems with the installation try the following links
      http://dgoins.wordpress.com/2010/05/01/esb-toolkit-management-portal-installation-notes-from-the-field-2/

http://msdn.microsoft.com/en-us/library/ee236738(BTS.10).aspx

Read More

Pre-requisite for BAM Portal

One of the important pre-requisites before trying to configure BAM portal on your machine is the installation of IIS 6 extensions. Following steps should be followed before configuring the BAM portal via the BizTalk configuration tool.

·         Right click My computer  and click Manage

·         Expand Roles and select Web Server (IIS) node

·         Scroll through Role Services and check status of IIS 6 Management Compatibility role service

·         If status is Installed then close the windows

·         If status is Not Installed then right click Web Server (IIS) node and select Add Role Services

·         Select IIS 6 Management Compatibility node and complete the installation

Read More

Hobbies and Interests

Several years ago I wrote an article summarising some of the key points I keep in mind while interviewing candidates for a test team. The article is called "Hiring Software Testers in an Information Age" and is available as a PDF on my main site. The article was originally targetted to recruiters who kept asking me for advice on hiring software testers and they would always be surprised at the level of detail that I went through in describing what it takes to hire a good person for a testing position.

Conversations with recruiters over coffee would always start the same. I would say something like: if you are just trying to find a warm body to fill a position, then you don't need to hear what I have to say.  If you want to hire someone who thinks and has a good chance of fitting in with the culture of the team and organisation to provide value, then it is a complex problem that requires insights into what the position actually involves.

There are about a dozen different checkpoints that I go through when considering and interviewing candidates, and the paper I wrote touched on some of the major points but not all of them.  Actually, I even removed some of them from the article as early drafts had too much information.  My intention was to get some of the important points across without writing a book.

Recently, a colleague and friend, Michael Mahlberg tweeted the following:

RT @NolanBushnell: At Atari we hired based on hobbies and not grades in school. We ended up with the best engineering group in the world.

I liked that comment and followed up with a supporting tweet:

On Hiring: if a résumé or cover letter doesn't describe Hobbies or other Interests, I usually skip it.

This sparked some conversation on twitter and I want to elaborate on my comment here.


First off, it is important to know that I place myself in the Context-Driven School of Testing. The principles of this 'school' puts the focus on the people working together to help deliver the right solution.  One of the things I have noticed over the last 22 years in the software industry is that the best testers are the ones who care about testing. They have fun with it. They believe they are working to improve things for the customer. They have a drive and motivation that lets them ignore or put up with a lot of crap that is dumped on them by unhealthy organisations and ignorant individuals.

Unfortunately, that kind of motivation, passion or drive doesn't come across on a standard résumé. I can spot it straight away when I am talking with someone, but how do you communicate it in a standard functional résumé?  In general, I don't see it in the "technical skills" or job description sections that focus on accomplishments and other task-oriented details. The majority of the time I notice evidence or hints of passion and motivation in the cover letter, if anywhere at all.

So, when I am hiring for a test team, a team that I want to integrate well with the rest of development and the organisation, a team that I want to focus on building human relationships as well as exercise systems and scientific thinking in their quality investigations, a team that I want to encourage fun and respect for their hard work in providing valuable information to help make timely decisions, where do I start when I am looking at a stack of résumés and job applications?

People who submit cover letters go to the top of the pile. People who include "Hobbies and Interests" in their résumés are next. People who don't submit a cover letter and don't tell me anything other than a bunch of dry technical information and job details are put at the bottom of the list and often fall right off the pile.

Having a cover letter or a "Hobbies and Interests" section doesn't guarantee an interview but your chances of having me give you a quick call are higher. So, what's the deal with this "Hobbies and Interests" section anyway?

Johanna Rothman wrote an awesome book called "Hiring the Best Knowledge Workers, Techies & Nerds." If you don't have a copy yet, and you are in the business of hiring technical people, I highly recommend you get a copy of this book.  Johanna is an awesome person and a terrific writer. She writes a few blogs and I learn a lot from them.

Johanna wrote a good blog post back in 2004 titled "Tips for Reviewing Resumes". One of the points she wrote in that post says: "Hobbies or other personal information. This stuff isn’t relevant to the job and should not be part of how you select candidates."

I partly agree with this sentence. I really don't care about personal information in an application. By "personal information" I mean things like marital status, age, sex, or anything else that the government might use to classify you in their census demographics charts.

What about hobbies and other interests? Things like: playing music (e.g. piano or guitar), likes to read, play video games, do magic, go cycling, martial arts, improv, and so on.

This stuff I find both interesting and relevant! Why? Creativity, passion and (skill) transference.

I am looking to hire people -- intelligent, thinking, caring, fun people and not drones. I honestly don't know whether the stuff I'm reading on your application is true, embellished or fabricated. I want to get an idea of the whole picture of who you are. I will test you for your technical ability during the interview, so I'm not worried about that part.  Finding someone who has the right mindset and will fit in with the rest of the team is harder to grasp from technical details and job accomplishments alone.

Based on how I currently understand, apply and teach Software Testing, I like to find candidates who exercise both their creative and analytical parts of their minds -- i.e. people who are right and left-brained (to use a dated, flawed model of the human mind). When I read through the candidate's hobbies and interests, I build a list of assumptions that I can check in a phone call or in-person interview.

Assumptions might be things like:

• if I see someone who likes to do creative things like play music, sing, knit, et cetera, these are right-brained/creative activities. This is a good sign that the candidate might be good at Exploratory Testing as I do it.


• if I see someone who participates in theatre, improv or role-playing games, then they may be good at user profiling and test design.


• if someone likes to read, this may tell me that they are learners and continue to feed their imaginations. This is also a good sign for Exploratory Testing, and I am happy to ask them about the kinds of books they read.


• sometimes their interests may align very well with the interests of the organisation and the product or solution in development. Things like playing video games or participating in sports are really good hobbies to have if the hiring company makes games or develops solutions for the Sports industry. 

There are too many examples for me to list here. The point is that sometimes the Hobbies and Interests section can provide me with insights that I won't get from the Work Experience, Technical Skills or Education sections of a résumé alone. I look for evidence of creativity, motivation, passion, balance, feeding imagination, professional activities, community involvement, and so on. Sometimes, the skills they develop in their hobbies are directly transferable to the testing tasks required -- e.g. learning, observation, relationship-building, organisation, analysis, design, problem-solving, and so on. Sometimes they aren't - it depends on the individual.

These are all assumptions, I know, so I treat them like assumptions. I make note of them and ask about them in phone calls and interviews.

If I like the candidate and hire them, I use their hobbies/interests as examples when coaching them on testing theory, models and practices. Since the hobby is familiar to them, I find this is a really powerful teaching technique. I also encourage them to come up with their own testing analogies using knowledge and experiences they are familiar with. It raises their self-confidence and helps them to remember abstract ideas in their own terms.

The likelihood of me finding a Testing Superstar by skimming through résumés is pretty slim. The likelihood of me finding a candidate who I can help become a Testing Superstar is much higher if I can learn more about them in their job application; learn more about what makes them a unique, creative, passionate individual who cares about others, doing excellent work, providing value and enjoying life.

But that's just me. All interviewers are different. Your mileage may vary.

When in doubt, I suggest candidates be themselves. If you are penalised in a hiring process for it, you don't want to work there anyway. You are allowed to have a life. Work is just a part of it.

Read More

Quality Center Must Die

It is not a matter of "if" -- it is a matter of "when" HP's Quality Center software will die.  And you, my dear readers will help make that happen.

"How?" you may ask? Simple. There are two things you should do: (1) think, and (2) don't put up with crap that gets in the way of delivering value to the customer and interacting intelligently with other human beings.

But I am getting ahead of myself. Let's rewind the story a bit...

Several months ago I was hired by my client to help train one of the test teams on agile and exploratory testing methods. The department has followed a mostly Waterfall development model until now and wants to move in the Agile direction. (A smart choice for them, if you ask me.) Why am I still there after all this time? That's a good question.

After attending the Problem Solving Leadership course last year, and after attending a few AYE conferences, I changed my instructional style to be more the kind of consultant that empowers the client with whatever they need to help themselves learn and grow.  It's a bit of a slower pace, but the results are more positive and long-lasting.

I am a part of a "pilot" agile/scrum team and am working closely with one of the testers (I will call him "Patient Zero") to coach him on good testing practices to complement the agile development processes. I have done this several times now at different clients, so this is nothing new to me. One of the unexpected surprises that cropped up this time was that this development team is not an end-to-end delivery team, so when they are "done" their work, the code moves into a Waterfall Release process and it all kind of falls apart. There are still some kinks to be solved here and I am happy to see some really bright, caring people trying to solve these problems. So that's okay.


Patient Zero and I are part of a larger test team, and the rest of the test team all work on Waterfall-style projects, use Waterfall-compatible tools, and they generally don't get how we work. :) Unfortunately, one of the tools mandated for our team's use is HP's Quality Center (HPQC).  I hadn't seen that tool in about a decade and it looked very similar to how I last remember it.

To my agile coach/practitioner friends I should clarify that at no time during our sprint development work does anyone ever touch HPQC! However, once the code is deployed/falls into the Waterfall Release process, regression test cases are created in HPQC and it is used for defect tracking. It is mandated, and so shall it be done. I can live with that. It's just a tool at this point and the impact to our ability to deliver a good solution is eliminated by the fact that we don't touch it until after we are "done". (Communication and collaboration FTW!)

Two days ago.

Our whole test team took part in a 2-day HPQC training workshop on something HP calls "Business Process Testing" or BPT. Being naturally curious to learn something new, I wanted to know what BPT was and how it fits into the bigger testing picture. Here we go.

We were given a handout with some "test scenarios" to be used for training. The test scenarios fell into this pattern:

• Scenario name/title
• Requirement description
• Test Situation (I am staring at this right now and I still don't know what this means)
• Role (kind of system user this requirement/situation applies to)
• Steps

That's okay information. The "Steps" are what you might typically expect to see if you have been testing for a while. Here is an example for working with a sample web app:

1. Login to the system with (a certain type of user)
2. Navigate to some module in the app
3. Click "Create" button from the tool-bar
4. Enter mandatory field values and save
5. Search for the information you created in the previous step
6. Logout of the system

And there were 6 of these scenarios.

I read through these tests and then I tried to follow them using the system. I quickly encountered a half-dozen bugs - some with the system, some with the test scenarios/cases, and some were open questions that I would follow-up with the Product Owner for requirement clarification.

But, woah-woah-woah-hey.. wait a minute. We only need to worry about *these* documented test scenarios! I struggled hard to keep my mouth shut about the value of time and the many different kinds of tests I would happily engage in at this point if I could leave the tool alone. But, I left that to my "inner voice" and we were now one hour into the first day's training.

At this point, we were given an overview of the HPQC modules and told to (1) enter the Requirements, (2) create BPT test scenarios, and (3) "componentize" the test scenarios into groupings of related steps. This last part required some explanation since this was new to me, and once we got it, we set to work on the task.  Since we could see (1) and (2) on the handout sheets in front of us, we went straight to work on part (3). That was kind of fun working in small group of 4 looking at these scenarios and trying to come up with solutions.

And then someone went and spoiled the fun.  We were "told" that we weren't supposed to do part (3) until after we had entered the information for (1) and (2) into the tool.  I was all like "really?" and then a team lead came by and repeated the exact same thing.  This may be summed up as: Enter data into the tool first, and think later.

I was kind of shocked with this comment and attitude and in retrospect it kind of foreshadowed the rest of the training experience -- i.e. Tool and process first; think later; maybe.  Okay. I'll play along and see where this goes.

During this exercise, I began to grasp this idea of "components" as HP uses them. I think they are like Page Objects - chunks of code (or in this case, test steps) that perform a certain function, promote reuse, and reduce duplication. Although it was never described in this way, I believe the HP "BPT" module is a proprietary Do-It-Yourself DSL. Aha! I have experience with those. I get that stuff.

So, once I got it, I started to explain the concept of YAGNI to my group team members.  That is, let's not overthink or over-engineer these components. Let's build/write them based on the needs of the requirements in front of us. We will modify the components in the future as new requirements appear.  This idea was well received and we quickly came to an efficient solution for our small group.

When we took up the exercise, I found I had to explain the YAGNI concept to the instructor/trainer (and rest of the class/team) as he proposed that we try to abstract out these components to allow for compatibility with other features and system elements. What a waste of time! We cannot know what we will need beyond our immediate needs so that kind of abstraction is a pointless exercise that leads to more headaches than you need.

Eventually, I started to get that the point of writing the test scenarios using BPT was that these components form a basic vocabulary that may then be automated at some point in the future -- yes, BPT integrates with HP's QTP automation tools.  Now, I'm all in favour of consistency, clarity, reuse, and automating tests that humans should never have to do more than once (and if there is value in re-running the test), so I struggled to understand why it was never explained to us this way.  As long as I kept the DSL/automation model in my head, I understood what we were doing and can see the potential benefits of it.

I saw many testers struggle with the exercises and models we were presented with. End of day one.

Day two began with a quick recap and then we were introduced to the bureaucracy that is Waterfall and HPQC. That is, there are review processes and workflows to cover each requirement, BPT and component. Welcome to Wasteland. (I mean the Lean Development concept of "waste" here, although other interpretations of "wasteland" may be just as valid.) Easily a third of the 2-day training was spent on the processes surrounding the management and review of the various HPQC objects. sigh.

We then moved onto the concept of "parameters" for the components we created yesterday.  Okay, I get method parameters when I am scripting with Ruby, so this was no sweat.  Given the length of time I spent parametrizing my components compared to everyone else in the class, I think I may be one of the few who really got it.

I learned a few more things about the main instructor. One was that he didn't know how to identify web page elements using commonly available browser tools. Umm, aren't these tools supposed to interact with web pages? This never came up before? You have never wondered how to find the name/id for an element on a web page? really?

The other was that he had a really bad sense of humour.  He made a reference to a "QA" joke that I shall not repeat here.  Needless to say that I found it insulting, offensive, it made my skin crawl and my blood boil.  Many unpleasant feelings and ideas arose in me and it took all my willpower and strength not to react to the blatant stupidity of insulting the profession of the students in your class and the market that the HPQC tool represents.

The final blow for the day came to me when we tried to "execute" these BPT test scenarios using the HPQC tool. THEN I discover that these "parameters" can have 2 kinds of values - fixed/hard-coded and run-time. Anyone who has done intelligent automation knows NOT to hard-code values in their scripts. Data-driven is way better, and as an exploratory tester, I may not know what value I will choose until moments before.

Here's where the HPQC tool gets stupid..er.  Regardless which parameter type you choose, they may ONLY be defined BEFORE the test execution! You cannot leave empty parameters so that when you get to a particular step, the tester can decide what value to input and feed it back into the tool.

What does this mean?  As a tester, let's say you want to create a new user profile in an app of some kind. The test parameters for things like Name, Email, Address, Country, and so on, must all be defined and set before the test is executed. You cannot decide while you are actually testing!

Why does this matter to me? It matters because it means that the humans who execute these BPT tests manually have no choice in what values they may input. Testing techniques like Equivalence Classes and BVA that help guide our choices to pursue interesting paths are completely cut-off! It turns out that HPQC treats humans worse than the automated counterparts. In discussion with an automation "expert" at the end of the class I learned that at least you can code some variability into the QTP automation scripts.  This is not possible with the same test scenarios executed manually by human beings.

So. Much. Wrong.

So after my first ever QC training session, here are some of my take-aways:

• It took me 2 days to "script" 6 test scenarios in this tool and they were rotten test cases to begin with! I suspect that outside of the training environment, it will actually take longer to complete since you won't have a "reviewer" sitting next to you waiting for you to finish your piece.
• And they weren't even automated! Who knows how long it would take to tweak the "components" to make them work with a particular automation strategy.
• HP QC will never be a useful tool for any agile or rapid development efforts
• (HP best practice) Put the tool, data and review processes first, before you think. Maybe always instead of it too.
• BPT is a DSL framework for test scripting
• BPT component parameters cannot be customised during test execution. They may only be set/defined before you start testing. => No thinking allowed while testing.
• The instructor didn't appear to be knowledgeable on anything outside of the tool itself. This includes how we might actually want to use the tool. No, no, no. And I quote: "Testers must change how they work to use the tool in the way it was designed."
• As long as there are people pushing these kinds of horrible tools that suck the life and intelligence out of people, inject mountains of wasteful activities that provide no value to the customers or end users, and continue to create barriers between testers and their developer counterparts, I will always have job security in helping organisations recover from these cancers.

At the end of the day, there were several people interested in my idea of randomising tests. The automation expert in the class insisted that it couldn't be done with automation, so I called up my "Unscripted Automation" presentation slides that I gave a few years ago. He said that what I proposed was not a "best practice" and that everyone, the whole industry, was using the tools in the way that he described how they should be used.

My response was to simply say "they are all wrong."

Read More