User Acceptance testing - Highly misunderstood testing jargon/phase!!!!
Me and my team organized our first conference for this year (2012-2013). We started off with a bang. We brought in Scott Barber (http://www.linkedin.com/in/perftestplus) to start the conference season and it was great. There were about 60- 70 people in attendance and Scott was his usual self engaging the audience right from the start.He talked about performance testing and other various aspects of testing but one thing which stuck in my head and had a very deep meaning was "User Acceptance Testing". It was amazing the way Scott barber talked about it and I wanted to reflect some of his and my thoughts on it. User Acceptance Testing - This simple three word testing jargon has such a deep meaning and is one of the most highly misuderstood testing phases based on my personal experience. Different people have different interpretations of it and have different expectations like-User Acceptance testing means-
Test how the user would see/use the system
Once this testing is "Passed/Failed", we will know how the user would feel about our system. If it "Passes" then it is "ready to ship".
Exercise the most commonly used tasks that the system claims would benefit the user. If these functionalities do not work properly then the product is most probably not going to be used by many people
Think like a user and test the system, then you will find most of the issues user would face in real time when the product is released.
Give the product to the user acceptance testing team, they are very skilled professionals who think like the user and can find all the defects in the system related to usability.
And much more...................I can keep going on and on......
The thing people tend to forget or do not realize is, there are millions of people in this world (and based on type of product you are launching some percentage of these millions of people are going to use your product) and it is close to impossible to replicate or think like each one of them. So, what do you really mean when you say "After Usability testing PASSES then we can ship the product"????
Do you have Usability Testing Lab? (Which is far better than JUST having a group of individuals testing the product manually without access to latest technologies related to Usability!!!!)
Do your business sponsors and stakeholders understand that not all usability testing issues can be found in this phase?
Do you realize that you cannot think like each and every individual and about each issue which could affect the user? And there are infinite possibilities of some issue or the other being uncovered by some type of user, using some computer, in some location??
Common guys.....think about it!!!!
The best we can do is,
Have a properly equipped Usability testing lab with "eye tracking" (helps to find out which part of the page the user sees the most by tracking user eye movements with sensors) and other relevant technologies
Bring in some users with various backgrounds and ask them to use your product in the usability testing lab.
Based on the findings try to modify some of the things in the product
Before going into production apart from all the other testing you do, finally you do a Smoke test to ensure Critical and most common features which the product claims to benefit the user or the user would exercise in real time are working properly (Can get the recommendation of various stakeholders to identify these modules but the truth is most of the people do not know what they are talking about and it ultimately boils down to the skilled tester testing the product :-) (True story) )
And after all the above steps are completed, hope that the user does not uncover any issue that would destroy the products reputation, when it goes into production and pray to whomever/whatever you think gives you that special power!!!!!
As this point of time I can think about only these scenarios.
To summarize, we need to understand what this term really means before just assuming things and using it wildly. If a person thought about the above points I would say, he would to a certain extent understand what are the consequences of misunderstanding this term.
Anyways there are my 2 cents and just my point of view. If any of you had similar experiences or have something to share hit me up here in the "Comments" section or contact me. I would love to have a discussion on this and know your perspectives.