Remember that the quality of usability testing and reliability of results depends on two factors – proper methodology and participants chosen for the study. Let’s focus on the second factor for a while 🙂

Choosing a group of testers

Testers taking part in the study should reflect end users of the product we are testing. This is why they should meet the criteria of end users for the product if it comes to demographics (age, gender, education, earnings) and if it comes to the behavioural aspect. The most important factor is that testers should fit the product if it comes to given behaviours (shopping intentions, motivations, preferences) which is presented in the below example.

Let’s imagine a person who’s planning to buy an Xbox. We can suspect that this will be a man, who is young (demographics). But these criteria are not enough because not every young man will use an Xbox. He needs to be interested in gaming, know it at least at a basic level (behavioural traits). This is why to our study we should invite participants who meet both criteria.

When choosing a group of testers you should remember not to test a product on users who earn money on testing, take part in too many studies (one study in 6 months is acceptable) or on the group of IT specialists. Results from testing on both these groups will be unreliable and will not add value. The IT specialists will pay too much attention to the technical aspects which can overpower the usability issues. At the same time, users who complete such studies too often will be so acquainted to them that they would automatically “click through” the test.

Ways of recruiting

You can recruit users in various ways. It’s important that you don’t get used to only one particular form of recruitment. If one way of recruiting users (e.g. individual recruitment) won’t be 100% effective, you can outsource it to the recruitment agency or panels.

But first, let’s take a closer look at particular ways of recruiting users.

Widget at a website

Box placed at the edge of a website or a pop-up with invitation to take part in the study. It’s displayed to users while they are browsing the website and encourages them to answer some questions / take part in the study.

Widget is a very effective solution but it has its limitations as well. To begin with, you can place it only at a website that you have access to (which means your own). As a result it’s impossible to test competitors’ websites.

Secondly, it will be available to users who are already using the website – know the site (more or less) and have their opinions about the site – which can influence the way the study is implemented. What’s more, the strongest motivation to taking part in such study is mostly negative (unsatisfied users). Positive motivations are always weaker than the negative ones (one unsatisfied customer will on average inform 9 people about his or her opinion). This is why you shouldn’t be surprised if the study will provide mostly negative opinions.

What’s also interesting is the fact that using a widget makes it difficult to evaluate some “novelties” at a website – changes, redesign. Users acquainted with a given system will have problems to objectively assess its new version. People don’t like changes and a new design can be evaluated negatively only due to the fact that it differs from the old one. Of course after a while people get used to new systems (very well-known examples of this are each new redesigns of Facebook wall), but during testing we need to take the assimilation bias into account.

Another aspect is the availability of the study – since the study is publicly displayed at a website, your competitors can also take a look (and this is what we don’t want to happen e.g. while testing layouts with innovative functionalities). This is why for layout and redesign testing it would be a good idea to use another method.

Widget would be the most effective tool for quick, short and small Customer Satisfaction Surveys – several questions on using a website. The plug-in should be displayed contextually in given scenarios of use and should be heavily targeted taking a given scenario of use in mind e.g. during the next session after cart abandonment we can display a question (directly at a widget) “We noticed that while recently visiting our page you were interested in some products but didn’t buy them. Could you please describe why did you decide not to proceed with the purchase?”

Mailing

If you have a database of your customers’ emails you can send out an e-mail inviting to the study. Remember that the subject of this email should be attractive and encourage customers to click e.g. Take part in the study and pick up a prize!

If you have some prizes for participation in the study, mention them (just like in the above example). This is a perfect moment to reward your users for taking the time to fill in the study and at the same time bring them closer to the company (with discount coupons, vouchers). Remember that users who take part in the study do you an enormous favour. Let’s face it, they are wasting their time to make your product better. This is why we strongly encourage to rewarding your testers and suggesting that they will be rewarded.

But recruiting via email has its disadvantages too. Similarly to plug-in at a website it gives access only to your clients (who can be reluctant to novelties and changes or might just want to communicate their general disapproval). Also, you don’t have full control over who is taking part in the study – sometimes it can be an employee of your competition (hidden behind jack.sparrow@gmail.com email address).

Social networks

Since most companies have a Facebook, Twitter or LinkedIn page it would be a shame not to benefit from this potential. If you’d like to invite your customers to the study add a post on your page or choose the sponsored ad (targeted at criteria and behaviours). If you’re curious what users from specific groups think – add a post in a group. Users who belong to such groups are usually open to share their opinions, review and help (which helps in finding a particular group of users for the study). But you should remember that similarly to internet fora, groups are full of trolls. If one person will publicly share a negative opinion about your study or product (e.g. in comments added to your post) it can result in an avalanche of negative comments and will destroy credibility of the study.

Another aspect which results from our experience with users, is also of importance. While searching for testers among followers of the brand you will mostly find users who are close to the brand and know its website pretty well. They are mostly users who are involved in your brand. What will be the result of this? It will be difficult to find light users among your Facebook followers.

Panel recruitment

If you don’t have time (or even the possibility) to recruit users on your own you can always leave it to an outside company. Remote usability applications usually support particular panels so it won’t be any problem. Panel recruitment is the easiest way of recruiting international users to your study.

However, you should keep in mind that the narrower the criteria of choosing users for the study are (e.g. women, age 40-45 who have cats and buy Whiskas products), the more expensive it would be to find and gain such users. What is more, respondent panels often have a minimal cost to project implementation and they won’t reduce such cost. In this case we suggest increasing the researched group because it’s very possible that the same price can give you twice or three times more results for your study.

What’s also important is the quality of the chosen panel. Some panels are more quantity-oriented, they don’t pay close attention to their database, its maintenance and the number of studies run on the same group of users. Other panels pay attention to the quality of a sample. It’s important to discover which type of panel suits your needs best.

Outsourced recruitment

Don’t forget that you can also outsource recruitment of users to the recruitment company – “field agents” who will find the right testers. The main advantage of this solution is the full outsourcing of actions – you provide your candidate criteria and don’t have to worry about anything else. At the same time the disadvantage would be the cost – such services can be much more expensive and sometimes you can pay even $100 for recruiting one user. Also it will be difficult to gain international users this way because recruitment companies work in their limited area. You would spend a lot of time to search for a similar company in another country in order to gain international users.

Individual recruitment

Of course you can always recruit testers on your own. It’s important to remember that it takes quite a lot of time: you would need to find users on your own, talk to them, persuade them to take part in your study and later on remind them about it. The advantage of this solution is a chance to test new solutions on an objective group of users (it will be enough to find out by asking if they used your product or competitors’ solutions).

What’s important here is the fact that this way of recruiting users is cost-effective because no intermediaries are involved and you don’t have to pay any commission.

The number of testers

Tested sample depends only on your personal needs. Usability studies can be conducted on a small sample of users, which is supported by Norman Nielsen. According to Nielsen’s studies it’s enough to use a group of 5 testers to catch the key issues at the site and each next person supports the results gained from the previous testers. If it comes to remote usability testing we suggest using a larger group 8/12 testers because usability tests are not fully controlled and sometimes there are users who e.g. don’t comment their activity at all (e.g. because they are introverts) or users who chose a completely different path from the one we wanted to test (individualists 🙂 )

What would be the summary?

We hope that we’ve cleared your doubts about recruiting users for the usability testing at least a bit. Remember the type of recruitment that you will choose depends only on your needs (what you want to test, what group of users should take part in the study, etc.).

• if you care about the time – outsource recruitment of users
• if you have a limited budget – try to take care of recruitment on your own
• if you want to verify new solutions / functionalities – don’t make the study publicly available

So, let’s do it! 🙂