There is an erroneous perception in the UX community that if your method is qualitative, then numbers somehow cannot or should not be used.
These perceptions come from an informal practice that stems back to the beginning of the usability profession and continues through training programs and some UX experts.
Unfortunately, this perception is misguided and can prevent perfectly good data from being used to gain accurate views of the user experience.
Qualitative data can in fact be converted into quantitative measures even if it doesn't come from an experiment or from a large sample size.
The distinction between a qualitative study and quantitative study is a false dichotomy. It doesn't cost more money to quantify or use statistics. It just takes some training and confidence--like any method or skill.
Here are five examples of how you can take common qualitative approaches to assessing the user experience and convert them into numbers which can then be treated with a range of statistical procedures.
4 . Why was that task so difficult?: I recommend asking just a single question after users attempt a task in an informal Steve Krug usability test. If a user provides a low rating (below a 5), ask them to briefly explain why they gave a low rating. Take these open-ended comments, categorize them and add up the frequency in each group. This process can help you and your stakeholders make more informed decisions about the likely causes of the trouble. Figure 1 below shows an example of the comments from a recent usability test.
5.Combining Net Promoter Scores and comments: A powerful way of making qualitative, open-ended comments more actionable is to combine them with a closed-ended question, like the Net Promoter Score. For example, quantify what users say they would improve on a website, then show what these customer's Net Promoter Scores are.
An example is shown in Figure 2 below. There were 110 comments in total, but to quickly identify what to focus on, we can see that comments related to website navigation and product filters are both high in frequency and come from users that are likely generating negative word of mouth (notice the negative NPS). In contrast, design/layout comments and advertisements while high in frequency appear to be minor issues for the users.
I'm not advocating quantifying data for an exercise in counting. There are of course many software applications and websites which have never been exposed to any input from users. In such situations there will likely be many obvious problems that just need to be fixed, regardless of how many users encounter the problem.
But once you've picked the low hanging fruit of a neglected interface, the benefits of structuring your activities and results lend themselves to quantification, where you can derive more meaning from your methods.
The advantage of converting qualitative data into quantitative data is that the source of qualitative data--a direct encounter of the user's experience--can reveal nuances in usability, perhaps otherwise missed in more formal quantitative experiments and surveys.
Not only can qualitative data be categorized into quantities, but it can prompt further questions and discovery for usability improvement.
Why was that task so difficult?: I recommend asking just a single question after users attempt a task in an informal Steve Krug usability test. If a user provides a low rating (below a 5), ask them to briefly explain why they gave a low rating. Take these open-ended comments, categorize them and add up the frequency in each group. This process can help you and your stakeholders make more informed decisions about the likely causes of the trouble. Figure 1 below shows an example of the comments from a recent usability test.
Designs are to Delight. Ensuring a memorable journey for the consumers is our responsibility as designers, believes Neha Modgil, Director at Techved Consulting. She has much more to say, express and share what’s on her mind. Put down on paper, here are some beautiful thoughts straight from our Director’s heart.
When I started Techved, it was passion turned to business. Over the years as I experienced world, Techved is my expression and gratitude to being a designer, woman, mother, CEO, entrepreneur and above all, a human.
With designing seated so close to my heart, I constantly strive to spread laughter and cheer through our designs. At Techved, I complete my desire to ensemble designs and emotions so that we deliver work that stays close to people’s heart too.
Design is ubiquitous. Everything around us is designed to be the way it is. Design is thus never a beginning; it is always an extension, refinement and distillation.
Design is representing of intent and is generally connected to terms like creativity and innovation or often seen as a form of an art. Every design is basically about revamping an already existing idea or design. It is about innovating and making it more acceptable to suit the needs that change with time.
by Dabney Gough and Holly Phillips on 2003/06/09 |
“If you’re considering using this technique, it’s a good idea to consider the pros and cons before diving in. Despite this method’s similarity to in-person usability testing, there are significant differences as well.”
After completing a remote usability testing study, the manager of research and usability at Agilent Technologies, and myself, a usability consultant at Socratic Technologies, decided to write a synopsis of our experiences conducting remote online usability testing for several areas of Agilent’s website. We feel that sharing pros and cons, as well as tips for success, will be useful for other usability practitioners who are unfamiliar or inexperienced in this particular technique.
Remote online testing versus traditional usability testing
Traditional, one-on-one usability testing in person is a great technique for uncovering usability issues on your website. It provides your team, as well as management, the opportunity to watch actual users use and give feedback about your site. This “reality check” is invaluable for stakeholders, designers, and developers. Unfortunately, though, in-person usability testing isn’t always feasible. Projects take longer than expected, budgets get tightened, and target users can be hard to come by. So what’s a usability crusader to do when in-person usability testing is impossible?
One option is to conduct usability interviews using an online meeting service, such as WebEx. (There are a number of other companies offering similar services, but our discussion will focus on WebEx since that is the service used for this particular study.)
WebEx is a web-based service that allows users in remote locations to participate in online meetings. WebEx offers many features, including the ability to share viewing and control of a web browser. Using this feature during a telephone interview allows the moderator to view the user’s mouse movements and web pages visited. Control of the browser is easily passed back and forth between moderator and respondent, and therefore, facilitates discussion similar to that of a traditional in-person usability interview.
If you’re considering using this technique, it’s a good idea to consider the pros and cons before diving in. Despite this method’s similarity to in-person usability testing, there are significant differences as well.
Benefits to remote online interviews
The “remote” aspect of this technique is certainly appealing; it makes it easier to reach respondents in diverse geographic areas, and you don’t need to limit your users to a single city or area (unless you want to). This is particularly useful if you are developing for a small, hard-to-reach, or decentralized group of users for whom it would be difficult to schedule interviews at a single location otherwise.
Another benefit is the potential cost savings, particularly if you typically travel and/or rent a focus group facility for your testing. It eliminates travel costs for the interviewer and team, which is useful when your team is scattered across the country, as was the case for our project. Instead of paying for travel costs and facility fees, you pay a per-minute, per-participant fee to use WebEx.
The interview schedule can be much more flexible, eliminates the need to “make the most” of a day’s rental at a focus group facility, and allows your users to select a time that suits their individual schedules.
Since most respondents choose to participate from their office or home, there’s also a slightly more ethnographic element to remote testing. However, keep in mind that, as with any ethnographic study, respondents may become distracted by people or things in their environment during the course of the interview. (One interviewee participated from the comfort of his kitchen—the clang of pots and pans were audible in the background.) These distractions could be considered a benefit or a limitation of the technique, depending on what type of feedback you’re looking for.
One final benefit of this type of testing involves exposure at the client’s company. Typically, the research manager and direct team are the only ones that can feasibly attend in-person usability interviews—a handful of very interested people, but still just a portion of the folks who could benefit from first-hand observation. Remote online interviews suddenly enable these other interested people, from senior management to the actual website developers, to “attend.”
Limitations and drawbacks
There are a number of limitations to using a meeting service to conduct usability testing: the most obvious being that you aren’t in the physical presence of the respondent. Consequently, there is an additional “degree of separation” that can be challenging. You cannot see the respondent’s facial expressions and other nonverbal cues, and it’s more difficult to build rapport and trust when you’re just a voice on the other end of the phone.
Another potential drawback of using a meeting service is cost. WebEx offers a monthly, unlimited-use subscription plan, but because of our limited needs, we elected to use the pay-per-use plan. In the latter scenario, charges are per minute, per “attendee.” At last check, a single hour-long interview with six attendees (including the moderator and user) could run up to $300.
Teleconferencing service is not included in the basic pay-per-use cost, and is an additional expense if you don’t already have access to such a service. (WebEx does offer teleconferencing, also charged on a per-user, per-minute basis.) Using an online meeting service could end up being more expensive than renting a focus group facility, depending on how many interviews you plan to conduct, the anticipated number of “observers,” and any travel costs you might normally incur. (Note, though, that some clients already have unlimited-use corporate WebEx accounts that could be used for this purpose.)
Online remote usability also requires a bit more finagling if you want to record the interviews. A traditional portable usability lab can be used, but this requires you to conduct the interview via speakerphone in order to capture both the moderator’s and the respondent’s comments. Keep in mind, however, that using a speakerphone feature also picks up a great deal of background noise and reduces the sound quality for everyone else on the call.
There are a few other recording options that can be exercised separately or in tandem. You can make an audio recording with a simple telephone-hookup tape recorder or through WebEx’s internet phone feature. (A word of caution about using the latter: Setup and use of this feature requires additional equipment and technological know-how on the part of the participant.) For video recording, a traditional portable usability lab (without audio) or WebEx’s screen recording feature can be used.
Tips for a successful study
Online remote usability studies can be recruited through any channel you might use for normal usability testing. However, in a WebEx study you never actually meet the respondent in person, making clear communication much more critical when recruiting respondents. It’s important that respondents know what they need to do and what to expect at the appointed interview time.
If you’re doing this type of project for the first time, this is the part that takes the most “figuring out.” Careful foresight will take you far in this endeavor…
Conducting the interview
Once you have respondents scheduled and the meetings arranged, you’ll be ready to conduct the interviews. There are a number of techniques that will make the interviews themselves go a lot smoother. Most of the following suggestions are aimed at reducing last-minute scrambling (on the part of the moderator) and avoiding possible “dead time” as observers wait for respondents to get up and running.
If you are using a teleconference service to allow your team to listen in, we suggest following the guidelines above. In addition, once the respondent has successfully logged in to the meeting, each team member should hang up and call in to the conference number. That way:
Now you can proceed with the same type of introduction you might use in a traditional usability test: describe the purpose of the research, ask the respondent to “think out loud,” reassure them that there are no right or wrong answers, etc. At this point, it’s also useful to cover a few other WebEx-related items:
After that, proceed with tasks just as you would normally. Throughout the interview, you may need to remind the respondent to point with the mouse, or to click to take control of the mouse.
If all of this sounds like a complicated endeavor, it’s only because there are a lot of small details that need to be taken care of in order to make everything run smoothly. However, it gets easier with each study you conduct.
This method can be used successfully in many situations. The tradeoff between slightly reduced insight and dramatically increased efficiency may be a beneficial one.
We don’t foresee this becoming “the next wave” in usability testing, for the limitations (inability to see respondent’s facial expressions, limited to participants who have high-speed connections, etc.) are not insignificant. However, this technique is very well suited for instances in which traditional in-person usability is not possible, whether for logistical, financial, or political reasons. In many situations, the benefits of getting user feedback, albeit of a somewhat limited type, far outweigh the possibility of getting no feedback at all.
Women are splendidly kicking it on in all fields today, be it the corporate world or the sports ground,the glamour world or the parliament house; ladies are leading as businesswomen, CEOs, mothers, wives and what not. I don’t have to look very far to find a case in point, as I come across one such leading lady every single day. And I wonder how she balances work and life altogether so smoothly.
Technology has bestowed us with many comforts and luxuries, and made life way easier for us today. In order to use technology and its various inventions, it is vital we know how to use, accept and adopt it. Human interaction with technology takes place through desktop computers, cameras, mobile phones, internet etc. The interface between humans and system may become problematic at one point when the systems are less useful and usable. An advanced move towards building an enhanced human interaction calls for a close understanding of humans, information and various interactions. Providing the users with multiple modes of interaction with the system and offering various benefits over the conventional unimodal interaction is the step ahead. This is termed as Multimodal Interaction as it refers to the different modes available to the user for interacting. Multimodal interaction has created engaging experiences for users across different systems with multiple modalities ensuring an enhanced communication between humans and systems. Humans interact via a wide variety of modalities for communication in daily life; like speech, gestures, facial expressions, touch etc. The multimodal interaction is an effort towards humanizing the human-computer interface. Smartphone is one universally used tool that offers users with such multimodal interaction experience.
User experience is an indispensable part of software development and has an immense impact on the success of the software. The term user experience closely involves the user and his emotions during the usage. Be it a mobile app, website, a software, a game, a product, a machine , a service or system , User Experience is of vital importance.It is directly related to the user’s perception about the experiential aspects , convenience & competence of use and the utility of the system. With changing circumstances, it becomes necessary to keep the user experience constantly updated with time by innovating and amending it; it is beneficial to look after its dynamicity.