Should job applicants be allowed to access online selection tests using mobile devices?

Written by Neil Morelli and A. James Illingworth

Online selection testing has grown tremendously in recent years due to its ability to process heavy applicant volume, efficiently score assessments, and increase test availability. As a result, how organizations deploy talent assessments, and the ways in which applicants access these assessments, has changed dramatically. Whereas in the past applicants reported to a central location to complete assessments using a desktop computer, now, with the proliferation of wireless technology, applicants can access and complete assessments remotely over the Internet on a variety of mobile devices (e.g., Smartphones, hand-held tablet computers).

Despite the added benefits of this new accessibility, such as lower delivery costs and greater contact with larger, more diverse applicant pools, organizations are faced with a dilemma: Should the administration of online assessments be extended to mobile devices? Before a clear recommendation can be made, two questions must first be answered. Do the hardware features (e.g., smaller screens, virtual keyboard) of mobile devices change how the test works, and thereby change the interpretation of the test results? And, does completing an assessment on a mobile device drive meaningful performance differences?

To address these questions, APTMetrics investigated the test equivalence, performance differences, and differential adverse impact of a text-based, non-cognitive test administered to a large sample of customer service job applicants (N= 937,243), using both desktop and mobile formats. It was discovered that the assessment could be interpreted the same way across devices, and that job applicants performed equally well regardless of the administration format. This finding is encouraging as it provides evidence that non-cognitive assessments can be just as reliable and valid when delivered on a Smartphone or tablet computer as they are on a desktop computer. 

Since this initial examination, two other studies were conducted to replicate and extend the previously described findings. The first research study was designed to replicate the results of the previous investigation using a different non-cognitive selection assessment and job family (e.g., retail sales job applicants). Using a similar study design and analysis methodology, it was discovered that the assessment operated similarly across devices and no meaningful performance differences between device groups were detected. The second study expanded the previously described findings by focusing on the potential differences between devices based on their operating system (e.g., Windows 7, OSX) and Internet browser (e.g., Internet Explorer, Mobile Safari). An examination of three different non-cognitive performance predictors discovered that operating systems and Internet browsers native to mobile devices did not change how the assessments function or result in meaningful performance differences between applicants.

So, should job applicants be allowed to access online selection tests using mobile devices? The evidence gathered thus far is extremely encouraging.  Based on this research, text-based, non-cognitive assessments of performance predictors appear to function similarly across devices without affecting applicant performance or resulting in adverse impact.

While these findings suggest organizations can give the green light to job applicants using mobile devices for text-based, non-cognitive tests, many unanswered questions still remain about the broader use for mobile devices for administering assessments. For instance, what happens as organizations and test developers design assessments that are specifically intended for mobile devices? Or, how will mobile devices react to more technologically advanced assessments, such as those with images or video? In addition, it will be important to know whether these results can be extended to cognitively oriented tests.  Research is underway to address these questions as mobile devices become more pervasive and assessments evolve in their reliance on audio and video multi-media.

Communicating the Value of Assessments to Line Managers

Communicating the Value of Assessments to Line Managers

By Ron Tepner and Dan Lezotte

Although those of us in the human resource field understand the value of using assessments in candidate selection, it is often difficult to convince line managers of their value and to gain complete buy-in to an assessment program.

It is particularly important to "make the case" for assessments in a down economy where we have the unusual luxury of more applicants than openings, and the opportunity to be very
selective in our new hires.

Additionally, like most HR tools, assessments must be applied consistently across the board to be both legally defensible and effective.

In implementing assessment programs for Bridgestone Retail Operations’ customer-service teammates and technician/mechanics, several challenges and obstacles were presented by the field managers that had to be overcome.

Making the Business Case

Common concerns expressed by our field managers regarding the use of assessments included:

There is a shortage of available talent for our open positions and we can't let anyone "get away;" we don't have the luxury of being selective.

Response: The benefits of being selective outweigh the perceived lack of luxury. The new practice of requiring the assessment assures a larger population of more productive, longer-tenured employees who help create customer loyalty.

Customers like to be served by the same consistently proficient technician every time they visit. Additionally, we will be offering a new staffing and scheduling tool from our workforce-management initiative that will help keep candidates in the pipeline, reducing the last-minute panic that leads to taking a higher risk approach to selection.

Our applicants will not put up with an involved, time-consuming selection process -- they'll just go somewhere else

Response: The poor economy has actually helped us. One of the few advantages of a down economy is that more qualified applicants are available for open positions. While we should always be sure we are hiring the best person for the job and a person who has a high probability of success in the job, having a larger applicant pool provides us the luxury of being even more selective.

Now more than ever, we need assessment tools to help us sort through the applicants and identify those that have the best chance of success. We need to take the time to make the right decision, without being concerned about losing a candidate because we require them to go through an assessment process.

In fact, we have found that many applicants (who are now employees) viewed our organization more favorably as a result of having gone through a rigorous selection process. Who doesn't want to become a member of an "exclusive" club? We also believe that it honors our existing workforce to not just allow anyone into the camp and expect good folks to work with those who are not the best.

It's too inconvenient and time-consuming to ask an applicant to drive to a district office or testing location to take an assessment. We need them to start right away.

Response: The assessment is part of a process to screen in the very best, which includes a physical, drug screen, interview and background check. The test will fit nicely into the time space required to be in compliance with the other screening requirements.

Not taking the time ensures the continuation of higher levels of turnover, more training investment and a significant portion of the workforce always ramping up to cover those who are exiting. The assessment changes this dynamic significantly.

The assessment will take control away from me in making hiring decisions. What if I really want to hire someone but the assessment says I can't?

Response: As the positive impact of less turnover and higher productivity takes hold, the candidate pool will be populated with more qualified candidates from which to select. If the
assessment results suggest a candidate is not a good fit, the manager will have several other qualified candidates from which to choose.

Our experience with other assessments has been that, as managers see the positive impact from using the assessments, they place more trust in the results and quickly embrace it as an effective tool to help them make more informed