Psychometric Assessments and Their Value in The Maritime Industry

By Evi Argyrou, Counseling Psychologist of SafeMetrix.

Evi Argyrou is a Counseling Psychologist currently coordinating the research and development initiatives related to the SafeMetrix product line of Safebridge. She is responsible for preparing company Integrated Reports of seafarer’s psychometric assessments and performs various statistical data analyses. She is also involved in various research-related projects aiming to enhance the SafeMetrix product line. Evi holds a Master’s Degree in Counselling Psychology and is experienced working with people suffering from anxiety, trauma, depression, phobias, relationship issues, and grief.

As humans, we all have certain capabilities and limitations. In an era of technological evolution, human beings and machines are in great competition. It is important to acknowledge both our strengths and limitations to make the most out of our potential. For instance, people are great at pattern discrimination and recognition. There is no machine in the world that can interpret and use a maritime radar screen as effectively as a trained deck officer would. On the contrary, no human can calculate numbers as quickly and accurately as machines can do. Additionally, people have a relatively limited memory capacity.

Besides these traits, human performance is influenced by other factors, including knowledge and skills we have acquired, motivation, alertness, and emotions. Within this context, the risk of accidents and the human factor that could cause them – is critical. According to the Journal of Safety Research 37 (2006), the human factor also seems to be the main cause of incidents at sea. In agreement with this theoretical framework, this article offers a review of the role of psychometric assessments in the maritime industry.

Industrial and organisational psychology is generally concerned with understanding and predicting human behavior in work organisations. As such, its practitioners are involved in the use of assessments to characterise or describe the status of individuals (e.g., workers or managers), collections of individuals (workgroups or teams), and/or organisations. Moreover, these assessments can be about attributes, processes, dynamics (changes), or effectiveness levels.

A psychometric assessment is a process of evaluation that uses a combination of techniques/tools (tests, interviews, case studies, behavioral observation, specially designed apparatuses, measurement procedures) to make hypotheses about a person and their knowledge, skills, behavior, personality, and attitudes. Psychological testing is nearly always interpreted by a psychologist or a highly trained professional.

The most common psychological tests used for recruitment, internal promotion, and individual or team development are the following:

• Aptitude tests (to measure a candidate’s cognitive abilities)
• Tests of attention
• Speed and accuracy test
• Situational judgment tests (SJTs)
• Personality questionnaires
• Behavior tests (to explore fears, motivators, values, and behavioral style)

These are often described as “norm-referenced” tests. That means the tests have been standardised so that test-takers are evaluated similarly, no matter where they live. A norm-referenced test of a deck officer, for example, may rank a seafarer’s ability compared to other seafarers of a similar job description. Norm-referenced tests have been developed and evaluated by researchers and proven to be effective. Tests’ results should only be considered if the test administered is statistically reliable (the results are consistent) and valid (the test measure what it is supposed to measure).

With a focus on the maritime industry, psychometric assessments have offered valuable help concerning human error and safe operations at sea. Dr. Anita M. Rothblum from the U.S. Coast Guard Research & Development Center, in her publication “Human Error and Marine Safety,” describes the maritime system as a people system. According to her, people interact with technology, the environment, and organizational factors. Sometimes the weak link is with the people themselves, but the weak link is the way that technological, environmental, or organizational factors influence the way people perform.

Assessment’s use lies in the recruitment process, especially when recruiting cadets and junior officers. It is also part of the promotion process or measures capabilities to ensure performance effectiveness on board.

Understandably, seagoing personnel requires a unique set of personality characteristics and capabilities. Crew and ship management companies believe that these assessments during the pre-recruitment stage would allow them to consider or reconsider their decisions on whether to employ or re-employ a candidate.

While cognitive skills have been on the forefront for many years in the maritime industry, attention is now given to different kind of (soft) skills that show promising results. Currently, core prerequisites in most companies are the following:

• Personality characteristics (opposed to a character)
• Emotionality (coping under stress & identifying risks)
• Control (managing work & commitments)
• Willpower
• Energy and ability profiling (perception, thought, and initiation) and determination
• The ratio of test accuracy compared with test completion speed

The main target of each companies’ testing may differ concerning the organisation’s current needs.

It is crucial to mention that a consensus of assessments is used as an ‘added value’ tool and not as a standalone approach to understand behavior and performance. For example, assessments are beneficial in potentially identifying someone “likely” to either directly cause or be a contributing factor towards a possible onboard situation or a marine incident. Thus, we could lessen human factor errors onboard along with reducing costs and expenses.

The assessment process begins when a company decides that a specific group/person should be assessed for a particular reason. For example, a crew management company is in the process of seafarers’ promotion. As a result, they consider possible candidates or a specific skill set that an ideal candidate should have.

It is essential to clearly define the assessment’s objective and how the results will be used later. This will help determine the scope of the assessment and select the right people to assess. Moreover, language or cultural barriers should be considered when selecting a test as ‘one size does not fit all.’

Next, it is important to identify who will perform the assessment and how the results will be delivered and shared. Usually, the assessor would be a company partner and a psychologist, depending on the organization’s needs and standards. However, many companies decide to leave the assessment process to external partners (psychologists or organisations specialising in this field). Further, it should be clear how results will be delivered and shared to ensure that the assessment results positively impact the organization’s future development.

Further, the company may revise all the candidates’ relevant information, including their education, SES, marital status, age, nationality, and previous experience, to identify the best fit or exclude people from the candidate list. It is important to note that this step depends on the company’s judgment.

For example, a company might require an engine officer above 40 years old speaking native English. This step can also be done after the test is administered (see below) and depends entirely on each company’s procedures. However, if the assessment’s goal is to recruit new coming employees and not to promote them, this step comes with the previous step in selecting candidates.

After the goals have been established and the candidates have been selected, the appropriate test will be administered. A psychological test can be described as a behavior and performance observation system that relies on numerical data or a categorical scale to establish changes and differences.

Testing measures the level of skills or knowledge that exists or has been reached. A psychological test can also measure differences between people or differences in the same person over time (Cohen, Swerdilk, and Sturman, 2013).

The tests are one type of assessment tool used by professional assessors (along with other tools, such as the interview). The value of a test, or any other tool, is intimately linked to the assessor’s knowledge, skills, and experience.

Results of a test would then be gathered and evaluated. Since most of the tests are online-based, they are scored and presented as standardised scores or percentiles. Psychologists seek to take the information gathered from psychometric tests and weave it into a comprehensive and complete picture of the tested person. Recommendations are based on all the assessment results. Psychological assessment is never focused on a single test score or number. Every person has a range of competencies that can be evaluated through several methods. Therefore, test results should not be the only criteria for selecting, promoting, or retaining employees.

A psychologist is there to evaluate the competencies and the limitations of the person and report on them in an objective but helpful manner. A psychological integrated report will note the weaknesses found in testing together with the individual’s strengths. Thus, the assessment goal is to make improvements or predictions (e.g., future job performance) instead of being judged. That kind of step is completely dependent on the assessor’s psychological competence and how much he/she has learned about basic and advanced psychology. Assessments incorporate the analysis and interpretation of data, and they are instrumental in defining and understanding the underlying causes of behavior and uncover subliminal aspects of performance.

It is well established that multiple sources of information, including candidate nervousness, training, attitude, and culture, can affect assessments’ objectivity, especially for risk-taking behavior. A candidate’s performance may also be affected by other personal factors such as family-related problems, life-changing situations, emotional difficulties, illness, sexual orientation, recent loss, divorce. Thus, there is a need for test results often to be re-checked by questions regarding how the person attacked and answered the test items. In general, the more complex the variable under investigation is, the less adequate is a single test to come up with representative answers. Therefore, some additional interview questions are necessary to acknowledge any undermining factors that might influence one’s performance.

Apart from an in-depth understanding of the factors and forces producing the status quo, assessment conceptualization also includes another step. The second step reflects an implicit or explicit model of change. Here, the assessor and the company must come to grips with just how the factors and forces operating in a particular venue can be modified in such a way as to bring about changes or desired outcomes.

An action plan with Specific, Measurable, Actionable, Realistic, and Time-bound (SMART) objectives might be developed, showing exactly how the investment in the assessment and associated improvements will return. Training courses have been well recognized in the industrial setting in the last decades.

After training, it is important to reassess candidates to see if their performance has been improved and if the training has been helpful. In the cases where a company wants to assess knowledge, learning objectives must be clear. A re-test should match those objectives to allow effective determination of whether employees have met those learning objectives. The re-test procedure can also be seen as a reflective process of the course/training. It is one of the most efficient ways to cement the knowledge, identify gaps in training, and identify the barrier(s) to a strong learning transfer. They can also be used to make recommendations for further development in areas of weakness. When it deems necessary, a candidate might be obliged by his company to re-take the training/course if the test results are not aligned with the company’s standards.