Blog

Kurumunuzdaki Yetenekleri Keşfetmek için Bilimi Kullanın (III)

Using Science to Identify Future Leaders:

Part III - The TALENTx7 Assessment™ of Learning Agility


Kenneth P. De Meuse, Ph.D.

Both practitioners and academics agree that the proper identification and development of future leaders is critical to the success of organizations (see DeRue, Ashford, & Myers, 2012; Silzer & Church, 2009). Numerous journal articles and books have been written on the topic of leadership. A recent Google search of the word "leadership" yielded 492,000,000 entries. And yet, executives often express dissatisfaction with their company's initiatives for identifying its next generation of leaders (Charan, 2005).

During the past decade, the construct of learning agility emerged to help organizations identify and develop high potential talent. And, when used correctly, research suggests that learning agility enables organizations to assess leadership potential successfully (Dries, Vantilborgh, & Pepermans, 2012; Goebel & Baskerville, 2013). This whitepaper is the last in a series of three that investigates learning agility and the science of leadership. The focus of this paper is to introduce the newest measure of learning agility - the TALENTx7 Assessment™. This online self-assessment is marketed by Leader's Gene Consulting, a firm headquartered in Shanghai, China. For the past 15 months, I have been involved in creating and validating this psychological instrument. In the following paragraphs, I describe how it was developed, how it measures learning agility, and how it deals with the issues of response faking and social desirability. In addition, I review its three feedback reports and highlight some of the unique features of this self-assessment.

The Development of the TALENTx7 Assessment™
A diverse team of eight subject matter experts (SMEs) were involved in the development of items for the TALENTx7 Assessment™. Two of the SMEs were industrial/organizational psychologists with a background in leadership assessment and development, three were senior executives with more than 70 years of combined corporate experience, and three were talent management professionals. A series of pilot studies in several major organizations were conducted during the summer of 2014 to develop the assessment items. Approximately one-half of the participants were female. Fully, 51% had job titles at the director or executive level. The participants were employed in organizations from a number of industry sectors, including communications, professional services, retail, hospitality, industrial, and pharmaceutical.

The assessment utilizes 118 items to measure learning agility. Items are written in the form of behavioral statements and work values. For example, "When there is conflict on the team, others frequently seek me out to help resolve it." And "I believe it is important to look at new ways to improve, no matter how successful things are currently." A 5-point rating scale is used to express the degree to which respondents agree with each item, ranging from strongly disagree (1), disagree (2), neither disagree or agree (3), agree (4), to strongly agree (5). In addition, the assessment includes 30 word pairs describing low or high learning agile individuals. Respondents are asked to select the word in each pair most descriptive of them. In total, the assessment takes approximately 30 minutes to complete.

As reviewed in Part II of this whitepaper series, research evidence indicates that there appears to be seven different conceptual areas of learning agility (De Meuse, 2015). Consequently, the TALENTx7 Assessment™ measures seven facets of the construct. Each facet is defined by the extent to which an individual displays the following behaviors:
1.  Cognitive Perspective: thinks critically and strategically, approaches organizational situations from a broad high-level viewpoint, and focuses on multiple inputs rather than from only one or two functional/technical perspectives.
2.  Interpersonal Acumen: interacts effectively with a diversity of people, understands their motives, values, and goals as well as their strengths and limitations, instills confidence in them, and leverages them to perform well.
3.  Change Alacrity: relishes change, demonstrates curiosity and eagerness to learn new ideas and ways of behaving, open-minded to new situations, and continuously seeks innovative (and at times risky) approaches to perform jobs.
4.  Drive to Excel: sets challenging personal and organizational goals, resourceful, and delivers outstanding results in new and untested jobs.
5.  Self Insight: understands themselves, their capabilities, weaknesses, beliefs, values, feelings, and personal goals as it relates to the workplace.
6.  Environmental Mindfulness: observes their external surroundings carefully, attends fully to changing job duties and requirements in new organizational roles, approaches environmental changes in a nonjudgmental manner, and regulates emotions effectively.
7.  Feedback Responsiveness: solicits, listens to, and accepts personal feedback from others, carefully considers its merits, and subsequently takes corrective action for performance improvement.
Overall Learning Agility is the composite of the seven factors and provides information on the individual's overall leadership potential. Scores on each of these seven factors are used to clarify specific facets of learning agility, providing diagnostic guidance on which areas need development.

Response Faking and Social Desirability
As mentioned in Part II of this whitepaper series, individuals who know they are being evaluated often try to present themselves in a very favorable light. This tendency to respond in a socially desirable manner is done in an attempt to positively impact some type of employment decision (Dunning, Heath, & Suls, 2004; Morgeson, Campion, Dipboye, Hollenbeck, Murphy, & Schmitt, 2007). In the case of learning agility, it may be performed to enhance one's chances of being labeled a high potential or being placed in an emerging leaders program.

The TALENTx7 Assessment™ incorporates four psychometric scales designed specifically to enable administrators and executive coaches to determine whether the scores on the self- assessment are accurate (i.e., truly reflect the respondent's learning agility). This methodology also permits organizational decision makers to have confidence in the veracity of the results of the assessment. Each of the four so-called "Accuracy Scales" is briefly described below.

Response Orientation Scale - This scale examines a systematic response bias frequently referred to as "social desirability" (i.e., the degree to which a respondent presented himself or herself in an overly positive - or negative - manner). Based on the extent to which a respondent attempted to "fake good" or "fake bad," the TALENTx7
Assessment™ adjusts scores accordingly. It is a common psychometric procedure for addressing the social desirability aspect of self-assessments (see Anderson, Warner, & Spencer, 1984).

Response Consistency Scale - This scale identifies the degree of consistency among assessment item responses. In some instances, the TALENTx7 Assessment™ includes similarly worded "item pairs" that describe nearly identical personal behaviors. In other cases, one item in a pair is worded positively and the other item worded negatively (then reverse scored). Inconsistency can be caused when an individual is distracted by a telephone call or someone coming into the office unexpectedly, multitasking, or simply not reading the items fully while completing the assessment. Whatever the reason, unless there is an acceptable level of consistency, scores on the assessment may not be an accurate indicator of the individual's learning agility.

Learning Agility Symmetry Scale - This scale likewise measures the degree of consistency in responses. In this case, two additional sets of items are embedded into the assessment. One set identifies specific behaviors representative of individuals who possesses the ability and aspiration for successive leadership positions (a so-called high potential or "HiPO"). The other set of items presents behaviors typical of individuals who are highly technical or functionally oriented (a so-called high professional or "HiPRO"). The Learning Agility Symmetry Scale reports the degree of relationship or symmetry between scores on these two item sets.
Word Pair Alignment Scale - The 4th scale also measures the degree of consistency in responses. Again, it uses yet a different approach to do this. In a separate section of the TALENTx7 Assessment™, 30 sets of word pairs are presented. Each word in the pair was carefully selected to be equally attractive (e.g., thorough verses decisive, resourceful verses dependable). However, one of the words largely represents a high potential orientation; whereas, the other word in the pair represents a high professional orientation. Based upon the pattern of words selected by the respondent, an overall score is computed. This score is compared to the respondent's overall learning agility score to derive the degree of relationship or alignment.

Based on these four accuracy scales, an Overall Accuracy Index score is calculated to indicate the degree of confidence that can be placed in the accuracy of the assessment results. The TALENTx7 Assessment™ employs a 5-star system. The more stars that are filled-in, the more confident one is that a respondent's learning agility scores are accurate. If no star or only 1 star is filled-in, it signifies that the respondent's scores from this assessment should be discarded. When it occurs, it is strongly advised that the respondent should be asked to retake the assessment. Specific instructions should be given to the individual to increase the likelihood of obtaining useable results during the next administration (e.g., read each of the items carefully and fully, refuse the inclination to change previous responses, take the assessment in a quiet setting, avoid multitasking, complete it in one sitting).

Three Reports - For the Individual, the Coach, and the Organization
The TALENTx7 Assessment™ generates the following three reports: (a) Individual Feedback Report, (b) Professional Coach's Report, and (c) Organizational Report. The Individual Feedback Report, as well as the Professional Coach's Report, is given directly to a coach certified in the TALENTx7 Assessment™. The coach, in turn, sends the Individual Feedback Report to the learner for the feedback session. In addition, an Organizational Report listing all the individuals and their corresponding learning agility scores is provided to the designated talent management professional in the company who is coordinating administration of the assessment.

For all three reports, the scores are converted into percentiles. Percentiles convey a learner's scores relative to others in the normative data base. For example, a percentile score of 20 signifies the learner is higher than 20 percent of all the other individuals who have taken the assessment. Whereas, a percentile score of 40 signifies that one is higher than 40 percent of all other individuals who have taken the assessment.

A special feature of the TALENTx7 Assessment™ is the statistical mean or midpoint of the scale is designated as zero (0). Consequently, one-half of the scores are situated to the left of zero and one-half to the right. This approach was applied to signify that it is not necessarily bad to be in one direction or the other. Rather, it depends on individual and organizational context. Scores on the left reflect primarily a "technical or functional orientation." If the learner's scores on the
various facets of learning agility are left of zero, it suggests that he or she is conscientious, structured, precise, detail-oriented, and possesses deep technical or functional expertise. Those behavioral characteristics often are highly desirable for roles in engineering, accounting, research, quality control, data entry, and manufacturing.

In contrast, if the scores are situated to the right of zero (0), the learner likely is observant of his or her environment, willingly takes risks, frequently multitasks, is tolerant of ambiguity, and generally embraces organizational change. Those behaviors are associated with "leading others." Depending upon where one is in one's career, percentile scores may be lower or higher. If the individual is just beginning his or her career in leadership, one might expect scores to be slightly to the left or right of zero. In contrast, if the individual has been an executive for many years, one might expect scores to be to the right to a larger degree (i.e., beyond 20). Scores also can be influenced by the structure, culture, and managerial philosophy of the organization where an individual works and leads.

Note that scores can appear in the "red band" of the graph - either on the left side or on the right (i.e., situated in the 45-50 percentile ranges). When scores are in the red, it denotes the learner possesses a higher probably of career derailment or performing overuse behaviors. Individuals who are in the red on the left of zero might be too rigid in their behaviors, steadfast in their views, or get bogged down with details. Individuals who are in the red on the right of zero might be too learning agile, creating problems for members of their teams and/or organizations by driving too much change too quickly, taking too many risks, or pushing themselves and others to take on assignments before adequate analysis and discussion.

An individual's Overall Learning Agility score as well as scores on each of the seven facets are presented in a series of graphs on a single page entitled, "Your Overall Learning Agility Profile." This approach permits the individual (and coach) to view all the scores simultaneously and glean patterns of learning agility that emerge. Leadership potential strengths and weaknesses become apparent very quickly. In addition, a descriptive summary of the behaviors exhibited by individuals with similar learning agility scores is provided below the series of graphs.

Various "Behavioral Tendencies" and "Potential Derailers and Overuse Behaviors" for each of the seven facets also are presented in the report. Based on the individual's facet score, specific tendencies and derailers are identified. As an illustration, the visual on the next page highlights the information an individual receives for the "self insight" facet of learning agility. Notice that the behavioral tendencies associated with a score of 10 to the right of zero are enclosed in a black box. For example: "Clearly understand your personal strengths, limitations, and blind spots." "Clearly understand your personal beliefs, values, feelings, motives, and goals." "Can easily explain the reasoning and thought processes behind your behaviors and decisions." A coach providing feedback would ask the individual to check the square directly to the left of each of the behavioral tendencies that most resonate with him or her. Ones that are not checked can be discussed in terms of why they were not identified, whether it would be useful to improve in that area, and what he or she can do differently in the future.

The last page of the Individual Feedback Report provides a "Developmental Plan" template for the individual to complete after the feedback session. As part of this plan, the learner pinpoints specific behaviors to increase, as well as decrease or stop performing, to enhance their leadership effectiveness.

Unique Features of the TALENTx7 Assessment™
Several consulting firms market online self-assessments of learning agility. Some of these assessments measure two facets, some measure four, and one assesses five. The TALENTx7 Assessment™ is the only instrument that measures seven facets of learning agility, hence the name "TALENTx7." As identified previously, recent research findings support the importance of including a measurement of "environmental mindfulness" and "feedback responsiveness" in a comprehensive assessment of learning agility (Hulsheger, Alberts, Feinholdt, & Lang, 2013; Sheldon, Dunning, & Ames, 2014).

The inclusion of a facet on "self insight" also is very beneficial. In addition to being a vital component of the learning agility construct, a learner's "self insight" score provides the coach valuable information about how aware he or she is with regard to their learning agility scores in general. When the score on "self insight" is substantially to the left of zero, it suggests the individual likely will be quite surprised by the results of the assessment. In contrast, an individual who has a score to the right of zero likely is more knowledgeable about himself or herself. As a coach who has given several hundred feedback sessions, it is one of the first areas of learning agility I examine. A session is generally much easier to facilitate when the learner is aware of his or her strengths and weaknesses.

To ensure correct personnel decisions can be made, it is vital that any assessment of learning agility should be scientifically developed and validated. And once validated and normed, it is important the instrument be updated periodically to ensure relevancy. With regard to the TALENTx7 Assessment™, all data were collected in 2014. Further, the participants were employed in a variety of industries. Importantly, one-half held job titles of director or executive - the precise population for which the instrument was intended to gauge leadership potential. For a copy of the technical report, contact the author of this whitepaper (see De Meuse & Feng,
2015).

Another advantage of the TALENTx7 Assessment™ is its methodology for identifying respondents who may have deliberately attempted to alter their learning agility scores. Four psychometric scales are embedded in the assessment, and an easy-to-read Overall Accuracy Index provides the degree of confidence one can have in the veracity of the scores. In addition, the level of social desirability of an individual's responses is adjusted for in the learning agility scores. Hence, differences in the magnitude of learning agility across learners are due to the individual rather than an artifact of the assessment.

Conclusion
What do General Electric, IBM, Proctor & Gamble, EMC Insurance, and Verizon Communications have in common? They represent the Top 5 companies in the world for leadership practices today ("The Best Companies for Leaders," 2015). These organizations actively manage the process of leadership identification and development. Talent assessment is part of their culture. Leadership potential is scrutinized carefully and career plans are revisited annually. One of the most difficult lessons executives and talent management professionals must learn is to recognize past performance does not guarantee future performance, especially when it comes to promotion into leadership. Technical and functional skills that once were valued at lower levels need to give way to building teams, inspiring confidence, and focusing on long-term strategic goals.

The construct of learning agility provides a scientific means by which to evaluate leadership potential. Rather than basing talent decisions solely on past performance, limited interactions and observations, and hearsay, scores provide objective, quantifiable, and independent data on an employee's ability and willingness to succeed in higher-level positions. The TALENTx7 Assessment™ does just this! Talent decisions are based on facts and informed discussions. Employee strengths and developmental areas are more clearly identified. Furthermore, it levels the playing field, giving all employees an equal opportunity for leadership development regardless of their gender, race, or who may know them in the organization. Overall, it enables organizations to develop its next generation of leaders to meet the needs of an increasingly diverse workforce and the dynamic conditions of tomorrow's marketplace.

References

Anderson, C. D., Warner, J. L., & Spencer, C. C. (1984). Inflation bias in self-assessment examinations: Implications for valid employee selection. Journal of Applied Psychology, 69, 574-
580.

Charan, R. (2005). Ending the CEO succession crisis. Harvard Business Review, 83(2), 72-81.

De Meuse, K. P. (2015). Using science to identify future leaders: Part II - The measurement of learning agility. Minnetonka, MN: Wisconsin Management Group.

De Meuse, K. P., & Feng, S. (2015). The development and validation of the TALENTx7
Assessment™: A psychological measure of learning agility. Shanghai: Leader's Gene Consulting.

DeRue, D. S., Ashford, S. J., & Myers, C. G. (2012). Learning agility: In search of conceptual clarity and theoretical grounding. Industrial and Organizational Psychology: Perspectives on Science and Practice, 5, 258-279.

Dries, N., Vantilborgh, T., & Pepermans, R. (2012). The role of learning agility and career variety in the identification and development of high potential employees. Personnel Review, 41, 340-358.

Dunning, D., Heath, C., & Suls, J. M. (2004). Flawed self-assessments: Implications for health, education, and the workplace. Psychological Science in the Public Interest, 5(3), 69-106.

Goebel, S., & Baskerville, R. (2013, September). From self-discovery to learning agility in senior executives. Proceedings of the Third International Conference on Engaged Management Scholarship (pp. 1-19), Atlanta.

Hulsheger, U. R., Alberts, H. J. E. M., Feinholdt, A., & Lang, J. W. B. (2013). Benefits of mindfulness at work: The role of mindfulness in emotion regulation, emotional exhaustion, and job satisfaction. Journal of Applied Psychology, 98, 310-325.

Morgeson, F. P., Campion, M. A., Dipboye, R. L., Hollenbeck, J. R., Murphy, K., & Schmitt, N. (2007). Reconsidering the use of personality tests in personnel selection contexts. Personnel Psychology, 60, 683-729.

Sheldon, O. J., Dunning, D., & Ames, D. R. (2014). Emotionally unskilled, unaware, and uninterested in learning more: Reactions to feedback about deficits in emotional intelligence. Journal of Applied Psychology, 99, 125-137.

Silzer, R., & Church, A. H. (2009). The pearls and perils of identifying potential. Industrial and
Organizational Psychology: Perspectives on Science and Practice, 2, 377-412.

The Best Companies for Leaders. (2015, March 11). Retrieved from http://www.chiefexecutive. net/2015-best-compaies-leaders

About the Author

Dr. Kenneth P. De Meuse is founder and president of the Wisconsin Management Group, a consulting firm specializing in leader identification, executive coaching, and research on high potential talent. Dr. De Meuse is a global thought leader on the assessment and development of leadership, and has presented his research on learning agility and leadership competencies at numerous professional conferences, including the Academy of Management, American Psychological Association, Society for Human Resource Management, The Conference Board, International Coach Federation, Society for Consulting Psychology, and the Society for Industrial and Organizational Psychology. His 2010 journal article on learning agility is considered the first scholarly publication on the construct
and lays the foundation for its scientific exploration.

Throughout his career, Dr. De Meuse has consulted on a variety of strategic and leadership issues at businesses such as Nestle USA, Lucent Technologies, Siemens, RMS McGladrey, Presto Industries, and Ayres Associates. Prior to establishing the Wisconsin Management Group, he was executive vice president of Research and Product Development at Tercon Consulting, a global consulting firm headquartered in Washington, DC. He also was vice president of Global Research at Korn/Ferry International for six years. In addition, Dr. De Meuse was on the faculties of Iowa State University and the University of Wisconsin (Eau Claire). He has published more than 50 peer-reviewed journal articles and authored five books.

Dr. De Meuse earned his Ph.D. in Industrial/Organizational Psychology from the University of Tennessee and his Master's degree in Psychology from the University of Nebraska. In acknowledgement for his contributions to the science and practice of talent management, he was elected Fellow by the Society for Industrial and Organizational Psychology.