Better Tools Available for Assessing Mid-Level Managers

Share Article

3D Group releases improved 360 degree feedback tool based on extensive research.

3D Group, a consulting firm of industrial/organizational psychologists specializing in 360 degree feedback, has announced an improved, expanded release of their Corporate Leaders assessment tool. First issued in 2003 as part of the company’s popular Leadership Navigator™ series, the 360 degree assessment package is more valid, reliable, and practical than ever as a tool for gathering feedback on the work-related behaviors of manager and director-level employees.

The significant research effort that went into the new release also produced results that went beyond validating the accuracy and reliability of the competency model and survey instrument. For example, 3D group’s analysis confirmed that the sex of managers did not affect any job-related competency in any significant way. The study found significant differences between senior and junior managers, but only in competencies related to work processes. And on a nationwide basis, managers are more highly rated for their abilities to manage processes and results than for their interpersonal and team-building skills.

Research gives validation, insight

The new release is based on a national sample of more than 12,800 respondents who rated the performance of more than 1,000 managers – three times the size of the original study. This sample spans a broad range of industries and geographic regions. Because one goal was to assess the appropriateness of the competency scales for different demographic groups, the study included sufficient samples of male and female participants at a range of job levels.

“Numerous studies have already shown that differences in performance ratings with respect to sex are either small or insignificant,” said Dr. Rose. “To make sure the Corporate Leader norms were accurate for both men and women, we analyzed our data to gauge the extent to which sex was responsible for differences in competency means. The large sample size allowed us to use effect size analysis (eta2) rather than statistical significance. The results – that sex accounted for less than 1% of the difference in all but one of the competency means – showed that sex was not at all responsible for differences in scores.”

Experience matters

Analysis of the data showed interesting results for the other demographic measured, job level. For competencies in the Interpersonal (Team Leadership, Inclusiveness, Developing Talent) and Core (Communication, Integrity) areas, the study showed no significant relationship between job level and scores. However, in the area of Work Processes (Business Focus, Customer Focus, Results Orientation), senior managers tended to score higher than their more junior colleagues.

“Most people believe intuitively that more experienced managers will be better at managing processes and results. One would hope that we could also confirm that in a quantitative way,” said Dr. Rose. “Although we found the effects of seniority to be fairly weak, senior managers did tend to score higher on competencies in the Work Process area – especially Business Focus.”

Nationwide norms, national implications

Another important result of the study is the refinement of norms for each job-related competency. Besides giving survey participants an important perspective for interpreting their own scores, these nationwide average ratings help validate the tool itself, and shed light on national trends in corporate leadership. On average, managers consistently scored highest in Business Focus and Customer Focus. The lowest scores were in interpersonal areas, including Developing Talent, Inclusiveness, and Team Leadership.

“The differences in normative averages were relatively modest, from 4.10 to 4.33 on a 5-point scale. But because they were remarkably consistent from 2003 to 2005,” explains Dr. Rose, “we can be fairly confident that no radical change in nation-wide management skills have occurred since the original data were collected. Also, the fact that managers scored highest in the areas most clearly linked to business outcomes is not likely mere coincidence, given the fact that management compensation and promotion often depend on business results. One might interpret this as confirmation that companies are fairly good at identifying and developing leaders. On the other hand, some would argue that low average scores for leading teams and developing talent show a misplacement of priorities, one that might have long-term consequences for companies concerned with retention of the employees who report to these managers.”

Measuring leadership, guiding development

The Leadership Navigator™ series provides effective, easy-to-use tools for assessing and developing job-related competencies using 360-degree or multi-rater feedback. Assessments are based on short, user-friendly surveys of an individual’s co-workers, customers, supervisors, and direct reports. The resulting reports assess competencies in three areas: Work Process-related, Interpersonal, and Core traits. The research undertaken for the 2006 release makes this version one of the most up-to-date, thoroughly validated tools available – as well as one of the easiest to use.

In addition to the expanded national norms, the 2006 release of the Corporate Leader survey adds several new features to enhance customization and usability. Customizable options include an additional fifth group of raters, company-specific norms, and a management summary. Improved graphics, color-coding, company-specific labeling of rater groups, and detailed definitions of competency models and key terms make the reports easier to read and interpret. Finally, the feedback reports can be ordered with an optional development guide, with reading lists, on-the-job activities, and Internet resources for each competency.

For more information

A complete technical report on instrument reliability, national norm characteristics

by job level and function, and comparisons of scores is available from 3D Group 510.525.4830). The Leadership Navigator™ series is also available through other accredited channels such as the Society for Human Resource Management, (SHRM: http://www.shrm.org/assessment/products/3DGroup/3dgroup.asp) and Performance Assessment Network (PAN http://www.pantesting.com/products/3D/360_navigator.asp)

About 3D Group

3D Group specializes in 360 degree feedback and other employee assessment tools. They use their expertise in psychometrics – the specialized discipline of psychological testing and measurement – to deliver a wealth of objective, reliable, and easy to understand data that can be utilized in a variety of ways to improve manager and company performance. 3D Group is dedicated to helping businesses and nonprofit organizations of all sizes increase the effectiveness of their people and programs. 3D Group researchers have authored numerous publications related to employee assessment, including the “Benchmark Study of North American 360 Degree Feedback Practices, 2002 Study” and “Current Trends in 360 Degree Feedback.” More information can be found at: http://www.3dgroup.net.

###

Share article on social media or email:

View article via:

Pdf Print

Contact Author

Donna Lehman
3D Group
770-565-7275
Email >
Visit website