همکاری با انجمن روان‌شناسی صنعتی و سازمانی ایران

نوع مقاله : مقاله پژوهشی

نویسندگان

1 کارشناسی ارشد روان‌شناسی صنعتی و سازمانی، دانشکده روان‌شناسی و علوم تربیتی، دانشگاه اصفهان، اصفهان، ایران

2 استاد گروه روان‌شناسی، دانشکده علوم تربیتی و روان‌شناسی، دانشگاه اصفهان، اصفهان، ایران

3 استادیار گروه سنجش و اندازه‌گیری، دانشکده روان‌شناسی و علوم تربیتی، دانشگاه علامه طباطبائی، تهران، ایران

چکیده

نقش ارزیابان و آموزش آن‌ها به ‌منظور افزایش دقت و کاهش خطا در نتایج کانون ارزیابی انکارناپذیر است. از این‌رو هدف پژوهش حاضر مطالعه نقش تعدیلی دو نوع آموزش چهارچوب مرجع (FOR) طرحواره‌ای و رفتاری ارزیابان در رابطه بین نتایج کانون ارزیابی با ارزیابی عملکرد خود، همکاران و سرپرستان بوده است. بدین منظور در مرحله نخست پژوهش مهارت ارزیابی به دو گروه از دانشجویان دکتری و ارشد رشته‌های روان‌شناسی با گرایش‌های مختلف به شیوه چهارچوب مرجعِ طرحواره‌ای و رفتاری آموزش ‌داده‌ شد و به گروه سوم به‌ عنوان گروه کنترل هیچ آموزشی ارائه نشد و از آن‌ها خواسته شد در کانون ارزیابی شایستگی‌های دو گروه 28 نفری از مدیران مجموعه صنایع واقع در شهر اصفهان را رتبه دهند. لازم به توضیح است شایستگی‌های مورد ارزیابی شامل قضاوت، رهبری، برنامه‌ریزی، روابط بین‌فردی و پردازش اطلاعات بوده است. این شایستگی‌ها در تمرین‌های شبیه‌سازی جستجوی حقیقت، بحث گروهی، ایفای نقش و ارائه شفاهی بود. سپس بعد از گذشت 6 ماه در فاز دوم پژوهش از خود مدیران، همکاران و نیز سرپرستان مدیران خواسته شد تا فرم ارزیابی عملکرد (Nouri et al., 2007) را تکمیل کنند که در تحلیل‌ها به‌عنوان متغیر ملاک در‌نظر گرفته شود. با استفاده از ضریب همبستگی گشتاوری پیرسون و تبدیل z فیشر به رابطه میان نتایج کانون ارزیابی و ارزیابی عملکرد (خود، همکاران و سرپرستان) و نیز تفاوت همبستگی پرداخته شد. از مهم‌ترین نتایج می‌توان به تأثیر معنی‌دار آموزش ارزیابان به دو شیوه چهارچوب مرجعِ طرحواره‌ای و رفتاری اشاره کرد. اگرچه مقادیر همبستگی بین ارزیابی عملکرد و نتایج کانون با ارزیابانی که آموزش طرحواره‌ای دیده‌اند کمی بیشتر از ارزیابانی بود که آموزش رفتاری داشته‌اند، لیکن این تفاوت معنی‌دار گزارش نشد. علاوه بر‌این، راجع به شیوه‌های آموزش ارزیاب و نیز جهت‌گیری پژوهش‌های آتی و همچنین پیشنهادهای کاربردی و پژوهشی بحث شده است.

کلیدواژه‌ها

عنوان مقاله [English]

Moderating Role of Schema and Behavioral Training in the Relationship between Assessment Center Results with Performance Appraisal of Self, Colleagues and Supervisors

نویسندگان [English]

  • Fatemeh Zamanpour Boroujeni 1
  • Hamid Reza Oreyzi Samani 2
  • Enayatollah Zamanpour 3

1 MA of Industrial Psychology, Department of Psychology, Faculty of Educational Sciences and Psychology, Isfahan University, Isfahan, Iran

2 Professor, Department of Psychology, Faculty of Educational Sciences and Psychology, Isfahan University, Isfahan, Iran

3 Assistant Professor, Department of Assessment and Education, Faculty of Psychology and Education, Allameh Tabataba’i University, Tehran, Iran

چکیده [English]

Introduction
The role of assessors and their training in order to increase accuracy and reduce errors in the results of the assessment center is undeniable. Two types of schema and frame of reference (FoR) training are commonly used in rater training. Also, there have always been differences between the results of the assessment center and the results of self, peer, and observer evaluation. Therefore, the purpose of this research was to identify the moderating role of two types of schema training and the behavioral frame of reference (FOR) of the assessor in the relationship between the results of the assessment center and the performance evaluation of self, peers and supervisors.
 
Method
For this purpose, in the first phase of the research, assessment skills were trained to two groups of PhD and senior psychology students with different orientations using a schema and behavioral of frame of reference, and the third group was not provided with any training as a control group that asked them to assess two groups of 28 managers of the industrial complex located in Isfahan city. It is necessary to explain that the evaluated competencies included judgment, leadership, planning, interpersonal relations and information processing. These competencies were in fact finding simulation exercises, group discussion, role playing and oral presentation. Then, after 6 months in the second phase of the study, managers, peers and supervisors were asked to complete the performance appraisal form (Nouri .et al, 2007) to be used as a criteria variable in the analysis. Pearson product-moment correlation coefficient and Fisher Z-Transformation, the relationship between results of assessment center and performance appraisal (self, peer and supervisor) and the correlation difference were investigated.
 
Results
One of the most important results has been the effectiveness of training of assessors. In other words, the training of assessors of the schema and behavioral frame of reference (FoR) has been effective. Although the correlation between performance evaluation and assessment center’s results with assessors who received schema training is slightly higher than assessors who received behavioral training, this difference was not statistically significant.
 
Discussion
Training the assessors of the reference framework, either behaviorally or schematically, is one of the basic aspects of the assessment center (AC) and providing accurate results along with error reduction. Although there are differences between these types of training, but based on the results of this research, it can be concluded that the appraisal of self, peer, and supervisor are more similar to evaluators who have been trained in a schematic way. This result is due to the fact that in this type of training, the generality of behavior is evaluate, while in behavioral training, individual behaviors are observed,  recorded, classified, and evaluated. In addition, assessor training methods as well as the direction of future research and practical and research suggestions are discussed.

کلیدواژه‌ها [English]

  • Assessment center
  • fram-of-reference traning
  • self-performance appraisal
  • peer-performance appraisal
  • supervisor-performance appraisal
Armoneit, C., Schuler, H., & Hell, B. (2020). Nutzung, validität, praktikabilität und akzeptanz psychologischer personalauswahlverfahren in deutschland 1985, 1993, 2007, 2020: Fortführung einer trendstudie. Zeitschrift Für Arbeits-Und Organisationspsychologie, 64(2), 67-82. https://doi.org/10.1026/0932-4089/a000311
Badali, Z., & Amirkhani, A. (2013). Determining the key competencies of evaluation center evaluators. Organizational Behavior Studies Quarterly, 1(4), 61-81. (Persian) https://sid.ir/paper/472425/fa
Balantain, Y., & Poe, N. (2006). Managers evaluation and development centers (Translated by Masoud Soltani). Tehran: Asia. (Persian) https://www.gisoom.com/book/1407759/
Baret, N. (2018). How does frame-of-reference training increase rater accuracy? A test of potential explanatory mechanisms. school of management studies. https://open.uct.ac.za/bitstreams/43cc09af-b6eb-4e51-89b3-f51f19117558/
Bastian, P. K., Leonore Titze, J., & Blickle, G. (2022). Improving assessment center criterion validity for salesperson selection: A socioanalytic approach. Journal of Personal Selling & Sales Management, 42(2), 1-16. https://doi.org/10.1080.08853134.2022.2037436
Bernardin, H. J., & Buckley, M. R. (1981). Strategies in rater training. Academy of Management Review, 6, 205-212.  https://doi.org/137.158.158.60
Breil, S. M., Lievens, F., Forthmann, B., & Back, M. D. (2022). Interpersonal behavior in assessment center role‐play exercises: Investigating structure, consistency, and effectiveness. Personnel Psychology, 76(3), 759–795. https://doi.org/10.1111/peps.12507
De Kock, F. S., Lievens, F., & Born, M. (2018). The profile of the ‘good judge’ in HRM: A systematic review and agenda for future research. Human Resource Management Review, 30(2), 100667 https://doi.org/10.1016.j.hrmr.2018.09.003
Fiske, S. T., & Taylor, S. E. (2013). Social cognition: From brains to culture. London: SAGE. https://books.google.com/books/about/Social_Cognition.html?id=FYth8i4Zg78C
Gibbons, A. M., & Rupp, D. E. (2009). Dimension consistency as an individual difference: A new (old) perspective on the assessment center construct validity debate. Journal of Management, 35(5), 1154-1180. https://doi.org/10.1177/0149206308328504
Gorman, C. A. (2008). Evaluating frame-of-reference rater training effectiveness via performance schema accuracy. PhD diss, University of Tennessee. https://doi.org/10.1037/a0016476
Gorman, C. A., Jackson, D. J., Meriac, J. P., Himmler, J. R., & Contreras, T. F. (2023). Beyond rating accuracy: Unpacking frame-of-reference assessor training effectiveness. Industrial and Organizational Psychology, 1(2), 1-14. https://doi.org/10.1017/iop.2024.6
Hauenstein, N. M. A., & McCusker, M. E. (2017). Rater training: Understanding effects of training content, practice ratings, and feedback. International Journal of Selection and Assessment, 25, 253-266. https://doi.org/10.1111/ijsa.12177
Hosseini, Y., Arman, M., & Mohamadizade, Z. (2017). The role of the evaluation center in staff improvement programs using artificial neural networks. Organizational Resource Management Studies, 7(3), 1-11. (Persian) http://ormr.modares.ac.ir/article-28-8427-fa.html
International Taskforce on Assessment Center Guidelines. (2015). Guidelines and ethical considerations for assessment center operations. Journal of Management, 41(4), 1244-1273. https://doi.org/10.1177/0149206314567780
Jamshidi, H., Badali, Z., & Hosseini, S. M. (2017). Key success factors in evaluation and development centers. Tehran: Industrial Management Organization. (Persian) https://www.gisoom.com/book/11331641/
Jansen, P. G. W., & Vinkenburg, C. J. (2006). Predicting management career success from assessment center data: A longitudinal study. Journal of Vocational Behavior ,68(2), 253-266. https://doi.org/10.1016/j.jvb.2005.07.004
Kark, R., Blatt, R., & Wiesel, V. (2023). A woman’s got to be what a woman’s got to be? How managerial assessment centers perpetuate gender inequality. Human Relations, 0(0), 1-12. https://doi.org/10.1177/00187267231161426
Khoshouei, M. S., Oreizy, H., & Nouri, A. (2014). Design and implementation of a manager promotion evaluation center. Quarterly Journal of Human Resource Management Research, Imam Hossein University, 6(3), 27-47. (Persian) https://hrmj.ihu.ac.ir/article_11748.html
Kogan, J. R., Dine, C. J., Conforti, L. N., & Holmboe, E. S. (2022). Can rater training improve the quality and accuracy of workplace-based assessment narrative comments and entrustment ratings? A randomized controlled trial. Academic Medicine, 98(2), 237-247. https://doi.org/10.1097/ACM.0000000000004819
Krause, D, E., & Thornton, G. C. (2009). A Cross-cultural look at assessment center practices: Survey results from western europe and north america. Applied Psychology: An International Review, 58(4), 557-585. https://doi.org/10.1111/j.1464-0597.2008.00371.x
Krause, D. E., Rossberger, R. J., Dowdeswell, K., Venter, N., & Joubert, T. (2011). Assessment center practices in South Africa. International Journal of Selection and Assessment, 19(3), 262-275. https://doi.org/10.1111/j.1468-2389.2011.00555.x
Landau, M. J., Meier, B. P., & Keefer, L. A. (2010). A metaphor-enriched social cognition. Psychological Bulletin, 136(6), 1045-1067. https://doi.org/10.1037/a0020970
Levin, K. (2023). Understanding the impact of age and gender demographic similarity in assessment center and individual assessment ratings. Doctoral dissertation, Illinois Institute of Technology. https://www.proquest.com/openview/
Lievens, F., & Sanchez, J. I. (2007). Can training improve the quality of inferences made by raters in competency modeling? A quasi‐experiment. Journal of Applied Psychology, 92(3), 812-819. https://doi.org/10.1037.0021‐9010.92.3.812
Lievens. F. (2001). Assessor Training Strategies and Their Effects on Accuracy, Interrater Reliability, and Discriminant Validity. Journal of Applied Psychology, 86, No. 2, 255-264. https://psycnet.apa.org/doi/10.1037/0021-9010.86.2.255
Maciejczyk, M. (1991). The relationship of self-ratings to assessor ratings of assessment center performance. Dissertation Abstracts International, 52, 4505. https://doi.org/10.31274/RTD-180813-9506
Macrae, C. N., & Bodenhausen, G. V. (2000). Social cognition: Thinking categorically about others. Annual Review of Psychology, 51(1), 93-120. https://doi.org/10.1146/annurev.psych.51.1.93
Meriac, J. P., Hoffman, B. J., & Woehr, D. J. (2014). A conceptual and empirical review of the structure of assessment center dimensions. Journal of Management, 40(5), 1269-1296. https://doi.org/10.1177/0149206314522299
Nouri, A., Labaf, H., Oreyzi, H., Rumi, H., Gorgani, H., … & Janatian, S. (2007). Psychological and comprehensive analysis of the performance appraisal system and career path of the managers of the National Iranian Gas Company and the province's gas companies. Isfahan. (Persian) Research project of Isfahan University
O’Neill, G., Travaglione, A., McShane, S., Hancock, J., & Chang, J. (2017). Converting values awareness to values enactment through frame-of-reference training. International Journal of Organizational Analysis, 25, 536-558. https://doi.org/10.1108/IJOA-02-2016-0975
Oreyzi, H., & Asadi, A. (2017). Design and implementation of Strategic Thinking Competency Assessment Center for managers of Iran Petroleum Products Refining and Distribution Company. Two Quarterly Journal of Contemporary Psychology, 12(1), 53-62. (Persian) https://bjcp.ir/article-1-1181-fa.html
Powell, D. M., & Bourdage, J. S. (2016). The detection of personality traits in employment interviews: Can “good judges” be trained? Personality and Individual Differences, 94, 194-199. https://doi.org/10.1016.j.paid.2016.01.009
Powell, D. M., & Goffin, R. D. (2009). Assessing personality in the employment interview: The impact of training on rater accuracy. Human Performance, 22, 450-465. https://doi.org/10.1080.08959280903248450
Pulakos, E. D. (1986). The development of training programs to increase accuracy with different rating tasks. Organizational Behavior and Human Decision Processes, 38, 76-91. https://doi.org/10.1016/0749-5978(86)90027-0
Radi Afsouran, N., Thornton, G., & Charkhabi, M. (2022). Leadership development assessment center: A review on advantages and disadvantages for developing leadership behavioral competencies. International Journal of Organizational Leadership, 11(1), 71-88. https://ijol.cikd.ca/article_60622_en.html
Roch, S. G., & O’Sullivan, B. J. (2003). Frame–of–reference rater training issues: Recall, time and behavior observation training. International Journal of Training and Development, 7, 93-107. https://doi.org/10.1111/1468-2419.00174
Roch, S. J., Woehr, D. J., Mishra,V., & Kieszczynska, U. (2012). Rater training revisited: An updated meta-analytic review of frame-of-reference training. Journal of Occupational and Organizational Psychology, 85, 370-395. https://doi.org/10.1111/j.2044-8325.2011.02045.x
Sackett, P. R., Shewach, O. R., & Keiser, H. N. (2017). Assessment centers versus cognitive ability tests: Challenging the conventional wisdom on criterion-related validity. J. Appl. Psychol, 102, 1435-1447. https://doi.org/10.1037.ap1000 0236
Schleicher, D. J., Day, D. V., Mayes, B. T., & Riggio, R. E. (2002). A new frame for frame‐of‐reference training: Enhancing the construct validity of assessment centers. Journal of Applied Psychology, 87(4), 735-746. https://doi.org/10.1037.0021‐9010.87.4.735
Shakeri, A., Mousavi, Z., Noori, A., & Oreyzi, H. (2018). 360-degree evaluation validity through the evaluation center in the National Iranian Gas Company. Quarterly Journal of Human Resource Management in the Oil Industry, 11(41), 243-262. (Persian) https://www.sid.ir/paper/196594/fa
Shakery, I. (2016). Validating the assessment of traditional performance evaluation and competency-based assessment through the assessment center. Industrial and Organizational Psychology Studies, 3(1), 57-70. (Persian) https://doi.org/10.22055/jiops.2018.21906.1049
Shakery, I., Mehdad, A., & Farhadi, A. (2023). Designing and validating the test of situational judgment mental toughness of managers in the Iran Alloy Steel Company. Industrial and Organizational Psychology Studies, 10(1), 43-56. (Persian) https://doi.org/10.22055/jiops.2023.43449.1347
Taylor, I. (2007). A practical guide to assessment centres and selection method: Measuring competency for recruitment and development. London: Kogan Page. https://books.google.com/books/about/
Thornton, G. C., & Lievens, F. (2018). Theoretical principles relevant to assessment center design and implementation. Assessment centres: Unlocking potential for growth. 1-36. Research Collection Lee Kong Chian School of Business. Available at: https:ink.library.smu.edu.sg.lkcsb_research.6222.
Thornton, G. C., & Rupp, D. E. (2006). Assessment centers in human resource management: Strategies for prediction, diagnosis and developmant. Lawrence Erblaum Associates, 1(2). 3-23. https://doi.org/10.4324/9781410617170
Thornton, G. C., Rupp, D., & Mueller-Hanson, R. A. (2004). Developing organizational simulations: A guide for practitioners and students. Lawrence Erlbaum Associat. https://doi.org/10.4324/9781410609458
Thornton, J. (2007). Centers for evaluating human resource management(Translation: Saeed Jafari, Saeed Shahbazi, Majid Salimi). Tehran: University of Tehran, Faculty of Entrepreneurship. (Persian) https://www.gisoom.com/book/1442483/
Tsai, M. H., Wee, S., & Koh, B. (2019). Restructured frame‐of‐reference training improves rating accuracy. Journal of Organizational Behavior, 40(6), 740-757. https://doi.org/10.1002/job.2368
Vanhove, A. J., Graham, B. Z., & Thornton, G. C. (2023). Moderators of sex‐and race‐based subgroup differences in assessment center ratings: A meta‐analysis. International Journal of Selection and Assessment, 31(1), 63-91. https://doi.org/10.1111/ijsa.12411
Wirz, A., Melchers, K. G., Kleinmann, M., Lievens, F., Annen, H., & Blum, U. (2020). Do overall dimension ratings from assessment centers show external construct-related validity? European Journal of Work and Organizational Psychology, 29(3), 405-420. https://doi.org/10.1080/1359432X.2020.1714593
Woehr, D. J., & Huffcutt, A. I. (1994). Rater training for performance appraisal: A quantitative review. Journal of Occupational and Organizational Psychology, 67, 189-205. https://doi.org/10.1111/j.2044-8325.1994.tb00562.x
Zamanpour, E. (2017). Validation of the competency model of oil industry managers in evaluation and development centers: Application of procedural theory. Doctoral dissertation, Allameh Tabatabai University, Tehran. (Persian) https://ganj.irandoc.ac.ir/viewer/0bace92075ee5dc46fcfeefb63636e6b?sample=1