Regrettably I admit that too often I hear the “small voice” that Kath etal. (Reference Kath, Salter, Bachiochi, Brown and Hebl2021) discuss telling me that I should update my Industrial and Organizational (I-O) psychology class. I believe that the time it takes to develop a class is often ignored by university administration. A recent Association for Talent Development (ATD) survey of learning and development specialists estimated that every hour of traditional in-class instruction requires 38 hours of design and development time (Defelice, Reference Defelice2018). Furthermore, every hour of e-learning using real-time interactions creating life-like sets of complex cues and responses requires 143 hours of design and development time (Defelice, Reference Defelice2018).
A 3-credit-hour course is expected to have 37.5 classroom contact hours per semester (U.S. Department of Education, 2020). Using ATD’s survey results suggests that it would require 37.5 x 38 = 1,425 hours to develop a traditional face-to-face course! Of course, professors are already content experts so this estimate is likely on the high side. However, the position could be taken that college professors are not necessarily instructional design experts, so what is gained in being a content expert could be somewhat lost from having fewer instructional design skills. In addition, an entire semester of e-learning using real-time interactions creating life-like sets of complex cues and responses would require an astounding 37.5 x 143 = 5,362.5 hours to develop—that is, if professors had the technical skills to actually create such a course in the first place. Simply acknowledging the time requirements to develop a course would be a major improvement. I believe the time commitment is one of the main reasons why we continue to use the same exercises, tests, and textbooks term after term even when we hear that “small voice” in our heads. Regardless of this, our students would benefit if we commit to making a few improvements to our classes. Fortunately, a few small changes can help students learn the foundational knowledge of I-O psychology and dig deeper into some of the common issues for practitioners. Over the years I have developed a number of activities that I believe achieve this. In addition, I have found a few sneaky ways to illustrate important concepts that could otherwise be easily overlooked or dismissed. I would like to share them with the readers because they are easy to implement and can have a significant influence on student engagement and learning.
Gender and work exercises
I have successfully used two exercises that illustrate gender and work issues and uncover implicit gender bias. One exercise involves an online discussion concerning gender and work–family conflict. The specific question is not of particular importance. At the end of each week, students are required to complete a Blackboard survey that addresses topics and issues we covered during the week. Blackboard surveys only tell you whether a student completed a survey, and student responses are anonymous. I generally use this survey to wrap up discussions. For online classes, I provide a written summary of the survey results and the discussion, adding my two cents. This approach sneaks in a little extra engagement with the course materials, as tracking in Blackboard shows students opening the document even though there is no requirement to do so. Students also often comment on the results and participate beyond what is required. Finally, student evaluations show that they like this aspect of the class. For face-to-face classes, the following class period can begin with a discussion of the survey results.
For this gender and work–family conflict discussion, one of the survey questions concerns if a family has a sick child who should stay home to care for the child, the man or woman. Of course, this also introduces issues related to nontraditional families, which are briefly mentioned in the question stem and further discussed in the survey feedback. I have used this question many times and have yet to see anywhere near a 50/50 split. In fact, normally it is above 70% that the woman should stay home and care for the child. However, I have seen percentages higher than 90%.
Nearly every semester I get told, “but, it depends,” and that is a fair issue to raise because the question did not have an “it depends” option. This allows the class to get into measurement issues. If I put in a neutral “it depends” option, it would have received, by far, the most votes, and we would have learned very little. However, when I forced students to choose and take a side, it is interesting to note that a large majority of students selected the woman.
If there was not an overriding opinion, I would have expected the answers to be evenly split between “the man” and “the woman” because people would likely have selected an answer at random. I would also have expected many more blank responses, which for some reason is rare. Neither of these things happen. In fact, “the man” is always the first answer option, making it the easiest option to choose. Students had to actively skip the first answer option to select the “the woman.”
Finally, the discussion can get into priming. Because the students had just completed a discussion that addressed gender and work family conflict, they were likely “primed” to more carefully consider the question and their responses. One might expect this to result in a more even split between selecting the man and selecting the woman. Yet, this does not happen.
I have found that this one simple survey question can lead to enlightening discussions concerning measurement issues and how perceptions of gender roles affect work issues. It gets students to think more critically about both issues and can lead them to question their own deep-seated beliefs regarding work and gender.
The second exercise to demonstrate gender and work issues is based on a discussion that is unrelated to gender—on the surface. For a leadership discussion, I have students identify a person in their life who they see as a leader and analyze that person using a leadership theory(ies) from the textbook. After all of the students have made their initial post, using names and pronouns found in the posts, I count the number of male and female leaders. It has always been heavily skewed male. I provide this information to the students as a jumping-off point to further discuss gender and leadership. The discussion gets into expectations of traditional gender roles and leadership, implicit bias, and the effect that these have on promotion and advancement. Of course, there are many other directions this discussion could take.
I have observed over the years that male students are highly unlikely to identify a woman. The most common female leader identified in the discussion is the student’s mother. This will occur once or twice with each class. Wives will commonly identify their husband as a leader. However, husbands identify their wife very rarely (in the many years that I have used this discussion I believe it has only happened twice).
I should mention that a large portion of my students are affiliated with the military. They are nontraditional undergraduate students, and traditional undergraduate students could respond differently. However, I have not noticed different patterns, but I have not been looking for differences either. I have been using these exercises for many years and have found strong consistency in the results. Both activities help to bring to the forefront gender issues that many students likely believe are a thing of the past.
Test development exercise
Test development is a core activity in I-O psychology, and nearly all practitioners have been called upon to develop test questions. When covering testing, I have students read Haladyna and Downing’s (Reference Haladyna, Downing and Rodriguez2002) article on item-writing practices. Really any item-writing guide could be used, and if you do not have one, many are easily found online. In an online discussion forum, students have to write five multiple-choice items based on the testing textbook chapter. They must explain why the correct answer is correct, why the other answers are wrong, and provide the source (page number, etc.) from the textbook. Students must then comment on the questions of at least two other students. The comments must include recommended edits and corrections to improve the items. I also edit at least one question from each student when class size permits. Item writing is a developed skill that is extremely difficult to do well and extremely easy to do poorly, so finding corrections and improvements is easy.
What makes this exercise unique is that I use student-developed test questions in the chapter test later in the class. Students do not know this ahead of time. In all honesty, at times it can be hard to find a handful of useable test items. After the test, I present the students with the items and the item statistics. Most students express an interest in this information. It encourages more active engagement about testing, what makes a good item, how to evaluate items, and so on. I also discuss the bigger process of test development including test specifications, what makes a good item writer, and other issues. Anyone who has worked on a few test development projects will have more than enough entertaining stories about working with organizational leadership, subject matter experts, test graders, and examinees to illustrate the many challenges and concepts that are associated with testing and test development.
Based on course feedback, I can confidently report that students gain a greater appreciation of the complexity, difficulty, and seriousness of test development. They also start to develop much-needed writing and editing skills, encouraging conciseness and clarity: two important concepts in item development.
Performance evaluation exercise
When we cover the textbook chapter concerning performance evaluation, I require that students make a discussion post answering an assigned question. Then they must evaluate another student’s post using criteria that I provide. I have used a variety of scales, such as critical thinking and written communication, where students must rate the extent to which the post “meets 300-level-class expectations.” They must assign numerical scores and provide written feedback in SMART form: specific, measurable, achievable, relevant, and time based. To further challenge students, I require that they provide at least one area for improvement rather than giving all positive feedback.
Again, at the end of the week, students answer the weekly survey. Students tend to report that they enjoyed giving and receiving peer feedback. I always find this surprising because in my experience giving and receiving feedback is something with which I believe most people are somewhat uncomfortable. I have found that students are generally tough graders, sticking to the middle of the scale rather than the high end of the scale. However, they often provide inaccurate written feedback, such as indicating that the required APA style citations and references are correct when there are obvious errors.
In the survey feedback discussion, it is easy to bring up aspects of evaluation such as who is qualified to evaluate another’s performance. Many times, it is assumed that all supervisors are, but in my experience that is not the case. Are peers really qualified in this case? The discussion will often turn to what exactly are “300-level expectations,” which allows me to go into common-frame-of-reference training. The difference between developmental evaluations and administrative evaluations that are used for decision making is another area worth exploring. I ask the students if it was easier to be a tough evaluator knowing that their evaluation scores would not be used for the actual grades. At times this can lead into the often-overlooked organizational norms and politics of evaluations. Another topic that comes up frequently concerns code words that are used in performance evaluations (particularly in the military) that subtly indicate performance difficulties and how this can lead to problems when trying to take aversive personnel actions.
Conclusions
In the last weekly survey of the class, I have students give me feedback on a variety of aspects of the class. I have received consistently positive feedback on the activities presented above. However, I have had instances of the very conservative students expressing their support and belief in traditional family values and traditional gender roles. At least I still had them thinking about the issues at the end of the class. For experienced I-O psychology instructors, the activities presented in this article should be fairly easy to implement, requiring minimal preparation time. They can be modified to play to an instructor’s own experiences and strengths. I hope that some readers will experiment with these suggestions, and I welcome feedback and suggestions.
I offer one final piece of information. The last class survey also has students rank the textbook chapters in terms of which they enjoyed the most to the least. I categorize each chapter as an “I” topic or “O” topic. Despite my affinity for the I-side of things, the O-side has always been the clear winner. Heck, it is never even close.
You have
Access
Regrettably I admit that too often I hear the “small voice” that Kath etal. (Reference Kath, Salter, Bachiochi, Brown and Hebl2021) discuss telling me that I should update my Industrial and Organizational (I-O) psychology class. I believe that the time it takes to develop a class is often ignored by university administration. A recent Association for Talent Development (ATD) survey of learning and development specialists estimated that every hour of traditional in-class instruction requires 38 hours of design and development time (Defelice, Reference Defelice2018). Furthermore, every hour of e-learning using real-time interactions creating life-like sets of complex cues and responses requires 143 hours of design and development time (Defelice, Reference Defelice2018).
A 3-credit-hour course is expected to have 37.5 classroom contact hours per semester (U.S. Department of Education, 2020). Using ATD’s survey results suggests that it would require 37.5 x 38 = 1,425 hours to develop a traditional face-to-face course! Of course, professors are already content experts so this estimate is likely on the high side. However, the position could be taken that college professors are not necessarily instructional design experts, so what is gained in being a content expert could be somewhat lost from having fewer instructional design skills. In addition, an entire semester of e-learning using real-time interactions creating life-like sets of complex cues and responses would require an astounding 37.5 x 143 = 5,362.5 hours to develop—that is, if professors had the technical skills to actually create such a course in the first place. Simply acknowledging the time requirements to develop a course would be a major improvement. I believe the time commitment is one of the main reasons why we continue to use the same exercises, tests, and textbooks term after term even when we hear that “small voice” in our heads. Regardless of this, our students would benefit if we commit to making a few improvements to our classes. Fortunately, a few small changes can help students learn the foundational knowledge of I-O psychology and dig deeper into some of the common issues for practitioners. Over the years I have developed a number of activities that I believe achieve this. In addition, I have found a few sneaky ways to illustrate important concepts that could otherwise be easily overlooked or dismissed. I would like to share them with the readers because they are easy to implement and can have a significant influence on student engagement and learning.
Gender and work exercises
I have successfully used two exercises that illustrate gender and work issues and uncover implicit gender bias. One exercise involves an online discussion concerning gender and work–family conflict. The specific question is not of particular importance. At the end of each week, students are required to complete a Blackboard survey that addresses topics and issues we covered during the week. Blackboard surveys only tell you whether a student completed a survey, and student responses are anonymous. I generally use this survey to wrap up discussions. For online classes, I provide a written summary of the survey results and the discussion, adding my two cents. This approach sneaks in a little extra engagement with the course materials, as tracking in Blackboard shows students opening the document even though there is no requirement to do so. Students also often comment on the results and participate beyond what is required. Finally, student evaluations show that they like this aspect of the class. For face-to-face classes, the following class period can begin with a discussion of the survey results.
For this gender and work–family conflict discussion, one of the survey questions concerns if a family has a sick child who should stay home to care for the child, the man or woman. Of course, this also introduces issues related to nontraditional families, which are briefly mentioned in the question stem and further discussed in the survey feedback. I have used this question many times and have yet to see anywhere near a 50/50 split. In fact, normally it is above 70% that the woman should stay home and care for the child. However, I have seen percentages higher than 90%.
Nearly every semester I get told, “but, it depends,” and that is a fair issue to raise because the question did not have an “it depends” option. This allows the class to get into measurement issues. If I put in a neutral “it depends” option, it would have received, by far, the most votes, and we would have learned very little. However, when I forced students to choose and take a side, it is interesting to note that a large majority of students selected the woman.
If there was not an overriding opinion, I would have expected the answers to be evenly split between “the man” and “the woman” because people would likely have selected an answer at random. I would also have expected many more blank responses, which for some reason is rare. Neither of these things happen. In fact, “the man” is always the first answer option, making it the easiest option to choose. Students had to actively skip the first answer option to select the “the woman.”
Finally, the discussion can get into priming. Because the students had just completed a discussion that addressed gender and work family conflict, they were likely “primed” to more carefully consider the question and their responses. One might expect this to result in a more even split between selecting the man and selecting the woman. Yet, this does not happen.
I have found that this one simple survey question can lead to enlightening discussions concerning measurement issues and how perceptions of gender roles affect work issues. It gets students to think more critically about both issues and can lead them to question their own deep-seated beliefs regarding work and gender.
The second exercise to demonstrate gender and work issues is based on a discussion that is unrelated to gender—on the surface. For a leadership discussion, I have students identify a person in their life who they see as a leader and analyze that person using a leadership theory(ies) from the textbook. After all of the students have made their initial post, using names and pronouns found in the posts, I count the number of male and female leaders. It has always been heavily skewed male. I provide this information to the students as a jumping-off point to further discuss gender and leadership. The discussion gets into expectations of traditional gender roles and leadership, implicit bias, and the effect that these have on promotion and advancement. Of course, there are many other directions this discussion could take.
I have observed over the years that male students are highly unlikely to identify a woman. The most common female leader identified in the discussion is the student’s mother. This will occur once or twice with each class. Wives will commonly identify their husband as a leader. However, husbands identify their wife very rarely (in the many years that I have used this discussion I believe it has only happened twice).
I should mention that a large portion of my students are affiliated with the military. They are nontraditional undergraduate students, and traditional undergraduate students could respond differently. However, I have not noticed different patterns, but I have not been looking for differences either. I have been using these exercises for many years and have found strong consistency in the results. Both activities help to bring to the forefront gender issues that many students likely believe are a thing of the past.
Test development exercise
Test development is a core activity in I-O psychology, and nearly all practitioners have been called upon to develop test questions. When covering testing, I have students read Haladyna and Downing’s (Reference Haladyna, Downing and Rodriguez2002) article on item-writing practices. Really any item-writing guide could be used, and if you do not have one, many are easily found online. In an online discussion forum, students have to write five multiple-choice items based on the testing textbook chapter. They must explain why the correct answer is correct, why the other answers are wrong, and provide the source (page number, etc.) from the textbook. Students must then comment on the questions of at least two other students. The comments must include recommended edits and corrections to improve the items. I also edit at least one question from each student when class size permits. Item writing is a developed skill that is extremely difficult to do well and extremely easy to do poorly, so finding corrections and improvements is easy.
What makes this exercise unique is that I use student-developed test questions in the chapter test later in the class. Students do not know this ahead of time. In all honesty, at times it can be hard to find a handful of useable test items. After the test, I present the students with the items and the item statistics. Most students express an interest in this information. It encourages more active engagement about testing, what makes a good item, how to evaluate items, and so on. I also discuss the bigger process of test development including test specifications, what makes a good item writer, and other issues. Anyone who has worked on a few test development projects will have more than enough entertaining stories about working with organizational leadership, subject matter experts, test graders, and examinees to illustrate the many challenges and concepts that are associated with testing and test development.
Based on course feedback, I can confidently report that students gain a greater appreciation of the complexity, difficulty, and seriousness of test development. They also start to develop much-needed writing and editing skills, encouraging conciseness and clarity: two important concepts in item development.
Performance evaluation exercise
When we cover the textbook chapter concerning performance evaluation, I require that students make a discussion post answering an assigned question. Then they must evaluate another student’s post using criteria that I provide. I have used a variety of scales, such as critical thinking and written communication, where students must rate the extent to which the post “meets 300-level-class expectations.” They must assign numerical scores and provide written feedback in SMART form: specific, measurable, achievable, relevant, and time based. To further challenge students, I require that they provide at least one area for improvement rather than giving all positive feedback.
Again, at the end of the week, students answer the weekly survey. Students tend to report that they enjoyed giving and receiving peer feedback. I always find this surprising because in my experience giving and receiving feedback is something with which I believe most people are somewhat uncomfortable. I have found that students are generally tough graders, sticking to the middle of the scale rather than the high end of the scale. However, they often provide inaccurate written feedback, such as indicating that the required APA style citations and references are correct when there are obvious errors.
In the survey feedback discussion, it is easy to bring up aspects of evaluation such as who is qualified to evaluate another’s performance. Many times, it is assumed that all supervisors are, but in my experience that is not the case. Are peers really qualified in this case? The discussion will often turn to what exactly are “300-level expectations,” which allows me to go into common-frame-of-reference training. The difference between developmental evaluations and administrative evaluations that are used for decision making is another area worth exploring. I ask the students if it was easier to be a tough evaluator knowing that their evaluation scores would not be used for the actual grades. At times this can lead into the often-overlooked organizational norms and politics of evaluations. Another topic that comes up frequently concerns code words that are used in performance evaluations (particularly in the military) that subtly indicate performance difficulties and how this can lead to problems when trying to take aversive personnel actions.
Conclusions
In the last weekly survey of the class, I have students give me feedback on a variety of aspects of the class. I have received consistently positive feedback on the activities presented above. However, I have had instances of the very conservative students expressing their support and belief in traditional family values and traditional gender roles. At least I still had them thinking about the issues at the end of the class. For experienced I-O psychology instructors, the activities presented in this article should be fairly easy to implement, requiring minimal preparation time. They can be modified to play to an instructor’s own experiences and strengths. I hope that some readers will experiment with these suggestions, and I welcome feedback and suggestions.
I offer one final piece of information. The last class survey also has students rank the textbook chapters in terms of which they enjoyed the most to the least. I categorize each chapter as an “I” topic or “O” topic. Despite my affinity for the I-side of things, the O-side has always been the clear winner. Heck, it is never even close.