%20-%20https://sketch.com%20--%3e%3ctitle%3e3ECBBF50-FD08-412A-943D-2CFC5E025A16%3c/title%3e%3cdesc%3eCreated%20with%20sketchtool.%3c/desc%3e%3cg%20id='Stylesheet'%20stroke='none'%20stroke-width='1'%20fill='none'%20fill-rule='evenodd'%3e%3cg%20transform='translate(-926.000000,%20-668.000000)'%20id='BUTTONS'%3e%3cg%20transform='translate(100.000000,%20597.000000)'%3e%3cg%20id='BTN-/-RIGHTS'%20transform='translate(816.000000,%2065.000000)'%3e%3cg%20id='ICON-/-Rights'%20transform='translate(10.000000,%206.000000)'%3e%3cpath%20d='M13.5752841,0%20L1.34943182,0%20C0.59375,0%200,0.586666667%200,1.33333333%20L0,22.6666667%20C0,23.4133333%200.620738636,24%201.34943182,24%20L17.6505682,24%20C18.40625,24%2019,23.3866667%2019,22.6666667%20L19,5.36%20L13.5752841,0%20Z%20M13.5752841,1.89333333%20L17.0838068,5.36%20L13.5752841,5.36%20L13.5752841,1.89333333%20Z%20M1.34943182,22.6666667%20L1.34943182,1.33333333%20L12.1988636,1.33333333%20L12.1988636,5.36%20C12.1988636,6.10666667%2012.7926136,6.69333333%2013.5482955,6.69333333%20L17.6235795,6.69333333%20L17.6505682,22.6666667%20L1.34943182,22.6666667%20Z'%20id='Shape'%20fill='%23595959'%20fill-rule='nonzero'%3e%3c/path%3e%3ccircle%20id='Oval'%20stroke='%23595959'%20stroke-width='1.5'%20fill='%23595959'%20cx='9.5'%20cy='13.5355339'%20r='5.5'%3e%3c/circle%3e%3cpolyline%20id='Path-2'%20stroke='%23FFFFFF'%20stroke-width='1.5'%20transform='translate(9.500000,%2013.035534)%20rotate(45.000000)%20translate(-9.500000,%20-13.035534)%20'%20points='8%2016.0355339%2011%2016.0355339%2011%2010.0355339'%3e%3c/polyline%3e%3c/g%3e%3c/g%3e%3c/g%3e%3c/g%3e%3c/g%3e%3c/svg%3e)
In summer 2018, the University of Chicago made national headlines by removing the requirement that applicants must submit the SAT or ACT when applying for undergraduate admission. Princeton University and Brown University soon joined this august group of “test-optional” universities, citing concerns about equity and diversity (Truong Reference Truong2018). Although these three universities are perhaps the most prominent schools to make this decision, more than 1,000 other colleges and universities have abandoned the standardized-test requirement for undergraduate admissions (Posselt and Miller Reference Posselt and Miller2018).
Whereas undergraduate admissions have been the subject of most of the national attention regarding standardized testing, it also is important to consider the role of standardized testing in graduate admissions. Studies questioned whether standardized tests—most notably the Graduate Record Examination (GRE)—discriminate against female and minority candidates (Miller and Stassun Reference Miller and Stassun2014). In addition, a host of studies queried the practices that Master of Public Administration (MPA) programs use when admitting students—that is, whether standardized tests are “an admissions obstacle” (Darolia, Potochnick, and Menifield 2014, 3) and whether those practices predict future student success (e.g., Bowman Reference Bowman1988; Leavitt, Lombard, and Morris Reference Leavitt, Lombard and Morris2011; Menifield et al. 2007; Thompson and Kobrak 1983). Despite these debates, most signs indicate that MPA programs have not moved away from standardized testing for admissions. According to the accrediting body for MPA programs—the Network of Schools of Public Policy, Affairs, and Administration (NASPAA)— “most schools require standardized test scores, such as the GRE or GMAT” for admission (NASPAA 2018).
This article catalogs the prevalence of standardized testing in MPA admissions.Footnote 1 The results should be of interest to graduate-school faculty and decision makers who face a fundamental challenge: how to design an admissions process that screens out applicants who are unlikely to succeed but does not provide too high an entry barrier for students who can be successful. This analysis should be of interest also to scholars of higher-education administration, more generally—particularly those who are concerned with issues of access, diversity, and barriers to admission.
STANDARDIZED TESTING IN GRADUATE PROGRAMS
According to previous work (Bowman Reference Bowman1988), the GRE is the most common standardized test used in MPA program admissions. Created in 1949 by Educational Testing Service (ETS), the test was designed to measure four areas: verbal reasoning, quantitative reasoning, analytical writing, and critical-thinking skills (Buchy Reference Buchy2017). Although these broad goals remain, many details about administration of the test have shifted over time. The GRE underwent a major revision in 2011, shifting from a 120–180 to a 130–170 scoring range. The 2011 revision also shifted the GRE from a “continuously-adaptive-test” examination to an “adaptive-by-section” examination. In the current version, students complete one verbal-reasoning and one quantitative-reasoning section; the computer then adapts the difficulty of the questions for the second quantitative and second verbal sections (Manhattan Review 2018). There also is an analytical-writing section that is scored on a 0–6 scale.
Research on the role of standardized testing in MPA admissions illustrates that the GRE has long been used in admissions decisions, although it always has been the subject of considerable controversy. Indeed, questions about its predictive accuracy and effect on program diversity have been pondered since at least the early 1980s (Thompson and Kobrak 1983). Bowman (1988, 871) summarized the state of the test in MPA admissions decisions as both “frequently used and contentious”; he reported that in 1985 80% of graduate programs and 66% of MPA programs used the GRE in admissions decisions.
This article catalogs the prevalence of standardized testing in MPA admissions. The results should be of interest to graduate-school faculty and decision makers who face a fundamental challenge: how to design an admissions process that screens out applicants who are unlikely to succeed but does not provide too high an entry barrier for students who can be successful.
Bowman’s (1988, 871) analysis not only reinforces the ubiquity of the GRE; it also supports its use in some circumstances, noting that “GRE scores, especially when consistent with other evidence…can lead to more informed judgments.” In a related study, Bowman and Mangelsdorf (Reference Bowman and Mangelsdorf1989) asked NASPAA principal representatives to participate in an experiment in which they made admissions decisions on a series of fictitious candidates who varied by grade point average (GPA), GRE, quality of letters of recommendation, race, and sex. GRE scores were a common and important predictor of the representatives’ ultimate decisions on whether to admit a candidate—falling second only to undergraduate GPA. Clearly, at the end of the 1980s, MPA stakeholders had received the message that the use of the GRE was common in admissions decisions and consistent with best practices in the discipline. Similarly, research in political science PhD admissions found that—at least in one elite program—the quantitative section of the GRE was an excellent predictor of graduate-school success (King, Bruce, and Gilligan Reference King, Bruce and Gilligan1993).
In the early 2000s, MPA programs began experimenting with GRE waivers. Although the specific criteria for waivers varied by school, undergraduate GPA, professional experience, and interview outcome seemed to be among the most common criteria for acquiring a waiver (Bowman Reference Bowman1988; Menifield et al. 2007). Some research found that the academic success of mid-career students who received GRE waivers is statistically indistinguishable from students who were required to take the GRE. This finding suggests that GRE waivers might be appropriate for programs that are attempting to recruit more mid-career professionals (Menifeld et al. 2007).
Similar studies likewise concluded that the GRE waiver may be a good option for programs that want to increase their share of part-time students without sacrificing quality (Gibson et al. Reference Gibson, Leavitt, Lombard and Morris2007; Leavitt, Lombard, and Morris Reference Leavitt, Lombard and Morris2011). Indeed, they found that the professionally experienced students who obtained a GRE waiver outperformed their less-experienced counterparts who were required to stake a standardized test before admission.
Others scholars provide a cautionary note for those who want to eliminate the GRE from MPA admissions. There is evidence that the GRE can provide value in predicting student success (Darolia, Potochnick, and Menifield 2014). Furthermore, mid-career students who receive a GRE waiver may have a lower academic profile and experience less success in graduate school than those admitted under traditional requirements (however, the effects on graduate-student success disappear when controlling for undergraduate academic indicators) (Darolia, Potochnick, and Menifield 2014). Clearly, there is a healthy debate within public administration and public affairs programs about whether and how to adapt to changing norms of standardized testing.
DATA AND THE HYPOTHESES
To better understand the role of standardized testing in admissions decisions, we coded the admissions criteria for 185 NASPAA-accredited programs. Following Reinagel and Gerlach (2015), this represents the universe of programs accredited as of September 1, 2017, with the exception of nine foreign universities, the University of Puerto Rico, and two programs at the Naval Postgraduate School. These 12 programs were eliminated from our analysis because of language differences (i.e., in 10 programs) and differences in program objectives (i.e., the Naval Postgraduate School). The unit of analysis in this study was the accredited program. Whereas most universities have one accredited program, some have multiple accreditations; in those cases, the university appears multiple times in our dataset but each program appears only once.
Most programs clearly delineated admissions requirements on their website, but some were housed on the graduate-admissions page, in the graduate catalog, or in other locations on the university website. After we identified the location of the information, we coded several factors for each program: (1) Do they require a standardized test for admission?; (2) If yes, which exam(s) do they accept?; (3) Do they make exceptions or grant waivers?; (4) If they make exceptions or grant waivers, what are the reasons that applicants may give for obtaining a waiver?; and (5) Which other materials are required for admission? We then combined these original data with two existing datasets. In an attempt to determine whether admissions standards differ by program prestige, we merged our original data with the US News & World Report Public Affairs (2018) rankings. To determine whether admissions standards differ by other program-level metrics (e.g., student diversity and program size), we also merged publicly available data about these factors from NASPAA. Although these program-level metrics are the best and most systematic data available, they rely on self-reporting and a significant number of cases are missing.
The logic behind the prestige variable is simple: We expect that more prestigious programs are concerned that they might be sending negative signals about program quality if they were to eliminate the GRE. We termed this the “prestige hypothesis.” We also included a variable that measures the percentage of a program’s student body that is from diverse backgrounds. Here, we expect programs that traditionally draw on more-diverse pools of applicants may have opted against the GRE, given the growing literature citing concerns about racial bias inherent in standardized testing. Finally, we included a measure for the program size, with the expectation that larger programs might not experience the market pressure and therefore be more likely to keep the standardized-testing requirement. We labeled this the “market hypotheses.”
RESULTS
Figure 1 shows the prevalence of standardized testing across MPA programs. The figure displays three bars representing the percentage of programs that: (1) require a standardized test for all applicants; (2) provide standardized-test waivers for some students; and (3) do not require a standardized test. As figure 1 suggests, far from being the ubiquitous choice of most programs, standardized tests are required universally for admission in less than one third of NASPAA-accredited programs. More than one third of MPA programs provide waivers for some students and more than one third do not require standardized tests for any applicant.
This initial analysis suggests that program directors and MPA faculty can rest assured that if they eschew standardized testing, they will not be alone.
Prevalence of Standardized Testing in NASPAA Accredited MPA Programs
This initial analysis suggests that program directors and MPA faculty can rest assured that if they eschew standardized testing, they will not be alone. The question remains, however, about which programs are most likely to keep the standardized-testing requirement and which are most likely to either adopt waivers or entirely eliminate standardized testing. The left-hand column in table 1 presents results of an ordinal logistic regression model testing the prestige, market, and diversity hypotheses discussed previously. The dependent variable is coded on a 0–2 scale in which “0” represents a program that has a standardized-test requirement for all applicants. A “1” indicates that the program has a standardized-test waiver; a “2” indicates that the program entirely eliminated standardized testing.
Modeling Program-Level Admissions Requirements
Note: The first model is an ordinal logistic regression model and the second is an ordinary least squares regression model. *p<.05, two-tailed test.
None of these variables achieves traditional levels of statistical significance, which suggests that neither diversity, nor market forces, nor prestige explains why programs adopt or eschew standardized testing. Focusing on prestige, less than half of the NASPAA-accredited MPA programs ranked in US News & World Report’s top 20 require a standardized test for all students; slightly more than a quarter offer GRE waivers; and slightly more than a quarter entirely eliminated the test. Clearly, requiring the GRE is not a prerequisite for cultivating a respected MPA program.
Of the programs that require a standardized test, there is no doubt that the GRE is the most popular. In fact, we identified only one program that requires a standardized test but does not accept the GRE. Despite the prevalence of the GRE, NASPAA-accredited MPA programs seem to be relatively open to accepting other standardized tests: 58% of the programs that require a standardized test for at least some students will accept another test. Figure 2 lists tests that are accepted in addition to the GRE. More than half of the programs that accept another score will accept the Graduate Management Admission Test (GMAT). The Miller Analogies Test (MAT) is accepted by another third of the programs that accept other scores, and a similar proportion accepts the LSAT. Although the GRE is the most common standardized test, students who took other standardized tests certainly can find MPA programs that will accept those tests in lieu of the GRE.
Standardized Tests Accepted, Other than the GRE
Recall that more than one third of MPA programs (i.e., 35%) allow standardized tests to be waived under some circumstances. Figure 3 presents the frequency with which programs accept various considerations for waivers. As shown in the figure, GPA requirements (paired with other considerations) are the most common line of demarcation between applicants who do and do not receive waivers,Footnote 2 followed by professional experience, graduate degrees, and various types of internal preferences.Footnote 3 Although GPA cutoffs are common, there is no consensus among MPA programs about which set of materials should be used to establish that a student is eligible for a waiver. Figure 3 demonstrates considerable differences across programs about the conditions that constitute a waiver; however, it understates those differences because there is no clear standard for the line of demarcation in what constitutes a “high-enough” GPA or “enough” years of experience to qualify for a waiver.
Qualifications for Standardized Test Waivers
Figure 4 presents the distribution of GPA cutoffs and years of experience that each program specifies. A 3.0 GPA and five years of experience are the most common, although there are programs that set the GPA threshold as low as 2.5 and as high as 3.5. Similarly, two years of work experience is sufficient to qualify a candidate for a standardized-test waiver in some programs, whereas others require as many as 10 years.
GPA and Professional Experience Cut-Offs for Standardized Test Waivers
The right-hand column in table 1 presents results of a multivariate model to determine whether diversity, prestige, or market forces explain the rigor of the GPA cutoff to qualify for a waiver. Although student diversity and program size are not correlated to the GPA cutoff, there is evidence to suggest that the more prestigious a program, the higher the GPA cutoff. The unstandardized regression coefficient indicates that 10 places higher in the US News & World Report ranking translates to an increase in the GPA cutoff of approximately 0.02.
Finally, if MPA programs are increasingly moving away from standardized testing in admissions decisions, what are they using to make decisions? There is surprising unanimity on this question. As figure 5 suggests, about 90% of programs require some form of personal statement. More than 85% in the sample require letters of recommendation (i.e., 50% of those require three letters and 33% require two letters). Résumés also are required by about 66% of all programs. Writing samples are relatively rare (i.e., only 8% require them for all students) as are other requirements (e.g., in-person interviews, ecclesiastical endorsements, and evidence of specific course-prerequisite completion). This list is strikingly similar to the admissions factors cataloged by Bowman (Reference Bowman1988). Despite a precipitous decline in the use of standardized testing in admissions decisions in MPA programs (i.e., from about two thirds of all programs in 1988 to one third in 2018Footnote 4), it does not appear that standardized tests have been replaced with new application materials.
Finally, if MPA programs are increasingly moving away from standardized testing in admissions decisions, what are they using to make decisions? There is surprising unanimity on this question. As figure 5 suggests, about 90% of programs require some form of personal statement.
Admission Requirement Other than Standardized Tests
CONCLUSION
Many graduate faculty and program directors have struggled with the efficacy of standardized testing. As a result, decision makers have been left with the dilemma of whether abandoning standardized testing would help the program attract a larger and more diverse student body or provide a signal of lower quality—perhaps dissuading the best and brightest students from enrolling in a program.
This study examined admissions requirements in MPA programs and found that they seem to be moving away from standardized testing. Bowman (Reference Bowman1988) reported that about two thirds of all MPA programs required a standardized test; in 2018, that number had decreased to less than one third. Whereas graduate faculty and program directors in the late 1980s could be legitimately concerned that eliminating the GRE would put them at odds with professional norms, that fear is no longer relevant. We also found evidence that the move away from standardized testing is not related to the prestige, size, or diversity of a program. Programs of all types are moving away from standardized testing.
Whereas the plurality of MPA programs do not require any student to submit a standardized test, the second-largest category opts for a waiver system in which students who meet certain criteria may waive the standardized-test requirement. We found that programs offer a variety of different criteria for waivers, although some combination of GPA and experience are the two most common. Apparently Gibson et al.’s (2007) advice to consider waivers as a viable alternative for programs that want to increase their share of in-service students has been adopted by numerous programs.
The implications of this investigation are clear: the standardized-test requirement is fading from the MPA landscape and programs that entirely eliminate the test (or provide a waiver option) do not risk losing accreditation or signal a lack of quality. In fact, maintaining a steadfast hold on the standardized test may make programs less competitive for many students—particularly in an environment in which online and hybrid programs have expanded the size of the graduate marketplace in many communities.
