3 Data, Descriptive Statistics and College Quality
3.1 Data and Descriptive Statistics
The Massachusetts Department of Elementary and Secondary Education (DESE) provided the
data, which include demographic information, test scores, and Adams Scholarship status for all
Massachusetts public high school students expected to graduate from high school from 2004-2011.
We use first time 10th grade MCAS test scores. In both math and ELA, we observe scaled scores
that determine scholarship eligibility, as well as the raw scores on which those scaled scores are
based. We use two main analysis samples, high school graduates from the classes of 2005-06, for
whom we observe six-year college graduation rates, and high school graduates from the classes
of 2005-08, for whom we observe four-year college graduation rates.
12
College outcomes come from DESE’s merge of its data on high school graduates with the
National Student Clearinghouse (NSC) database, which covers 94% of undergraduates in Mas10The most recent cohorts are allowed to use the scholarship within six years of graduating high school, but such
cohorts are not included in our analysis.
11Scholarship users must also be a U.S. citizen or permanent resident of the U.S. and must have been a permanent
legal resident of Massachusetts for at least one year prior to entering college as a freshman.
12We limit the sample to high school graduates, as only graduates were ultimately eligible for the Adams scholarship.
Of those who receive the Adams scholarship letter in the fall of 12th grade, over 98% ultimately graduate from high
school. We find no evidence that receipt of this letter affected high school graduation rates, so this restriction does not
create selection bias.
7
sachusetts.13 We observe for each high school graduate every detailed college enrollment spell
through 2012 and graduation if it occurs.14 We add to this additional characteristics such as college
costs and quality measures from the U.S. Department of Education’s Integrated Postsecondary Education Data System (IPEDS) and the 2009 Barron’s rankings of colleges. We separate colleges into
Adams eligible institutions (U. Mass. campuses, state colleges and community colleges) and other
institutions, such as in-state private or out-of-state colleges.
Table 1 shows the mean characteristics of the two analysis samples.
Columns 1-3 contain the
classes of 2005-06 and columns 4-6 contain the classes of 2005-08. Column 1 contains the full
sample, column 2 limits the sample to students eligible for the Adams Scholarship, and column 3
limits the sample to those within a certain distance of the eligibility threshold, as will be described
below. Panel A shows that Adams eligible students are half as likely than the average high school
graduate to be low income, black or Hispanic, because these characteristics are all negatively associated with the test scores determining eligibility. Panel B shows that 25% of high school graduates
are eligible for the scholarship and that those eligible score about one standard deviation higher
on their MCAS exams than the average high school graduate.
Panel C shows that 79% of Adams eligible students enroll full-time in a four-year college by
the fall following their high school graduation, which we refer to as immediate enrollment. Of
these, one third (26%) enroll in in-state, public, four-year colleges (Adams colleges).
Panels D
and E show that only 54% graduate from a four-year college within four years of high school
graduation but that 71% have graduated by their sixth year. Statistics for the sample comprising
the classes of 2005-08 look quite similar. Comparison of the graduation statistics to the enrollment
statistics across college sectors in these samples suggest that Adams colleges have substantially
lower graduation rates than do the in-state private and out-of-state colleges.
13This figure comes from comparing NSC enrollment numbers to those contained in IPEDS. The remaining 6% come
largely from for-profit institutions and those whose highest degrees take less than two years to complete. Such institutions tend to enroll students with relatively low academic skill, so that the overall match rate for those eligible for the
Adams Scholarship is likely even higher than 94%. Coverage is slightly lower in the Northeast as a whole, with 90% of
undergraduate enrollment covered by the NSC. Again, excluded colleges are mainly technical institutions.
14We exclude part-time enrollment spells and those less than 60 days long, though this has little effect on our results.
8
3.2 College Quality
Figure 1 confirms this difference between college sectors, plotting by initial enrollment sector the
fraction of students graduating within a certain number of years. We generate these figures using
NSC’s data on four-year college enrollers from Massachusetts’ high school class of 2004, prior to
the existence of the Adams Scholarship. About 40% of those who enroll in U. Mass. campuses
graduate within four years. The comparable figure for Massachusetts state colleges is well under
30%. For in-state private colleges and out-of-state colleges, that figure is about 60%. A large
fraction of students in in-state public colleges use a fifth or even a sixth year to graduate. Even
so, six (and even seven) years out of high school there exist large gaps in the graduation rates
between these sectors. This evidence makes clear that Massachusetts’ public four-year colleges
have substantially longer times to degree completion and lower ultimate completion rates than
the alternative colleges available to Massachusetts students.
To explore why these sectors differ so dramatically in their on-time completion rates, Table 2
provides a more detailed description of the college market facing Massachusetts students. Quality and cost measures reported by IPEDS in the fall of 2004 are weighted by enrollment of Massachusetts students and thus represent the average student’s experience of that sector. In panel A,
IPEDS’ measure of four-year completion rates tells a very similar story to NSC’s measure, namely
that U. Mass. campuses and state colleges have far lower on-time graduation rates than do nonAdams colleges.15 Some part of this variation may be due to the academic skill of incoming
students. Students enrolling in state colleges have much lower SAT scores than those enrolling
in other sectors, although the U. Mass. campuses look fairly similar to non-Adams colleges in
this regard. Non-Adams colleges also spend an annual average of nearly $15,000 per student
on instruction, nearly twice the spending of U. Mass. campuses and more than three times the
spending of state colleges. This resource gap may reduce students’ access to coursework or to
academic support necessary to complete such coursework and may thus help explain some of the
completion rate gap. Relative to their competitors, Massachusetts’ public colleges thus have sub15Note that IPEDS measures the completion rate of all undergraduates in these institutions, whereas Figure 1 measures the completion rate only of students coming from Massachusetts public high schools.
9
stantially lower graduation rates, attract students of somewhat lower academic achievement and
spend much less money on instruction.
Whether differences in graduation rates between these sectors are due to differences in incoming student achievement, resources available for instruction or other factors is beyond the scope
of the paper.
We follow Black and Smith (2006), who argue that because each such of these variables measures college quality with error, relationships between them and outcomes of interest
will be biased toward zero. We adopt their suggestion to measure college quality by combining
information from multiple variables in order to reduce such measurement error. Specifically, we
construct college quality from our student-level data as the first component from a principal component analysis of each college’s four-year graduation rate, SAT math 75th percentile of incoming
freshmen, and instructional expenditures per student, all of which are measured by IPEDS as of
2004, prior to the Adams Scholarship. We think of the first variable as capturing the ultimate outcome of interest, the second as capturing a measure of student quality and the third as capturing a
measure of available resources.16 The first principal component from this analysis captures 64% of
the variation between these three variables and nearly equally weights all three. We standardize
this quality measure to have mean zero and standard deviation one.
The final row of panel A shows that, by this measure of college quality, U. Mass. campuses and
state colleges are 0.32 and 0.94 standard deviations lower than the average quality college attended
by Massachusetts high school graduates. Non-Adams colleges are 0.29 standard deviations higher
in quality. It is important to note here that this measure of quality is not necessarily a measure of
how effectively the various college sectors are using their available resources. Though the Adams
colleges have lower graduation rates and instructional expenditures, these facts may be explained
in part by the fact that those colleges have much less funding per student. Panel B shows that
the total cost of U. Mass. campuses and state colleges, including fees, room, board and books, are
$15,000 and $11,000 respectively.
This is about half of the $29,000 sticker cost of their competitors.17
16Black and Smith construct their quality measure using a slightly broader set of variables. We find that all of these
quality measures are so highly correlated that it makes little difference whether we include more than three of them.
17In-state community colleges, at which the scholarship could also be used, are essentially open admissions campuses. In fall 2004, they charged on average $831 in tuition, $2,073 in fees, and $5,797 in other expenses, so that their
sticker and net prices were roughly two-thirds those of state colleges.
10
When grant aid is taken into account, U. Mass.
campuses charge their students an average of
$8,000 a year, relative to the $15,000 charged by their competitors. Students, particularly those
facing credit constraints, may thus make a seemingly rational decision to forgo college quality in
order to attend a lower-cost public option.18
Table A.1 provides specific examples of four-year colleges commonly attended by the Massachusetts high school class of 2004. In 2004, U. Mass. Amherst, the college most commonly
attended by Adams Scholarship recipients, had a four-year graduation rate of 43% and almost
perfectly average overall quality. The other Adams colleges had substantially lower graduation
rates and overall quality. Non-Adams colleges similar in graduation rate and quality to U. Mass.
Amherst include Johnson & Wales University and Merrimack College in the private sector and
the University of Connecticut, the University of Vermont and the University of New Hampshire
in the out-of-state public sector. Elite private colleges which also enroll relatively large numbers
of Massachusetts students, such as Boston University, Tufts University and Harvard University,
have four-year graduation rates 50-100% higher than U. Mass. Amherst, perhaps because they
attract more academically skilled students or because they spend three or more times the amount
of money on student instruction