Fairtest's Alternative Assessment Model
FairTest
has worked with the Massachusetts Coalition for Authentic Reform in Education
to develop an authentic assessment and accountability program. We also have
collaborated with a coalition of organizations in Chicago to develop a similar
plan. We enclose descriptions of each plan. Below, we outline key elements
of these proposals and then suggest how the federal government could modify
the proposed ESEA accountability provisions to meet this model.
Components of the model.
1) Classroom-based information. Evidence of teaching and learning is retained
by each teacher showing assignments, student work, and the teacher's observations
of the learning processes, strengths and weaknesses of the students. Teachers
would be expected to use a variety of appropriate assessment methods and
tools, including observations, student classwork, homework, projects, essays,
tests, presentations in a variety of formats, and portfolios to summarize
and evaluate the evidence of learning. These are the basis for teacher reports,
qualitative and numerical, as to each student's progress in meeting state,
district and school standards and goals as well as goals the teacher or student
may have. The summary information about each student should be supported
by the classroom evidence. [This approach is found in the Learning Record
http://www.learningrecord.org and the Work Sampling System http://www.rebusinc.com.]
2) Limited standardized testing. Testing should be in literacy and numeracy
only and provide one means of checking on school level information. For accountability
purposes, sampling could be used. Marked discrepancies between test results
and school's classroom-based information should be investigated. No major
decisions about students or schools should be based on test scores alone.
3) School quality reviews. Independent, well-prepared teams periodically
conduct reviews of every school. Each school first prepares a self-evaluation.
Then the team visits, sitting in on classrooms, shadowing students, interviewing
students and educators and parents, reviewing random samples of student work,
considering academic and other aspects of school life. The team then prepares
a report with recommendations, which is given to the school and is available
to the public in summary and complete form. Schools respond to the review
and use the review in planning improvement.
Having the data does not mean schools are able to use it. Schools set up
systems to summarize this information. More importantly, they develop means
through which teachers look at each other's work and student work as a means
of learning how to improve curriculum and teaching.
Each school prepares an annual public report. The report summarizes information
from the assessments and evaluations, and then describes how the school uses
information to improve, changes in processes and results over the years,
and steps to be taken to improve processes and outcomes that need improvement.
It would use the variety of kinds of evidence, quantitative and qualitative,
to "paint a picture" of the school.
The information on learning outcomes would provide disaggregated data by
race, gender, income (poverty), English proficiency and disability. The data
would be reported when the sample sizes are large enough to be meaningful
and when individuals are not identifiable. School reports would have to discuss
the progress of the groups for whom data is disaggregated.
School reports would also include data on grade retention, suspensions, graduation/drop
out, and school climate (e.g., through surveys). Over time, schools would
gather information about their graduates, from how well elementary students
succeeded in high school to whether high school graduates attended college
and successfully graduated to employment or civic participation or satisfaction
data. The goal for school, after all, is not only academic success but life
success.
The plans call for intervention in schools which clearly demonstrate they
are not succeeding, according to the multiple measures, for all or for significant
portions of their students. The first step is an investigation. Indeed, the
Quality Review process should first include schools where current information
suggests low levels of performance. A review must include: factors beyond
a school's control, the resources a school has, what it does with its resources,
and how it might use resources better.
The investigation and support should lead to carefully targeted and planned
assistance. Where a school does not make progress even with assistance, stronger
interventions should take place. However, interventions such as school reconstitution
or state takeovers have been tried with at best limited success (Mary Levy
of Lawyers Committee did a study for DC Voice; most efforts at reconstitution
failed). Too little is known about how to make such strong interventions
succeed. Therefore, states should develop and implement the stronger actions
with caution and keep very close track of what does and does not work so
as to learn from them.
[The examples are the Massachusetts Coalition for Authentic Reform in Education
(CARE) Call for an Authentic Assessment System, on the web at http://www.fairtest.org/arn/masspage.html
in short and longer versions; and the New ERA plan from a coalition of Chicago
groups, on the web at http://www.pureparents.org./ERAplanAug00email.htm .]
Federal implications.
The conference committee on ESEA should rewrite the legislation to adopt
the provisions described here. That is, states would be required to implement
comprehensive assessment, accountability and assistance plans. Schools and
districts would develop plans based on the outline above. Two cautions:
- This will take substantial time to implement. The key is further education
of educators and the time and resources to do it. By the end of this authorization,
schools, districts and states should have the basic structures in place and
operating: quality reviews, school improvement plans and processes, and a
range of data emanating from classrooms that can be used to understand school
progress. Thus, in most cases interventions based on this approach are not
likely during this authorization.
- It will not be cheap. While all the elements of this plan exist somewhere,
we have not seen any summary research on likely costs. Nonetheless, based
on informal assessments of costs, we believe the cost is feasible. More importantly,
it would enable a means to develop accountability that will support development
of high-quality schools with strong outcomes, not just gains on test scores.
The largest costs will be professional development and paying teachers for
the time to engage in the sharing and reflective processes required for genuine
school improvement.
Some questions and answers about the plan:
Q: Shouldn't the current test-based accountability processes be used to identify
and act on schools?
A: Only very cautiously. States can use current data to determine which schools
should first undergo Quality Reviews, and it can use current data to identify
which schools most need additional assistance (though this is more mystification
than mystery: almost all schools serving low income children need additional
assistance). The final sanctions in the current ESEA versions should not
be employed in any event as they will cause more harm than good (as discussed
above). Disaggregated data should be compiled as soon as it can, and it should
be reported and used as part of making initial determinations for QR or assistance.
Q. Without numerical targets with firm sanctions, won't schools just ignore
the process?
A. This gets to the heart of the improvement issue, and thus deserves some
discussion.
First, most teachers currently working have substantial knowledge, work hard,
and care. They do not always have sufficient knowledge. We believe teachers
are mostly willing to address that issue, but need support and resources
to increase their skills. They do not need attacks or attempts to de-skill
them. Too often, current accountability is, or is perceived by educators
as, a means of attack and blame, while teaching to the test is de-skilling.
If it is true that teachers are less skilled or motivated than we believe,
we are in a quandary that current accountability attempts cannot solve. High-stakes
testing does not improve skills or increase motivation to teach well (as
distinguished from seeking short cuts for boosting test scores) or to learn
more. There is no reason to believe that in a high-stakes environment, high-quality
applicants will flood the teaching field, which already faces severe and
growing shortages. The nation has little choice but to work with current
teachers and to create positive inducements to attract new teachers. The
approach to school improvement we propose would be far more likely to accomplish
this goal than is high-stakes tests with numerical triggers. We would add
that teachers are particularly likely to leave schools which are likely to
fail to make adequate improvement on tests, which will only exacerbate the
problems.
In short, school improvement cannot succeed without or against teachers,
and it mostly needs more skilled teachers who actively participate in improvement
efforts. This cannot be well attained by threats or fear.
The second critical element is the target: is it a target of ensuring high
quality educational opportunities for all students, or will low-income and
minority-group students be consigned to test preparation for "basic skills"
in a process that never gets beyond such basics? Evidence clearly demonstrates
that most all students can in fact learn to rather high levels in most areas,
but only when provided rigorous high quality intellectual environment and
opportunity. Test-based reform fails to provide those opportunities, which
is why progress toward higher levels is often so slow in schools with low-achieving
students (see recent reports from Chicago Consortium on School Research)
and why parents who believe their children have good schools are increasingly
angry about tests even when most of the children pass them. Finally, the
goal of focusing only on tests seems to lead to schools which ignore too
many other academic and non-academic dimensions, which deprives children
of opportunities and supports they need and deserve.
The argument here is that real school improvement must proceed with teachers
and must have goals worth honoring. Current accountability procedures often
proceed by attacking or de-skilling teachers to focus on too-limited targets.
Thus, it probably won't succeed on its own narrow terms and surely cannot
succeed in providing a truly high-quality education for all students. Thus,
schools and teachers may pay attention – but they will pay attention to the
wrong things or to inadequate things.
The plans also provide for transparency: claims of success or progress must
be backed up with public and verifiable information. That it will be public
will be a spur to taking the process seriously even if one is not initially
motivated to do so. That it is verifiable is a spur to accuracy, rather than
to inflated claims. As it is based in student work, it will actually be far
more understandable to the public than are test scores, about which most
people understand very little. Samples of high-quality student work can be
used to help parents, students and teachers learn what such work looks like
and thus seek to attain similar levels of quality.
All that said, we also recognize that for a variety of reasons, even given
adequate resources, some schools may fail to do a good job and will not improve
despite publicizing the situation and providing support for improvement.
FairTest does not believe such schools should be left to continue to fail
their students, and the assessment and accountability plans call for stronger
measures if needed. That would occur based on a variety of evidence, not
simply test score gains that are of questionable validity and reliability.
Q: But won't this take forever?
A: There is no reason to believe that schools serving predominantly low-income
children can provide high quality opportunity for all unless they have a
major infusion of resources, and no reason to believe they can succeed unless
this country develops as strong a will to solve poverty as it now has to
develop tests. That, barring such changes, success is not likely for more
than a relatively few.
The goal of all students reaching a level of "proficiency" equivalent to
"proficient" on NAEP is not feasible in the 10-12 years called for in ESEA.
This is indeed unfortunate, but it will not get better by pretending the
impossible can be attained. That is, the goal proposed is itself not reachable
and thus will "take forever."
Schools and districts taking the approach we propose will, we conclude, be
more likely to make substantive, real progress on valuable, real-world outcomes,
than will schools focusing on just boosting test scores every year. The reason
is that it will tap into the energy, talent, and caring of educators and
students – the human factor that a test-based approach ignores and undermines.
That is not to say it will happen quickly, but that the goals will be superior
and reached as quickly as happens with test-based accountability. It will
happen faster if the nation decides it really means to leave no children
behind and takes serious measures, in education and elsewhere. We have proposed
one such serious measure. It should not be dismissed in favor of narrow and
illusory test-scores.
August 1, 2001