The following charts on TESTING were presented at the November 2000 meeting by

Mr. Ken Wesson

and he has agreed to posting this replica of his informative presentation on our web site

The BAAPC wishes to thank Mr. Wesson for his approval to post this informative package

Are We Placing Too Much Faith
in Standardized Tests?

Kenneth A. Wesson
Office of the Chancellor
San Jose/Evergreen Community College District
(408) 223-6728
(408) 531-8827 (fax)
(408) 323.1497 (home)
[email protected]

What Standardized Tests Are Designed to Do

1. Spread students along the performance continuum beneficial to whom?)

2. Provide us with a wide range of student scores. Variations in scores are vitally important Standardized tests must produce consistent indicators that can be interpreted as achievement, intelligence, or ability differences, otherwise students cannot be assigned a rank.

3.The more a test (and the contributions from each test item) spreads out test-takers, the more valuable that test is in differentiating students, as long as those differences produce acceptable outcomes.

Standardized Tests

o Reward students who fit a particular narrow profile like that which was found in our schools of two generations ago (when students came from similar neighborhoods, and had many common family, language, and life experiences).

o Reveal the kinds of enriched opportunities that a child has been exposed to in the past more than what that particular student is capable of learning in the future.

What Standardized Tests Were NOT Intended to Do

• They were
1. never intended to be tests that measure "educational quality".
2. never purported to be gauges of teaching excellence.
3. never intended to be "accountability" yardsticks, or measurements of educational accountability’ ‘merit" or reflect "high standards"

• They are grossly inappropriate as evidence-seeking instruments for evaluating "quality" in any school.

• Other tests can be effective diagnostic tools identifying those areas where future teaching or learning may be of tremendous value However, standardized tests are not used for this purposes.

Questioning Standardized Test Content

1. Test items that are impervious to high quality classroom instruction are the items most likely to remain on standardized tests.

2. Only items that show evidence of helping to distribute youngsters across the performance spectrum are allowed to remain, (important content and skills mastered by most kids at a given grade level will invariably be replaced by items that some students get right and many others get wrong).

Questioning Standardized Test Content

3. Test developers know that scores are likely to reflect just as much of what a child has learned outside of school as he or she has learned in school, rendering test items that fall into the former group far more valuable to assist with the "spread" or range of scores during test construction.

4. The emphasis is on dissecting information rather than constructing subject matter content and ideas.

5. In timed tests, speed is premium factor. Thoughtful consideration of any question is an unwise luxury. Thinking is penalized.

Tests of Differentiation

IQ test questions actually penalized demonstrations of intelligence such as:

Emperor is the name of:
(a) a string quartet
(b) a piano concerto
(c) a violin sonata
While a mediocre student might confidently select (or guess) b is the correct answer, a more knowledgeable youngster night recall that not only did Beethoven write the Emperor Concerto, but that Haydn also composed the Emperor Quartet. Thus, a more intelligent student is penalized for knowing too much about the subject matter In addition to likely making an incorrect response, that "brighter" student more minutes pondering the merits of both responses. Since these are timed tests his cumulative score is jeopardized.

Percentile and Percent Correct

Publicly reported scores produce the harshest/most unwarranted criticisms of our pubic schools.
Few people are aware of the differences between "percentile" and "percent correct"

my fourth grade classroom taught skills in mathematics
they team mathematical concepts that are developmentally appropriate
my classroom average is at the 51 percentile indicating that my students have mastered the concepts in math that we have deemed appropriate for 9-year olds.
If they are performlng at the sixth grade level (91 percentile) - Indicates understanding of mathematical concepts and skills that certainty have not taught them.
If my students high level at mathematics mastery Is due to experiences outside at my classroom I cannot, in good conscience, take the credit for these high levels at achievement. Yet some schools do, if they are strategically located In the "right" neighborhoods.

What about these important traits and talents
that should be cultivated in every school?

perserverence	intution	adaptability
responsibility	sensitivity	healthy self-confidence
empathy	self-control	honesty
trustworthiness	motivation	communication skill
open-mindedness	generosity	creativity
originality	cooperation	kindness
commitment	loyalty	friendliness
ingenuity	insight	emotional maturity
inventiveness	compassion	flexibility
enthusiasm	integrity	ambition
initiative	collaboration	motivates others
accepts new ideas	imagination	accommodates diversity
listens objectively	adjust to change	learns from errors
thinks beyond the obvious

Factors Influencing Standardized Tests

Teacher Training

Gender

Teacher's credentials

Family income

Parental involvement

Nutrition

Enriched environment

Stimulating home environment

Subject-matter teacher training

Parents' educational attainment

Test-taking practice

Degree of familiarity with standard English>

Public vs Private school aggregate scores (student selection)

Access to earlier preparation

The API (Academic Performance Index -- more accurately Affluence Poverty Index -- extremely high and predictable correlation between test scores and socio-economic postion)

When living in a specific kind of evironment influences test scores outcomes, why pretend that we are exclusively measuring "school quality"?

Questioning Standardized Test Content

1. More honestly reflections of economic advantages and disadvantages that mirror American society. We can predict general test "results' with fairly high degrees of accuracy.

2. Before the first bubble IS filled in, we know by historical data and zip code how certain schools will stack up in the standardized testing process

Language in Standardized Tests

There is a distinct language advantage, if one comes from a home where he (1) regularly hears his parents use language and express ideas in a fashion consistent with (2) the language used during classroom instruction (3) found in textbooks, and (4) later encountered standardized tests. ESL children who are heavily dependent on their language are among the predictable "losers" on the language portion of typical standardized tests.

Limited exposure to the English language - test scores indicative of their lack of English language mastery rather than concept understanding. Few years in language acquisition (of English) further handicaps a young test-taker for any test printed in the English language

Tests Penalize Teacher Effectiveness

All 3rd grade teachers effectively teach a particular mathematical notion a All students give the correct response a We'd eliminate the test item (it does not promote respondent distributions)

Highly effective teaching a 98% correct answers a that test item gets removed (no variation in scores). 98% correct = effective instruction. Their efforts are subject to back-firing if too many students indicate that they have been taught too well!

This process drives important content standards and student achievement in directions that are precisely the opposite of that test

Penalizing Teacher Effectiveness

Conversely, 98-100% incorrect responses à test item would have little value to the test-makers.
If 50% students get an item incorrect à distributing students along "a performance continuum".
University of Michigan: 20-50% correlation between classroom instruction and test content (or a 50-80% mismatch between taught/tested in the schools.

This gap actually helped produce the variations in scores, so the test’s usage was enthusiastically endorsed.

Test Development: SAT

These tests received wide acceptance as general "predictors of college success" although that was never their intentions.

Test results over-predict male college success in general and under-predicted future college performance for girls Although boys commonly received higher SAT test scores, girls consistently earned higher grade point averages in both high school and later in college than their male counterparts who had received higher SAT test scores. Any educator or parent of girls should find this fact quite disturbing.

Test Development: The SAT

The predictive value was based on its correlation with success during the freshman year of college only, not one's entire college career.

Instruction in the expensive, elite, New England-based, college. preparatory, male-only schools matched the expensive SAT prep courses and the first-year course work in the correspondingly expensive Ivy League colleges (Wealth and SES class were dominating factors)

The Ivy League freshman year was the logical "next step" from the college prep schools -- the very purpose of paying for a college preparatory education, The Ivy League schools were precisely the colleges to which the "college preparation" for these boys had been directed.

Hosted by www.Geocities.ws