Quarterly measurements invite skewed and misleading data.

Example Goal

Imagine the following goal is being proposed for your student:

Given content-area vocabulary (English, History, Science, Math), STUDENT will earn 85% accuracy on 3 out of 4 vocabulary assessments per quarter.

The Problem with Quarterly

In the case of vocabulary, students are provided content-area vocabulary on a daily basis.

Depending on the type of assessment (more on this below), the student could be assessed on vocabulary once-to-a-few times a week during each grading quarter. Which assessments count toward the goal? The first four consecutively administered assessments? The last four consecutively administered assessments? Four randomly chosen assessments administered throughout the grading quarter? Four cherry-picked assessments administered throughout the grading quarter?

A student could, conceivably, achieve at least 85% accuracy on three assessments throughout the quarter, yet achieve below 85% on ten other assessments administered during the same period. Since the goal specifies three assessments, but isn't specific on the type of assessment, the format of the assessment, and if the assessments must be consecutive, the data collected could be cherry picked to meet the goal. The data collected would portray a student who has met the goal. Yet, an examination of all data available would portray a struggling student.

Depending on the type of goal, the measurement could be weekly, bi-weekly, monthly, and so on. There is nothing written in stone stating the measurement must be quarterly.

In addition, if there is a problem, waiting until the end of the quarter is another quarter wasted. If the student is measured at more frequent intervals, and the student isn't exhibiting an iota of progress, then the goal and/or approaches and services being provided to support achievement of the goal could be adjusted more frequently. If something isn't working, why wait a quarter to change course?

Determining the measurement period is dependent on focus, format, and type of assessment, too.

More Problems with the Vocabulary Goal

In addition to quarterly measurement, there are few other problems with the example of the vocabulary goal.

What is an Assessment?

Depending on the school and/or teacher, "assessment" might be defined as informal check-ins by teachers, formal end-of-quarter tests, and/or quizzes. For example, one teacher I knew didn't consider quizzes to be assessments, so she denied a student his testing-to-completion, small-group, and testing-in-a-quiet-environment accommodations for the weekly quizzes. On another occasion, she refused the student his testing-to-completion accommodation because she said the assessment was a take-home assessment and she'd given all students time and a half.

Does it count if it isn't graded? What if the assessment is an ungraded quiz, solely for the teacher to gauge whether students are grasping the concepts or need more instruction before closing out that learning unit and administering a graded evaluation?

Bottom line: You must define "assessment". Is it a quiz, an informal assessment by a teacher administered just to your student, a large exam, graded, ungraded? What is it?

(Additional Reading: "The Language of IEPs & 504s: Just When You Thought You Knew the Definition of "Assessment", Wrong Again")

What Format?

With the vocabulary goal, are the assessments being used fill-in-the-blank, use-it-in-a-sentence, write-the-definition, match-the-word-and-definition, isolated, and or in-context assessments?

If it is a fill-in-the-blank assessment, is the student provided a word bank from which he can chose words for the blanks or does the student have to remember the words that fit the blank spaces and write them in?

Is the student provided a list of words in isolation and asked to write a definition of each word?

Is the student provided an assessment that has a list of vocabulary words in one column and a list of definitions in a second column, and then asked to draw a line from each word to its correct definition?

Is it multiple choice? Is the student provided a word in isolation and asked to pick the definition from a list of three definitions underneath the word? Is the student provided a sentence, with the vocabulary word underlined and asked to pick the definition of the underlined word from a list of three definitions underneath the sentence?

Is the student provided a word and required to provide a definition out loud to the teacher?

A student who struggles with processing speed might achieve low assessment scores because he is unable to recall it during a test, however he can use the vocabulary word correctly during conversations in class—or he might have poor scores because he didn't have time to finish the assessment.

Another student who struggles with reading and/or writing might have poor assessment scores that are more a reflection of his reading and/or writing struggles than of his vocabulary comprehension. If asked to provide the definition out loud to the teacher, can he provide it, even though he couldn't provide it in writing?

Some students can identify the meaning of a word because the words surrounding it in a sentence and/or full paragraph lead the student to understand it in context. Yet when the word is in isolation and/or if the student is asked to provide a definition of the word understood in context, the student isn't able to provide a definition. In the case of in context, the student might simply have an understanding of what the work likely means. However, if asked to provide a sentence with the vocabulary word used within the sentence, the student might use it incorrectly.

The format of the assessment can provide a portrait of a student who has achieved the goal, and/or who is good at process of elimination, and/or who needs more time, and/or a slew of other things.

Ask yourself exactly what is being measured and if the student is really being assessed on mastery of vocabulary, ability to show mastery within a specific period of time, and/or compensation skills, and so on.

What Focus?

While vocabulary is used daily throughout all courses, the focus of other goals might not be able to be assessed as frequently—or possibly could be assessed even more frequently.

For example, a goal related to early writing and coordination might measure a student's ability to hold a pencil and draw three circles in a row. Why wait until the end of the quarter to identify whether the student achieved this goal? If the student is able to accomplish this goal in four weeks, why not adjust the goal to challenge the student and move him toward being able to consistently write letters or a more complex shape? Writing occurs throughout the day, so there are more opportunities to work with the student and to assess achievement and/or continued need.

Another example can be found in a math goal. Math doesn't pop up as often as vocabulary throughout the day. While it is used heavily in science in later grades and appears in stories, and other formats during the school week, math class (and science in later grades) is one of the only places to assess it. Hence, there might not be as many assessments to use for data. In this case, the assessments used for data might be the last four administered, under the assumption that the difficulty level increased throughout the quarter, so assessing progress at the end might make more sense than at the beginning. Another option might be to include all assessment data and determine if, in total, the student is trending up, the same, or down. Math builds upon itself, so if a student is doing worse at the end of the quarter and/or trending down, there's a gap in need of filling. If it isn't filled, the lessons built upon the missed material will fall into the gap, too.

"And" & "Or"

The vocabulary goal example includes four content areas in parenthesis. However, neither the words "and" nor "or" appear in the parenthesis. Hence, is the content-area vocabulary being assessed in all four of those areas? One or two? What about electives and other courses? Vocabulary across the board or just in one or a few select classes?

Back to Quarterly and Vocabulary

The example goal above could be rewritten as the following:

Student will earn 85% accuracy on 3 out of 4 consecutive assessments monthly in each of the student's core classes (English, History, Science, and Math). The format of the assessments will be words in isolation. The student will be required to write a definition of the word.

There are numerous ways it could be written. Bottom line: Determine what is being assessed—knowledge, speed, ability to read or write, comprehension skills, compensatory skills, etc.—and then write a goal that measures exactly that.