Update: Quality indicators for Pre-K

This morning I had the opportunity to attend a demonstration of a new assessment tool for Pre-K developed by the AppleTree Institute for Education Innovation, hosted by the Fight for Children and the Early Childhood Academy Public Charter School. The new tool is called Quality Indicators and grew out of work done by AppleTree under a federal grant from the Early Reading First Program, so it's based largely around language and literacy, which researchers have shown to be key predictors of a child's success in school. AppleTree which was just selected as a recipient of an i3 development grant from the Department of Education to expand upon this work and pilot the tool in its 3 schools (more about that announcement here).

Quality indicators is a progress monitoring tool, rather than an end of year or semester assessment. It's meant to be used as a means of ongoing assessment throughout the year that can spark constructive discussion and development of a teacher's performance in the classroom.

I'm not a technical person or an educator, so I'll leave it to the folks at AppleTree to get more into the nitty gritty of the tool and how it compares to other assessment systems that are already out there. The best way I can think to describe it is a matrix. There are 20 Quality Indicators, including things like Student Engagement; Meaningful, Relevant Instruction; Language Instruction; Literacy Instruction; Writing Instruction; Time Management; Classroom Organization (you get the picture). Underneath each indicator are more specific measures (I guess you could call them sub-indicators). For instance, under Writing Instruction, you have Drawing Attention to Print; Explicit Instruction; and Incorporating Writing. Teachers are scored against this grid on a 6-point scale -- a 1 for Insufficient or Rarely and 6 for Consistently or Systematic demonstration of the ideal behavior. 

For the demonstration, members of the audience -- mainly teachers and center directors -- were presented with various descriptions of classroom scenarios and asked to determine which Quality Indicator was most relevant and later, to score the teacher against the matrix. I happened to be the only non-educator at my table, and let's just say my brain is matrix-challenged. With so many indicators, it was difficult to quickly identify which was most relevant, and the four of us often disagreed. After we went over the responses with the entire room, it quickly became clear that there would rarely, if ever, be one right answer. The presenters agreed that while there can be many right answers, some are more on the mark than others. (And I figure it would take more practice to grasp the best answer immediately.) 

My take is that Quality Indicators is a useful framework to guide meaningful discussions about what makes teachers of Pre-K effective in the classroom and a means to engage teachers in self-reflection and improvement. But if test scores represent the most objective measure we have to gauge teacher performance, this would be at the far other end of the spectrum -- something much more flexible and multidimensional, but also very subjective.

That said, it was quite amazing to me how the educators at my table quickly reached a consensus when it came to scoring a teacher in a short classroom video. Watching the video, I quickly formed the impression that I was observing a highly effective Pre-K teacher. She was very energetic and upbeat. She led the students in a participatory song (something like "If you're happy and you know it, clap your hands"), prompted them to guess why many of their classmates were missing that day, which led to a discussion of the weather (it was pouring) and then did a brief lesson about reading a calendar (it was the first day of February). The children seemed engaged and excited. When it was over, we were asked to rate the teacher on one of the indicators.

I quickly rated her a 6 out of 6. The rest of the table was silent, taking more time to be critical about her performance. Eventually, withdrawing my high score, the table settled on giving her a 2! They agreed that she missed many opportunities to engage in more explicit instruction, wasted time by taking attendance twice and tried to teach many concepts rather than focusing on getting children to master one. Across the room, the ratings were similarly low --often even lower than what the experts had come up with. 

I've always known that being a teacher is one of the most challenging jobs -- and also one of the most important. I've also always understood why teachers bristle at assessments that may not take every factor into consideration. But I guess what I took away from this morning's session is that teachers are probably also their own harshest critics. By involving multiple observers and multiple observations (45-minute sessions), Quality Indicators probably accounts for that. No matter how good we feel about ourselves, it's always tough to scrutinize yourself in the mirror -- especially if you are looking for flaws and are being videotaped! But without such tools for reflection, we are leaving our teachers to fly blind in the classroom. High-quality, multi-dimensional assessments that engage teachers like Quality Indicators can make a big difference in early learning in a way that is constructive for both teachers and students.