[Reading-hall-of-fame] negative 'gain'

[Yetta Goodman]

Jay --

I want to encourage you to write up your proposal for a forum. Post it 
on this list serve to get the responses of others.  In any case you 
could present it
in May at the annual RHF meeting.  I think there would be an audience 
for this kind of meeting and a number of RHfamers would like to participate.
If you are not going to be at the breakfast, we can present a written 
proposal from you and let you know how the group responds to the idea.
I think the idea has good merit.





Jay Samuels wrote:

> Colin and David,
>
>Thanks for your comments. Yes, at times I think that doing research is like
>painting by numbers.
>
> This morning I read Ken Goodman's idea that we as a group endorse or
>criticize some aspect of education. What I found interesting about ken's
>idea was that at that moment I was thinking about writing to the Dean of
>Education at Minnesota about the No Child Left Behind legislation. The
>legislation seeks to bring almost every student to some level of academic
>achievement. Now that Bush believes he has a mandate, he is pushing to have
>the law extended to include high school testing.  My purpose in wanting to
>contact the Dean at the College of Education at the University of Minnesota
>was to ask what he thought about the idea of having a forum on the
>legislation. Given what we know about factors that affect academic
>achievement, is the legislation psychologically sound? Can we raise
>achievement for all groups, and, if so, what will it take? Philosophical and
>political goals are not always psychologically sound or feasible.  Perhaps,
>the Reading Hall of Fame sponsor such a forum. Then, after we have come to
>and arrived at an educated opinion, if our position runs counter to what the
>No Child Left Behind desires, we can voice our opinion as a group. My time
>table is about a year from now. By then we will have heard a variety of
>opinions on the Bush education mandate. There are think tanks such as the
>Manhattan Institute for Policy Research that have published papers by highly
>educated researchers who are highly critical of our schools. So, not every
>one is against the Bush legislation. I am reasonably certain that if we want
>to conduct such a forum the topic is important enough that IRA could work
>with us to find a prominent spot at the convention in San Antonio, TX. TX,
>what a suitable place for such a forum. Jay samuels
>-----Original Message-----
>From: reading-hall-of-fame-admin at nottingham.ac.uk
>[mail:reading-hall-of-fame-admin at nottingham.ac.uk] On Behalf Of Colin
>Harrison
>Sent: Sunday, November 07, 2004 6:00 PM
>To: reading-hall-of-fame at nottingham.ac.uk
>Subject: [Reading-hall-of-fame] negative 'gain'
>
>Hi everyone
>
>I wish to make two points: first is to remind ourselves that there is simply
>loads of 'error' in reading tests, and this alone means that negative 'gain'
>in post-tests will be common; and second, that we should not get hung up on
>it, since the stats we commonly use will tell us whether or not we have a
>stable picture emerging from our data.
>
>I led an evaluation a few years back in which we tested using a standardised
>two-form reading test the reading skills of 700 young people aged 11-14 who
>were in 7 inner-city schools. The participants received one of two parallel
>versions of the test in the October and took the other version as  the
>post-test eight months later, in the June, following 8 months of exposure to
>a mixed programme of reading development activities.
>
>After we scored  the post-test, I was astonished at how many of these
>inner-city kids had post-test scores that were lower than their
>pre-intervention scores. On visual inspection of the results, the data
>looked a nightmare to interpret. The scores were all over the place.
>
>But guess what?  Sure enough, when we did the stats, in the two schools that
>had teachers who had really got into their staff-development program (and
>crucially, where reading development at system level was not in its first
>year), there were statistically significant differences favouring the kids
>from those two schools. Yep, although the scores in all seven schools
>included plenty of 'negative gains', the pooled results were indeed stable
>enough for significant differences to show through clearly. 
>
>Moral- don't expect statistical tests that are meant to be based on grouped
>data to do the job at the individual level; poor readers do not all tend to
>make massive leaps of progress. Contextual factors, local testing factors,
>test administration factors, text factors that reflect the effects of
>motivation, background knowledge ect, will all impact the data, and produce
>error at the individual level, which means that an individual's 'true' level
>is within an error band whose width may be greater than the grade-level
>improvement delivered by the programme.
>
>We're painting by numbers here, it ain't a Durer oil.
>
>Colin
>
>
>Best wishes
>
>Colin
>
>---------------------------------------------------
>
>Colin Harrison                    * Colin's office:
>(+44)-115-951-4441
>School of Education          * Research Office:  (+44)-115-951-4990
>Jubilee Campus                 * Colin's assistant    (+44) 115-951-4518
>University of Nottingham    * Colin's mobile:      (+44)-788-777-5154
>Nottingham  NG8 1BB, UK * Research  Fax    (+44) 115 846 6188
>
>
>
>_______________________________________________
>Reading-hall-of-fame mailing list
>Reading-hall-of-fame at nottingham.ac.uk
>http://lists.nottingham.ac.uk/mailman/listinfo/reading-hall-of-fame
>
>
>_______________________________________________
>Reading-hall-of-fame mailing list
>Reading-hall-of-fame at nottingham.ac.uk
>http://lists.nottingham.ac.uk/mailman/listinfo/reading-hall-of-fame
>
>
>  
>