Value Added Modeling: Administators v. Mathmaticians

@georgeob1,

Here's a list of schools that don't require SATs: http://fairtest.org/university/optional/state. The list grows every year. Many schools don't view the SAT as a valid measurement or predictor of student achievement.

Maybe the LSAT is on the way out too.

As to IQ tests, nobody said it better than Alfred Binet:

Quote:

According to Binet, the scale was designed with a single purpose in mind; it was to serve as a guide to identify children in the schools who required special education. Its intention was not to be used as “a general device for ranking all pupils according to mental worth.” Binet also noted that “the scale, properly speaking, does not permit the measure of intelligence, because intellectual qualities are not superposable, and therefore cannot be measured as linear surfaces are measured.”
Since, according to Binet, intelligence could not be described as a single score, the use of the Intelligence Quotient (IQ) (a notion coined by Terman in America in 1916) as a definite statement of a child's intellectual capability would be a serious mistake. In addition, Binet feared that IQ measurement would be used to condemn a child to a permanent “condition” of stupidity, thereby negatively affecting his or her education and livelihood:
Some recent thinkers…[have affirmed] that an individual's intelligence is a fixed quantity, a quantity that cannot be increased. We must protest and react against this brutal pessimism; we must try to demonstrate that it is founded on nothing. (Binet)

It sounds very similar to what the mathematicians say about VAM (BTW - he cites all the original papers that he drew his information from, and includes links to all of them, if you're interested in tracking down whether they're shills for the union)

I suppose I'll have to look for the WSJ article because I think that is simply isn't true that a teacher can't be fired based on their performance.

0 Replies

@georgeob1,

Quote:

A recent Op ed in the WSJ by the former head of the New York city schools

You mean, the lady who had no qualifications for the position and no clue what she was doing? And was basically fired in disgrace herself? That former head? Not exactly swayed by her testimony.

Re: the standardized testing, it's funny that you bring up the LSAT; I just got out of a meeting, in which the prime topic was the school considering putting less emphasis on high LSAT scores due to a lack of evidence that this correlates with achievement either in school or afterward.

Cycloptichorn

1 Reply

@boomerang,

Quote:

But.... but.... but..... God is magic so how can math be god if math isn't magic?

Math = God and Magic = God, therefore Math = Magic.

There's a huge push to tie student scores in standardized testing to teacher evaluations. Our super told us (no I haven't checked his numbers) that only around 17% of our teaching staff can be directly assessed by our state tests (if that's even possible).

No teachers in pre-k through grade 2 give MCAS tests.
No social studies teacher does.
Only two (I think) grades of science have them.
No gym, art, health or special ed teachers, no speech/language or reading specialists.

So, this would be an assessment on ELA and Math teachers for the most part.

physical/occupational therapists, school psychologists, staff do......

1 Reply

@sozobe,

Quote:

9. Any patient who entered a doctor's care when already terminally ill would be expected to make a full recovery -- or the doctor would be considered incompetent.

Love this one! Of course, no teacher would ever call a child incapable of learning anything (which I'd equate to terminal in this bit of fun).

0 Replies

@georgeob1,

Quote:

Despite these distorting effects these standardized tests have proven themselves to correlate very strongly (not perfectly, but very strongly) with subsequent academic performance and success in various professional education programs and the professions themselves.

From my anecdotal experience I do not believe this to be true - do you have some resources to support your view?

2 Replies

@littlek,

littlek wrote:

Quote:
Despite these distorting effects these standardized tests have proven themselves to correlate very strongly (not perfectly, but very strongly) with subsequent academic performance and success in various professional education programs and the professions themselves.

From my anecdotal experience I do not believe this to be true - do you have some resources to support your view?

Like, linking to a source? Don't hold your breath!

Cycloptichorn

0 Replies

@Cycloptichorn,

Cycloptichorn wrote:

Quote:
A recent Op ed in the WSJ by the former head of the New York city schools

You mean, the lady who had no qualifications for the position and no clue what she was doing? And was basically fired in disgrace herself? That former head? Not exactly swayed by her testimony.

No I was referring to the recently retired Administrator of the new York City Public schools system appointed by Mayor Bloomberg after he got authority from the state to take control of a seriously failing school district.

Tests of all kinds, ranging from essay exams, to so called standardized tests, to extended oral examinations for PhD candidates, and even check flights for candidates for qualification as military aviators, are subject to numerous irregularities resulting from largely extraneous values ranging from how one feels that day to varying "test taking skills" among students. Despite that they are an almost universally used way of discriminating bertween the highly proficient and those who are seriously deficient. In general we have not found an effective substitute for them. The Marine corps very carefully tracks the success of recruits in subsequent trasining to measure the effectiveness of their drill seargents, as does the Navy with flight instructors and instructors in their Nuclear Power training program. In my company we track the performance of our project managers, professional geologists, engineers and scientists in carrying our successful projects for our clients. Promotions, bonus payments and the occasional termination depend on the results. All of these measurment systems are prone to individual errors, but they do provide an objective measurement that in almost all areas of productive human activity has proven to be both useful and necessary for sustained excellence in the activity involved.

Apparently none of this applies to those who teach our children how to read, write and perform basic arithmetic functions in our generally failing public schools. Remarkable.

3 Replies

@georgeob1,

Except that in evaluating teachers, we are not just looking for whether the students are highly proficient or seriously deficient.

An excellent teacher may still have seriously deficient students. (Those with disabilities, for example.)

A terrible teacher may still have highly proficient students. (In a district with highly involved parents and low poverty levels, for example.)

1 Reply

@sozobe,

All true, but these are all easily resolved statistical matters that numerous applications routinely resolve in applications ranging from medicine to the examples I cited above. Certainly they do not constitute a reason for abandoning all measures of the output of teaching professionals - which is what the AFT demands, in addition to monopoly control of employment in the schools.

We would all like to find a secure position in which we are utterly free of accountability for the quality of our work or the effort we make (or don't make). Most of us don't ever find it. The fact that the goods and services we rely on are usually of acceptable quality is a result of that factor and the ability of consumers to chose an alternative. Interestingly the AFT wants no accountability for its members, lifetime tenure, monopolistic control of employment, and no charter schools i.e. no consumer alternatives in public education. It is certainly a sweet deal for the unions and its members - it is the parents, the general public and eventually the nation (in a still competitive world) that gets screwed. One must be either a beneficiary of this scam or an amazingly credulous fool to buy their propaganda.

1 Reply

@georgeob1,

No, they're not easily resolved. That's the very center of this problem. People keep trying and failing to find accurate measures. Test scores have obvious problems that are not easily resolved statistically or mathematically. I'm here for a second, can find it back later, but there was a promising initiative to evaluate videos of teachers that ended up not being so promising. (Especially, it turned out to be too hard to find qualified evaluators.) This happens over and over and over again.

I come at this from the teacher perspective (not only, but mostly), and again I would LOVE for there to be accountability. I really would like for the best teachers to get the most money and prestige. It's been very frustrating to me as I see how difficult that is.

1 Reply

@littlek,

Quote:

There's a huge push to tie student scores in standardized testing to teacher evaluations. Our super told us (no I haven't checked his numbers) that only around 17% of our teaching staff can be directly assessed by our state tests (if that's even possible).

No teachers in pre-k through grade 2 give MCAS tests.
No social studies teacher does.
Only two (I think) grades of science have them.
No gym, art, health or special ed teachers, no speech/language or reading specialists.

So, this would be an assessment on ELA and Math teachers for the most part.

This is very interesting. Thanks for posting it.

I'm going to chew on this a bit.....

0 Replies

@littlek,

Quote:

From my anecdotal experience I do not believe this to be true - do you have some resources to support your view?

This is an excellent question.

0 Replies

@georgeob1,

So you track the performance of adults to determine the fate of other adults?

Grown up people who are responsible for their own lives are quite different from children whose lives are under the control of their parents. Some parents are awful and some parents are wonderful (and some wonderful parents are too poor to make sure the kids are fed every day) and this has an impact on how a child does on tests.

And teachers aren't promoted the same way people are in other professions. A second grade teacher doesn't get promoted to the third grade based on having done a good job. There's no "if you do a good job here you'll get promoted to the 4th grade in two years" carrot. Does this mean they should be paid the same amount year after year after year with no job security to speak of?

0 Replies

@sozobe,

sozobe wrote:

No, they're not easily resolved. That's the very center of this problem. People keep trying and failing to find accurate measures. Test scores have obvious problems that are not easily resolved statistically or mathematically. I'm here for a second, can find it back later, but there was a promising initiative to evaluate videos of teachers that ended up not being so promising. (Especially, it turned out to be too hard to find qualified evaluators.) This happens over and over and over again.

Nonsense.Relatively simple ststistical techniques for stratifying comparable data, separating the effects of other variables and the like have been in steady use for years in analyzing multi variable data ranging from consumer preferences to the predicted reliability of complex systems (aircraft, powerplants & automobiles), the analysis of public health data to associate cause & effect in disease, the effectiveness of medical treatments and many, many other common aspects of our lives.

None of these measures are perfectly accurate - anymore than are the single event measures, such as pass/fail decisions on PhD exams, the State examinations required to license professionsl engineers, bar exams for lawyers or anything else. However, they are indispensable for the protectionof the public interest in terms of quality services and products.

Certainly the alternative to them - no performance measurement and action based on it at all - , is not adeemed acceptable in almost any other area of commerce, government operation or professional service. For example, increasing beneficial use is made by both government and private organizations of the therapy choices and the results achieved by individual doctors. Our tort laws hold individual engineers (and their employers) accountable for the individual structures or systems they design; same goes for medical professionals. even scum bag lawyers can sometimes be held accountable for their work (sadly not often).

My experience in a fairly broad range of activities has consistently shown me that performance measurement and action based on it is an indispensable prerequisite for sustained excellence in almost any organized activity. More to the point, however, is the observation that a complete absence of it is a sure way to achieve permanent mediocrity and the easy manipulation of the system by the worst 2% in a process that insidiously corrupts the values of the entire organization. And that , sadly, is a fairly accurate description of the public shhool systems in most areas of this country.

A few decades ago there was an explosion of interest on American college campuses in doing away with the perceived inequities of student testing and all the blather about teaching for and studying for the test, etc. - all supposed to divert the eager professors and students from the higher intellectual aspiratione which were presumed to be the chief motivations of them all. The result was pass fail course evaluations based on the subjective view of the professors and the loss of accountability and control of the course work and content by the universities. It lasted in some places for a decade or so, but has largely dissappeared from the scene as a seriously corrupting threat to the universities and their students.

Testing of learning is required to motivate students and discriminate between those who learn and those who don't. Measurement of the results and effectiveness of all the things we use is required to ensure their safety and quality (particularly important in areas like education where we have limited consumer choice). Measurement of the effectiveness of the services of most professionals is commonplace in our society and deemed necessary to ensure quality and reliability, None of these measures are perfectly accurate, but their use is obviously beneficial to all.

Why the hell should public school teachers be any different?

2 Replies

@georgeob1,

Ok, you seem to be having a hard time grasping this.

Measuring the skill level of teachers is a uniquely difficult enterprise.

Almost every other profession has much more clear data sets upon which to base job performance.

If someone is managing 20 employees, and five of them consistently fail to meet standards, the manager should fire them. The manager will be judged by how his or her employees carry out their duties. If the manager does not fire those five employees and hire better employees, that reflects badly on the manager.

Teachers don't have that option. If they have five students who consistently test below grade level, then they can do their best to bring up the test scores -- but there is no guarantee that even the best teacher can get those students to test at grade level. Those students cannot be "fired."

0 Replies

@georgeob1,

Quote:

It lasted in some places for a decade or so, but has largely dissappeared from the scene as a seriously corrupting threat to the universities and their students.

Really? Over a quarter of my classes as an undergrad at the University of Texas were taken on a pass/fail basis. And so were those of pretty much everyone else I know who went to school there. I don't know what 'corrupting effect' you think it has Laughing

Quote:

Testing of learning is required to motivate students and discriminate between those who learn and those who don't. Measurement of the results and effectiveness of all the things we use is required to ensure their safety and quality (particularly important in areas like education where we have limited consumer choice). Measurement of the effectiveness of the services of most professionals is commonplace in our society and deemed necessary to ensure quality and reliability, None of these measures are perfectly accurate, but their use is obviously beneficial to all.

Why the hell should public school teachers be any different?

Well, I'd say because a single child is far more complex than any of the situations you outlined in your post. It is far more difficult to accurately measure achievement and progress amongst young children than it is for adults, for a variety of reasons.

Cycloptichorn

1 Reply

@georgeob1,

georgeob1 wrote:

Cycloptichorn wrote:

Quote:
A recent Op ed in the WSJ by the former head of the New York city schools

You mean, the lady who had no qualifications for the position and no clue what she was doing? And was basically fired in disgrace herself? That former head? Not exactly swayed by her testimony.

No I was referring to the recently retired Administrator of the new York City Public schools system appointed by Mayor Bloomberg after he got authority from the state to take control of a seriously failing school district.

Yeah - that's the same person. She was a fool who had no clue what she was doing and left in disgrace after massive public pressure was applied.

She came from the corporate world, didn't attend public schools herself, and had no experience working with schools or the political challenges of managing a large district. Her opinion on the matter of standardized testing couldn't possibly mean less to me.

Cycloptichorn

1 Reply

@Cycloptichorn,

Cycloptichorn wrote:

[Really? Over a quarter of my classes as an undergrad at the University of Texas were taken on a pass/fail basis. And so were those of pretty much everyone else I know who went to school there. I don't know what 'corrupting effect' you think it has

What did you study?

Cycloptichorn wrote:

Well, I'd say because a single child is far more complex than any of the situations you outlined in your post. It is far more difficult to accurately measure achievement and progress amongst young children than it is for adults, for a variety of reasons.

Cycloptichorn

That's an intyeresting and rather novel proposition. Do you have any factual basis on which to support it?

Beyond that you have missed the central point. None of the measures in the broad array of very comples systems I cited above are perfectly accurate in individual cases. However their use has proven to be essential to ensure the quality and reliability of the system or organization they support - even in cases where the users & consumers of the services and articles involved have a free ability to choose alternatives. Performance asessment is even more crtitical in cases in which consumers have no or limited free choice - such as public education.

1 Reply

@Cycloptichorn,

Cycloptichorn wrote:

[Yeah - that's the same person. She was a fool who had no clue what she was doing and left in disgrace after massive public pressure was applied.

She came from the corporate world, didn't attend public schools herself, and had no experience working with schools or the political challenges of managing a large district. Her opinion on the matter of standardized testing couldn't possibly mean less to me.

Cycloptichorn

You are dead wrong. It was a he, not a she and he had long term experience in public education. Moreover it was the New York public schools, not those in Washington DC.

You are referring to (and slandering) Michelle Rhee who for a brief period brought some improvements to the Washington DC public school system - an institution that uniquely has for decades been among the nation's most expensive in terms of cost/pupil and near the bottom of the heap in terms of measured student performance, graduation rtates and college admissions.

The "massive public pressure" to which you refer came from a massively funded campaign by the AFT which felt (with justification) seriously threatened by her. The DC schools have since drifted back to their former miserable state.

The Washinton local of the AFT has an interesting record. A decade ago the Local President was convicted of embezzling about six million dollars from the union over a several year period. A Congressional investtigation revealerd the parent union hadn't audited the local's books for about 14 years.

1 Reply

@georgeob1,

Quote:

That's an intyeresting and rather novel proposition. Do you have any factual basis on which to support it?

no more so than you do your endless string of assertions. Let me simply say that the current regime of standardized testing doesn't seem effective in the slightest, teachers report that scores on those test don't accurately reflect learning amongst their students, and my personal experience with these tests doesn't inspire confidence that they do the job they are supposed to do.

Quote:

Beyond that you have missed the central point. None of the measures in the broad array of very comples systems I cited above are perfectly accurate in individual cases. However their use has proven to be essential to ensure the quality and reliability of the system or organization they support - even in cases where the users & consumers of the services and articles involved have a free ability to choose alternatives. Performance asessment is even more crtitical in cases in which consumers have no or limited free choice - such as public education.

Why is that performance assessment so critical? So you can be assured you are getting a 'good value' for your money? I'm not concerned with that as much as I am implementing a system that actually is effective in getting kids to learn. And as someone who grew up with the regime of standardized testing, let me assure you - they are horrible to go through, suck up valuable class time, and so tied to school funding nowadays that they crowd out learning opportunities and add basically no value.

I'm asserting that the data you are receiving from these tests doesn't accurately measure performance of individuals or groups in any way other than their ability to fill in certain bubbles on a scan-tron. That has been found to be the case at my place of employment; we have not seen that LSAT scores are tied to what we consider to be a good student, despite most people's belief that this test is a good predictor of that.

Cycloptichorn

0 Replies

Value Added Modeling: Administators v. Mathmaticians

Related Topics

Quick Links

My Account

able2know