I'm confused by "experimental questions"....

Forums: School, Education, Testing, Experimental Questions
Email this Topic • Print this Page

Did you read about the hubbub in New Jersey?

Quote:

Some New Jersey parents are steamed about a question on a statewide standardized test this week that asked some third-graders to write about a secret and why it was hard to keep.

Richard Goldberg, a Marlboro dentist, was appalled when he asked his twin 9-year-old sons about the New Jersey Assessment of Skills and Knowledge and they told him about the question, which state officials say was given to about 4,000 students as a tryout.

......

He said the question itself is being tested and that it was vetted for appropriateness by both the department and a panel of teachers. He said it was given in 15 districts to about 4 percent of the third-graders statewide who took the exam. Like other experimental questions, the answers will not count toward students’ scores.

http://www.washingtonpost.com/national/question-on-some-nj-standardized-tests-asks-3rd-graders-to-reveal-a-secret-upsetting-parents/2012/05/11/gIQANGJKIU_story.html

It seems like a strange question to ask 4th graders on a test but really, I'm curious about why they would ask "experimental questions" at all?

It really seems that they could come up with better writing prompts for short answer test questions -- ones that would have more to do with some kind of scholastic activity.

What do you think about "experimental questions"?

Does this bother you at all?

Topic Stats
Top Replies
Link to this Topic

Type: Question • Score: 10 • Views: 3,839 • Replies: 39

I don't have a problem with the idea of experimental questions--however, i don't think they should be in any test for credit, even if they aren't for credit, but should be presented as what they are in a separate exercise. This particular effort is rather bizarre, though. Telling a secret is betraying a trust. Are we telling children to betray a trust if someone in authority demands it? Did the children know it was not for credit? Were they aware that they were not obliged to answer that question? Strange days.

1 Reply

i wonder what i might have done

i was a fairly precocious kid, with a good imagination and a certain disdain for the educational system (even then, the first time i skipped school was grade four)

i imagine i would have made up something pretty preposterous

0 Replies

@boomerang,

This is new to me, sounds like the point is to test the question rather than the kid?

As in, the point is to ferret out ways that it could be misunderstood or otherwise not work well, before putting it in a test where it would count.

0 Replies

@boomerang,

Quote:

Some New Jersey parents are steamed

seems like it doesn't matter what the situation is, there will be some parents upset about it

Test questions are often tested on a small group before being applied to the whole/larger group. nothing new there. I'm not sure how else test developers are supposed to determine if they will get measurable/valid results.

1 Reply

@Setanta,

They're so secretive about these tests that I don't know what the kids knew about the question. I did come across this:

Quote:

A further complication may be that at least some teachers tell their students that they can make up their answers if they don't have real-life examples to give. What matters, the teachers say, is the form of the writing, not whether what they say is true.

Which leads me to believe that both the teachers and kids thought the answer would count towards their score.

Of course, the scorers wouldn't know whether the kid had made it up or not. Like djjd, given the opportunity I probably would have come up with something extreme.

Also, I thought that the teachers weren't supposed to give any explicit instructions on the tests so I question why some teachers were giving instructions and others weren't.

0 Replies

By the way, when i was in the 5th grade, some students from a nearby university (i would later understand that these were graduate students) came to our school, to confer with the teachers. Those teachers chose students in their classes (i have no idea by what criterion), and the test was administered. I was one of the test takers. We knew it was not a test for credit insofar as our school was concerned, we knew we were not obliged to take the test, and we knew we were not obliged to answer all the questions.

0 Replies

Yeah, soz and ehBeth, I get that.

But are these tests the right place to do that? Is it okay that nobody knows which scores count and which ones don't?

These scores have an effect on school funding (so on your property value too), teacher pay, student "outcomes", etc.

Something seems very hinky to me about this.

Maybe they should pay focus groups of students to try out their experimental questions.

2 Replies

I'd really like to see the actual question. It could have been: "Did you ever have to keep a secret? Why or why not was that difficult." The answer could reveal thinking and organizational skills in writing.

If the question was what secret they had to keep, that would be totally inappropriate.

1 Reply

@boomerang,

boomerang wrote:

Maybe they should pay focus groups of students to try out their experimental questions.

you're definitely not going to get valid results (speaking as a paid focus group student from 40+ years ago - great $ but we didn't represent anything like an average group of students)

1 Reply

@ehBeth,

I don't have the answer to how they should test these questions but I don't think it should be on these achievement type tests.

Maybe it should be like Setanta's example.

Maybe it should be a random group of kids who remain anonymous.

1 Reply

@ehBeth,

Quote:

seems like it doesn't matter what the situation is, there will be some parents upset about it

Thank goodness for that!

It makes me think of the years where groups were trying to call attention to ALEC and they were being ignored until Treyvon Martin got killed.

The fact is, most people don't really pay attention until something gets called out by a few people.

1 Reply

@boomerang,

boomerang wrote:

Maybe it should be a random group of kids who remain anonymous.

that is what New Jersey was doing til the dad went public - no one would have known about it otherwise

~~~

teacher's selecting doesn't work well for randomness

~~~

which all goes back to ... no matter what they do at school, some parent will complain

1 Reply

@PUNKEY,

Yeah. We don't really know and they won't release the question because the tests are secret.

0 Replies

@boomerang,

I know that they include experimental, or pilot, questions in certification exams in my industry. Basically, it's a cost-saving measure.

They can either create test exams, and pay people to take them, or they can include experimental questions in the real tests that people are already taking.

That said, it seems like this question is particularly troublesome.

1 Reply

@ehBeth,

These tests are anything but anonymous.

0 Replies

@DrewDad,

But the people taking these certification exams aren't 8 years old and being asked about keeping secrets.

0 Replies

@boomerang,

boomerang wrote:

Quote:
seems like it doesn't matter what the situation is, there will be some parents upset about it

Thank goodness for that!

I couldn't disagree more. It's hard enough to get good teachers/educators to stay. Every complaint by a parent seems to lead to more bureaucracy and less opportunity for teachers to teach. It's bad in Canada, but it seems much worse in the U.S.

1 Reply

@ehBeth,

These tests waste such enormous amount of time and money that I'm glad that people are starting to ask "WTF?"

If we eliminate these tests we'll save billions of dollars so that we can hire more teachers and allow them to actually teach during the hundreds of hours we'll get back from doing away with test preparation.

0 Replies

What is the big deal about this question?
And what is the problem with testing out whether a question is usable?

Jeez
What eight year old hasn't ever been told to keep a secret?
"Shh, we're taking Mommy out to Olive Garden. Don't tell.!"
Schoolyard
"Dennis likes Allison."
"So?"
"So don't tell Allison."

meh.

And one more thing:
Writing tests is harder than writing comedy and writing comedy is as hard as the biscuits my ex-wife used to bake on Sundays.

Joe(I still have some I use to drive nails into concrete)Nation

1 Reply

I'm confused by "experimental questions"....

Related Topics

Quick Links

My Account

able2know