Page 1 of 5

The subjectivity of judging

Posted: Thu Nov 09, 2017 5:04 pm
by David Campbell
“Even when all the competition guidelines are scrupulously followed, there can never be any guaranteed winning formula for the simple reason that judging is extremely subjective.”

I wrote that in the August/September issue of the ABPA magazine in response to a member who claimed that the awards market had been cornered by a select few because “some people have understood and worked out the judging assessment system”. He was making this comment because Tom had picked up several awards at Dunedoo.

It concerns me that potential entrants to written competitions might be put off by misguided assumptions like this, and it’s worth illustrating the reality of the situation in the light of the results for the serious section in the just-concluded National Championships at Toodyay. I won with “Sweetwater” and Will came second with “Driftwood”, but that result jogged a memory, so I checked back through my files to the outcome of last year’s Bush Lantern Award, another competition that used three judges. In the Bush Lantern the judges chose their top 20 poems independently (and anonymously) and allocated a mark out of 100 to each. The marks were then added to reach a total, with the highest overall score winning. Fairly obviously, you were only in with a chance if all three judges placed you in their top 20. And that’s where it gets interesting.

Terry won with “One Man’s Prison”, which earned two second placings and a fifth. Although none of the judges put it first, it was clearly the most consistent result and thus a deserving winner. But further down the list is something quite intriguing that links to the National Championships. In 10th place was “Sweetwater” with a second, a fourth…and a big fat zero from one judge who didn’t include it in the top 20. And one above it, in ninth place, was “Driftwood” with…wait for it…two FIRST placings and a big fat zero from one (different) judge. I remember looking at that result and thinking that whoever wrote “Driftwood” must have been really cheesed off, wondering how on earth a poem could make the top of the list with two judges and miss out altogether with another. And, as a footnote, down at number 19 overall, with 13th, 15th and zero, was Tom’s “Bobby”, which I assume is the same poem that won first prize at Corryong this year.

But that’s the way it goes. What one judge likes, another won’t consider…often because of the subject-matter. That’s why poems can wander around in the competition wilderness for ages and then suddenly win an award. As I’ve said many times, belief in your work has to go along with persistence. When it comes to decisions about the merit of any sort of artistic endeavour, there is always a considerable amount of luck involved.

Cheers
David

Re: The subjectivity of judging

Posted: Fri Nov 10, 2017 4:39 pm
by Shelley Hansen
David - your comments are so insightful!

I also remember seeing that strange result for "Driftwood" in the 2016 Bush Lantern Award - namely that two judges had placed it first and the third judge had not ranked it in the Top 20 at all - thus dragging its overall ranking way down the list. Like you, my sympathies were with the poet who wrote it as I thought "how unlucky can you be"? At the time I didn't know it was Will's poem. At least it has now received deserved recognition with its second placing in the Australian championships.

As you say, that perfectly illustrates the point that absolutely NONE of us have the judging "worked out", as was suggested by someone in response to your ABPA magazine articles a while back. We all have the same experiences - poems that are completely overlooked at one competition may do well at another. Sometimes I have persisted with poems for several years in multiple competitions - then have suddenly achieved success when on the point of giving up. Of course, each competition includes a fresh field, so our poems may rank higher or lower depending on the quality of the other entries.

As you say, we can't let any of this undermine our belief in our own work or our passion for writing. Sometimes, when you just have that "good" feeling about a certain poem, it's worth the effort to persist.

Congratulations again on your success with "Sweetwater"!

Cheers
Shelley

Re: The subjectivity of judging

Posted: Tue Nov 21, 2017 11:24 am
by David Campbell
Thanks, Shelley. It's always fascinating (and often frustrating) to trace what happens to a poem (or a short story, for that matter) in written competitions over a number of years. Regularly I go back to something asking the question "What's wrong with it?" With traditional poems I double-check the metre and rhyme, wonder if the language could be improved, and generally come back to the greatest conundrum...the storyline. That's usually the big question mark. Something about the subject-matter itself, or the way the story is presented, is likely to be causing a problem. But I have no intention of changing it, so that's where having faith in what you've written becomes important. And that's certainly the case with "Sweetwater".

Cheers
David

Re: The subjectivity of judging

Posted: Tue Nov 21, 2017 10:44 pm
by Terry
G/day David

I think we've all have had similar experiences, some judges like your poems; others aren't so keen.
I don't let it worry me overmuch, because that's poetry and the way it ought to be.
I think I mentioned somewhere before, that in my opinion there's no right or wrong style - just different ones.
The worst thing you can do, is to try and write in a way you may suspect a certain judge, or judges might like.

You write what you love to write, and if you get lucky once in a while, that's a bonus I reckon.

Cheers

Terry

Re: The subjectivity of judging

Posted: Wed Nov 22, 2017 9:04 am
by Terry
An after thought
David you mentioned how yours and Will's poems missed out with one judge at the Bush Lantern.
My winning poem met a similar fate earlier at Cooryong (don't know how to spell that) with the judges there.
Mind you I seldom if ever do well at that one - I ran out of time and didn't enter it this year.
If you enter competitions on a regular basis, you soon notice that you seldom if ever do well with some judges.

Cheers

Terry

Re: The subjectivity of judging

Posted: Wed Nov 22, 2017 3:30 pm
by Shelley Hansen
Hi Terry

It's interesting you've had that experience at specific competitions. It has been the same for me.

I suppose if a particular judge regularly judges a particular competition, this is always going to happen. If the judge is fortunate enough to be able to select a shortlist of poems, all of which have correct rhyme and metre and fit the criteria in other respects, then it has to come down, not just to personal preference of subjects, but styles of writing.

I'm sure that like me, you've had the experience of being able to admire the crafting of a particular poem, while at the same time not actually being "moved" by it. On the other hand, you read some poems which just strike the right chord. The fact that it's different for all of us is, as you say, part of the wonderful diversity of our craft.

That's why, as David says, if we believe in our work, then we should persist.

And as you so rightly say ...
You write what you love to write, and if you get lucky once in a while, that's a bonus I reckon
Cheers
Shelley

Re: The subjectivity of judging

Posted: Fri Dec 08, 2017 7:22 pm
by David Campbell
Yes, Terry, I've also had that feeling on occasions. Just the way it goes if the same judge is used regularly. On the other hand, there are also competitions which seem to draw judges from all over the place and a few really odd results slip through. Result sheets are occasionally sent out and some of the written comments leave me shaking my head in bewilderment.

Cheers
David

Re: The subjectivity of judging

Posted: Fri Dec 08, 2017 10:47 pm
by Brenda Joy
Just adding a different aspect of how judging can seem to be into the mix.
When raw scores were used, the judge with the greatest 'swing' in marks would decide the winners in both written and performance competitions. For instance if one judge marked between say 75 and 85 that would mean that judge's swing would be 10. Another judge might have a swing between 85 and 95 = still only 10 between highest and lowest. However, a third judge (not necessarily the most experienced) might have scores between 60 and 90 = a 30 mark swing. That judge therefore would largely determine the results.

With rank order, each judge's positioning gives the same weight. This eliminates that disparity. However, it does not always come across as well. If there are many poems that are very close in merit, there might only be two or three 'marks' between poem 1 and 10 e.g. 1 = 88, 2 = 87.75, 3 = 87.50, 4 = 87.25, 5 = 87. If poets received the result that 1st scored 88 and 5th scored 87, the 5th would think he/she was very close to winning (which indeed is the case) but when given a 5th in rank order, it doesn't sound the same. Take this across three poets and you can see that although it might seem as if there is a great difference in marking, the results may be closer than they come over.
Probably just trying to say that there are some wonderful poems coming through the system and deciding between a large number of extremely well-written poems is a very tough task indeed. "Who'd be a judge?" as Milton once said.

Re: The subjectivity of judging

Posted: Fri Dec 08, 2017 11:04 pm
by Terry
Hi David & Shelley

I must be honest and say that I've never been a fan of the present judging sheet (assuming it's still the same).
But that's just me, mind you it doesn't stop me from entering a comp.

Speaking of old poems sometimes popping up and winning; my poem 'The Old Brush Hut,' was one of those poems that often did well without actually winning - perhaps such poems just circle around looking for a sympathetic judge.

Another thing that strikes me is that if a judge, or judges, likes someones poetry, they may pick up two or even three (more if you're Tom) awards in the same comp.

Terry

Re: The subjectivity of judging

Posted: Sat Dec 09, 2017 11:10 am
by David Campbell
Yes, Brenda, it can get rather complicated with multiple judges. I judged the Bush Lantern a couple of times and it was always fascinating to see the variations between the three of us...and I never knew who the other two judges were. I've never been a fan of allocating scores because of our different understandings of what the numbers mean. There always seemed to be one judge who never gave anything much over 80 out of 100, and what was required to get a mark in the nineties from him/her remained a complete mystery.

Terry, that's a good point about winning multiple awards when a judge likes a particular style. By the same token, if a judge doesn't like that style, you won't get anywhere. If I'm entering multiple poems in one section (usually no more than two) I always try to make them as different as possible...different styles, structures, and subject-matter. So that, if the judge encounters them one after the other (which will happen unless the organisers mix them up), it's (hopefully!) not possible to tell they were written by the same person. That, of course, can never stop judges thinking: "Hmmm...bet I know who wrote that!"

Cheers
David