Tags: * improve matching * ensure each paper has senior reviewer * AC should read paper * max of 4 papers per reviewer per cycle * for opt-out * against structured review forms
"1) I'm not convinced that reviewer mentoring is what is needed because bad quality reviews are a result
of reviewers not taking enough time to read and understand a paper or poor reviewer assignment rather
than reviewers not knowing what is expected. There are enough tutorials, blog posts and podcasts out
there now about how to write a good quality review. Better to focus on 1) better reviewer assignments,
2) making sure that each paper has a senior reviewer, 3) making sure the action editor actually reads
the paper rather than just summarising the reviews (maybe this is what you have in mind for action editor
mentoring?), and 4) assigning a max. of 4 papers to a reviewer per cycle. I also have the feeling that
we would get better value from action editors if they were reviewers instead. 2) I'm not convinced by
ARR but it seems too early to give up on it, and I don't think trying another system would improve the
situation. It's frustrating at this point when conferences decide to use both the old system and ARR.
3) I'm fine with the opt-out reviewer scenario as long as it is always ok to opt out or to set a maximum
paper load, no questions asked. Above a certain level of experience, you don't get any credit from your
institution for continuing to review so you are essentially doing it as a hobby at night or at the weekend
- at certain stages in life, family and work commitments can make this almost impossible. And it isn't
always possible to delegate reviews to graduate students because often graduate students have already
been assigned their five or six papers. So please trust people who opt out of reviewing for a few cycles
and understand that they are doing it because they simply don't have the time at that point to do an
acceptable job. Better to opt out than to deliver a useless review or to let everyone down by not submitting
at the last minute. Also, don't penalise graduate students by preventing them from submitting papers
because the last author hasn't got time to review! 4) Finally, can we also please stop expecting a particular
structure in both reviews and in papers? It's all becoming very formulaic! I'm pretty sure we are not
writing better quality reviews as a result of the structured review form, just ones that make it easier
for an area chair or action editor to write often not particularly insightful meta-reviews. The structure
isn't turning bad reviewers into good reviewers and I think it's constraining good reviewers."