Tags: * need new reviewing criteria * against junior reviewers * increase number of / load of senior reviewers * for subreviewers
"I do not believe that all authors of papers are qualified as reviewers. Many articles have applied
scientists and engineers who are technically qualified at experimentation, but less qualified in scholarship
and awareness of prior work. This leads to a review pool dominated by concerns around the last 6mos
of incremental advances with a focus on shared task improvements, and it allows works to be accepted
that are largely duplicating prior insights that were explored with previous generation underlying models.
More work should be done to insist our papers are making meaningful contributions, rather than being
fast to apply the latest model pushed to hugging face (and claiming extreme novelty). In addition, an
early career graduate student is also often not qualified to be a reviewer. I would rather see group
leads (in industry and academia) being asked to review many more papers, but their junior staff/students
being removed from that process. Secondary reviewers with a senior group lead has historically allowed
group leads to work with their junior members to educate them on how to write a review. As it is, many
of my students are now writing reviews that I don’t see, and can’t give feedback on unless I overtly
engage in their own assignments. The notion of Area Chair is not a solution: area chairs are not mentors,
they are over busy people that at best may calibrate the review scores against the seniority of the
review author. If you tag senior faculty (like myself) with a dozen papers to review but leave my students
out of it, then I’ll ensure you get higher quality reviews than what my individual students would
write themselves, and those students will improve over time until they are senior enough to give high
quality reviews themselves."