Tags: * for discussion * link all versions in openreview * link meta/reviews to acceptance * differentiate tier 1 and tier 2 conferences * increase (S)AC diversity * increase AC seniority * publish papers before conference * equal spacing of conferences * ban arxiv * appeals process for bad reviews * incorporate community feedback on ethics/reproducibility * improve interface for reviewer matching
"- ARR papers should have a discussion forum enabled, and revisions should be uploaded and reviews updated
in the same thread, with everything easily viewable in a timeline form. - A rolling review process should
also return decisions to specific conferences, not only reviews which can then be subjectively reinterpreted
by the conference committee. - Aim to introduce some tier of acceptances e.g. ACL/EMNLP, followed by
regional conferences (NAACL, EACL, AACL), Findings. The paper is eligible for either NAACL, EACL, AACL
based on their affiliation. Right now, all regional conference aspire to be international conferences
and creates in my opinion unhealthy competition and encourages paper resubmission without changes. -
Introduce some minimal criteria for ACs/SACs and improve diversity/rotation. ARR currently has some
inexperienced area chairs. Conference senior area chairs, on the other hand, are often cycled from a
small select group of people on a given track e.g. see Computational Social Science or Sentiment Analysis
in the past 5 years. I think it would be beneficial for quality and spread of the work to involve other
people and select from people who have published papers in that specific area in recent years. - Decouple
paper publication in anthology/acceptance to a specific conference from when the conference is actually
taking place. The main effect of this will be that papers can be accepted to a conference throughout
the year, which will remove the rush of a single yearly deadline for a conference and make paper publication
in the anthology faster. Right now, realistically from submission/preprint to presentation at a conference
there is a 6-8 month lag (if the paper is accepted on the first try), so I would argue conference presentations
are usually not presenting the latest work anyway e.g. ACL now is happening after the next conference
(NAACL) had its acceptances and people mostly are talking about the NAACL papers as the most recent
work. - Coordinate across the ACL organization to space out the conferences across the year - Aim to
ban arxiv submissions down the line, rather than the ineffective current preprint policy. Submitting
to a rolling review system that is well executed would give open preprints and the author names can
be revealed after the review round if the authors want, which should take 2 months at most - Create
a method to appeal on factuality of reviews (e.g. reviewers asking for experiments already in the paper)
and replace reviewers on these grounds, rather than 'no questions asked', which encourages the current
resubmission roulette. - Have proper community feedback and discussions on Ethics/Reproducibility, rather
than imposing or experimenting various checklists or reviews as currently done - Rather than completely
automatic review assignments, have tools that actually facilitate area chairs to find the relevant reviewers
(e.g. based on authors cited/doing similar work, overall seniority, seniority on the topic). Currently,
ARR assigns reviewers automatically, then ACs can only theoretically change assignments, because all
reviewers are filled up in their review quota, there is no wiggle room to add the reviewers you want
or rebalance seniority."