Comment 133

next comment

Tags: * decisions should be made based on evidence and research not based on surveys

"The discussions about reviewing in our community are focused way more on personal opinions than actual 
evidence and research. As you can see, this survey also collects opinions. I really like Nihar's survey 
about research on peer review http://www.cs.cmu.edu/~nihars/preprints/SurveyPeerReview.pdf The folks 
organizing ICML, NeurIPS, AAAI actually use this research for bettering their processes and as we all 
know they have run many smart experiments to help themselves. https://blog.ml.cmu.edu/2022/04/01/peer_review_experiments_2022/ 
https://blog.neurips.cc/2021/12/08/the-neurips-2021-consistency-experiment/ https://blog.ml.cmu.edu/2020/12/01/icml2020exp/ 
http://inverseprobability.com/talks/notes/the-neurips-experiment.html We need to get better at this!! 
Thanks for doing this survey btw"