NeurIPS Is Once Again Amid Controversies Due To “Terrible” Reviews
This year, the thirty-fourth annual conference on Neural Information Processing Systems, NeurIPS 2020 is going to be held virtually from 6th to 12th December. The paper submissions for this year is 38% more than last year, i.e. a total of 1,903 papers were accepted, compared to 1,428 last year. The review period of the papers began in July, and in August, the popular AI conference, NeurIPS has sent out the paper reviews for this year’s conference. This has brought the popular machine learning event once again amid the controversies as it has been claimed that the reviews of the papers are “terrible”, i.e. either they are not clear, or the sentences were incomplete by the reviewers, among others. In a tweet, Director of AI Research at NVIDIA and California Institute of Technology Professor, Anima Anandkumar shed light on the paper reviews. Anandkumar said that she has been witnessing terrible paper reviews for the “nth time” and it is worthy of banning the reviewers from submitting as authors “if they can’t write a review in good faith”. https://twitter.com/AnimaAnandkumar/status/1291951068314660865 Replying to this tweet, an alumnus of IISc Bangalore, Suraj Krishnamurthy stated that the reviews of his paper were worse, and there were “absolutely no takeaway messages from the reviewers.” https://twitter.com/surajkmurthy/status/1291954730202095617 Google AI researcher, Rohan Anil mentioned in a tweet that the reviews are poor in nature with incomplete sentences by the reviewers. Reviewer #4 – atleast complete your sentence. I don't have access to GPT3 to complete the "Th" you left in the one sentence review, do you mean thunder, or Thursday.#NeurIPS2020 Somehow the poor reviews does not hurt this time around given the situation we are going through.— rohan anil (@_arohan_) August 7, 2020 Some of the machine learning researchers suggested a few solutions that can help in solving the “terrible” reviews. Christian Szegedy, an AI research scientist, suggested that the system should propose a different review process. One suggested way that can improvise the system is by decoupling the task of judging and analysing the papers. I think one problem is that reviewers task is mostly focused on judging the paper's quality. What I think could improve the system greatly is decouple the task of judging and analyzing the paper. I would propose a different review process.[1/n]— Christian Szegedy (@ChrSzegedy) August 8, 2020 Szegedy added, “I would suggest a review process in which in the first phase, the (fewer) reviewers’ task is to analyse the paper. The reviewers should have the opportunity to give an enthusiastic recommendation of acceptance or reject if they feel strongly, but are not forced to rate at all.” I would suggest a review process in which in the first phase the (fewer) reviewers' task is to analyze the paper. The reviewers should have the opportunity to give an enthusiastic recommendation of acceptance or reject if they feel strongly, but are not forced to rate at all.— Christian Szegedy (@ChrSzegedy) August 8, 2020 Replying to his own tweet, Szegedy further mentioned that in the second phase, the reviewers should stack rank the paper based on the first round’s analysis. This would be useful for borderline papers, for which the reviews are lukewarm. In the second phase the reviewers should stack rank the paper based on the first round's analysis. This would be useful for borderline papers, for which the reviews are lukewarm.— Christian Szegedy (@ChrSzegedy) August 8, 2020 While Victor Zakhary, SWE at Oracle tweeted that the system should have something similar to Arxiv with public reviews in a decentralised manner or utilise a blockchain-like solution where papers reviews are public and tied to their writers. https://twitter.com/vzakhary/status/1291961506016292869 However, some of the machine learning enthusiasts had a different take for the review system. In a comment, a Reddit user mentioned that the “NeurIPS reviews received by the user were “actually really fair and detailed for the most part.” The user also mentioned that the problem is NeurIPS wants publications to be new and not published elsewhere (except Arxiv). Comment byu/Yuqing7 from discussion indeeplearning Also, Graham Neubig, Associate professor at CMU, mentioned in a tweet that 70-80% of the reviews at NeurIPS2020 were reasonable and fixing any complaints will make the papers better. To go against the grain a bit: 70-80% of our reviews at #EMNLP2020 and #NeurIPS2020 were reasonable and fixing any complaints will make our papers better. Sure, we did get a couple 1-sentence reviews, major misunderstandings, "not novel" (no citations), but overall things were OK— Graham Neubig (@gneubig) August 9, 2020 This is not the first time that controversies have scarred the reputation of the conference. In other words, it can be said that controversies are not a new thing for this popular machine learning conference. In 2018, the organisers of the “NIPS” (Neural Information Systems Processing) conference had changed the event’s name from NIPS to NeurIPS after heading into a controversy about whether “NIPS” is an offensive name or not. According to sources, signed a letter calling on NIPS to be rebranded following reports of inappropriate behaviour. The letter stated that the “acronym of the conference is prone to unwelcome puns.”
This year, the thirty-fourth annual conference on Neural Information Processing Systems, NeurIPS 2020 is going to be held virtually from 6th to 12th December. The paper submissions for this year is 38% more than last year, i.e. a total of 1,903 papers were accepted, compared to 1,428 last year.  The review period of the papers […]
The review period of the papers began in July, and in August, NeurIPS has sent out the paper reviews for this year's conference.
["NeurIPS", "virtual AI conference"]
["Go", "machine learning", "AI", "NeurIPS", "NLP", "GPT", "Aim", "virtual AI conference", "GAN", "AI research", "R", "llm_models:GPT"]
["AI", "machine learning", "NLP", "Aim", "R", "Go", "GPT", "GAN", "AI research", "llm_models:GPT"]
A Technical Journalist who loves writing about Machine Learning and Artificial Intelligence. A lover of music, writing and learning something out of the box.