by Hanie Sedghi, Hsuan-Tien Lin, Sungjin Ahn, Tristan Naumann
We’re excited to announce the NeurIPS 2022 workshops! We received 106 total submissions — a slight decline from last year. From this great batch of submissions, we accepted 63 workshops, 48 to appear in-person (Dec 2 & 3) and 15 to appear virtual (Dec 9).
Given the exceptional quality of submissions this year, we wish we could have accepted many more, but could not in order to accommodate logistical and technical constraints. This marks another year that workshop selection had to be selective, and we expect that many of the excellent proposals that we could not accept will be revised, resubmitted, and presented at other ML conferences.
Of course, we want to thank all the people who put in the effort to propose workshops. As Workshop Chairs, all we can do is guide the development of these proposals. The work is done by the organizers. So thank you!
New This Year: Hybrid NeurIPS and OpenReview
This year NeurIPS returns with an in-person portion of the event, but importantly continues to hold a virtual portion as well. As outlined in Get Ready for NeurIPS 2022, NeurIPS 2022 will take on a two-week format comprising both in-person and virtual. For workshops, this means that in-person workshops will be held Friday (Dec 2) and Saturday (Dec 3), as was traditionally the case with previous in-person events; and, virtual workshops will be held the following week on Friday (Dec 9).
In soliciting workshop proposals, we asked organizers to specify a preference for in-person, a preference for virtual, or whether they would only consider one of these formats. While we were expecting many submissions to express a preference for in-person, a substantial number expressed a preference for virtual or to be considered only for virtual format. As we think about the opportunities that hybrid NeurIPS presents, we will keep an eye on how to best refine workshop opportunities in the future to best leverage the advantages of in-person and virtual portions of the program.
Another change that we made for this year was to switch from CMT to OpenReview. Superficially, this aligns our reviewing platform with many of the other NeurIPS submission tracks. However, it also allowed us to experiment with OpenReview’s capacity to match reviewers to proposals while managing conflicts-of-interest. There were some notable challenges in this regard—particularly around the quality of matches for reviewers with incomplete OpenReview profiles—but overall the platform was able to support the identification of high-quality proposals. Additional details about the selection process are provided below.
Refinements: Submission format and workshop reviewers
Among the first things we did this year was take into account feedback from last year’s submission template. This submission template diverged substantially from prior years and was designed to improve the ease of submission and reduce the variance in the workshop reviews by standardizing the length of the workshop proposals and clarifying the content relevant for reviewer evaluation.
Specifically, in response to feedback, we increased the length of the main proposal to three pages (over last year’s two pages) while keeping the prior year’s two-page restriction for organizer information, along with unlimited references.
As with last year, we included specific details that were intended to both facilitate review and solicit content that could be readily converted into a CFP. Some specific details we requested included:
- A list of invited speakers, if applicable, with an indication of which ones have already agreed and which are tentative.
- An account of the efforts made to ensure demographic diversity of the organizers and speakers and an account of any efforts to include diverse participants.
- An estimate of the number of attendees.
- A description of special requirements and technical needs.
- A very brief advertisement or tagline for the workshop, up to 140 characters, that highlights key information for prospective attendees to know, and which would be suitable to be put onto a web-based survey.
- A URL for the workshop website.
- The two pages (or fewer) for information about organizers must include: The names, affiliations, and email addresses of the organizers, with one-paragraph statements about their research interests, areas of expertise, and experience in organizing workshops and related events.
- Finally, a list of Programme Committee members, with an indication of which members have already agreed, anticipating that no one is committed to reviewing more than 3 papers.
Another point of feedback that was incorporated was to further refine the reviewing recruitment process. In particular, we started with a larger reviewing pool of experienced workshop reviewers and asked each workshop to elect one or more organizers that would be willing to serve as a reviewer.
Due to the large reviewing pool, many of the nominees were not required. However, the nomination of reviewers allowed us to balance reviews from senior members of the NeurIPS community alongside those that were representative of the organizers making proposals. We thank all the reviewers for their timely and professional efforts to provide quality reviews that greatly assisted our decision making.
In making our selections, we asked the reviewers to closely follow our Guidance for Workshop Proposals, which was also shared with the proposers. Workshop proposals must be reviewed somewhat differently from academic papers, and we asked the reviewers to consider both scientific merits and broader impacts in their assessments. We recognize that workshop reviews are perhaps more subjective than academic paper reviews. Following the practice of previous years, we will not be releasing the reviews directly to the proposers. However, following the precedent of last year, we released a short meta-review alongside the decision for each proposal — aiming to highlight what could be improved, or explaining how the proposal was perceived by the reviewers.
Individual evaluations of proposals by reviewers were important to the decision process, but they were not the only consideration for acceptance. For example, we also strived for a good balance between research areas and between application and theory. Because interest in research areas is not uniform, some areas were more competitive than others. For example, there were many strong reinforcement learning proposals. We also received many submissions on important current topics such as federated learning, privacy, fairness, causality, NLP, and needless to say various topics on deep learning. We attempted some balance of topic areas to cover both mainstays and emerging topics.
It is also worth noting that we saw many of the pitfalls outlined in previous years. This included leaning too heavily on past success, unconfirmed or irrelevant speakers, insufficient time for discussion, going too big and broad, and diversity lip service.
We learned a lot from last year’s NeurIPS Workshop Chairs about the constraints and opportunities of virtual workshops, e.g. (and perhaps most importantly), encouraging the use of a shared technology platform. These challenges and opportunities are amplified in this year’s hybrid format, which in many ways requires decisions that are beneficial to both in-person and virtual workshops. Our chosen technology platforms allow us to tackle technology issues more effectively, pool resources across workshops, and simplify the user experience for workshop attendees.
The next step is your contributions! Many workshops have begun soliciting submissions, many using our suggested submission date of September 22, 2022. We typically let each workshop advertise its own call for papers if they plan to include workshop papers. This year, we are imposing a few deadlines because of the choice to use a common platform for the talks, which appeared to be necessary for hosting 60+ events at the same time in a smooth manner. More technical and contextual information is coming soon!
NeurIPS 2022 Accepted Workshops
On to the best part: the preliminary list of accepted workshops for 2022!
FRIDAY, DEC 2 (IN-PERSON)
SATURDAY, DEC 3 (IN-PERSON)
FRIDAY, DEC 9 (VIRTUAL)
– Adversarialism in Machine Learning and the Law – CANCELLED