Menu

2023 Conference

Announcing the NeurIPS 2023 Paper Awards 

,

Announcing the NeurIPS 2023 Paper Awards 

By Amir Globerson, Kate Saenko, Moritz Hardt, Sergey Levine and Comms Chair, Sahra Ghalebikesabi 

 

We are honored to announce the award-winning papers for NeurIPS 2023! This year’s prestigious awards consist of the Test of Time Award plus two Outstanding Paper Awards in each of these three categories: 

  • Two Outstanding Main Track Papers 
  • Two Outstanding Main Track Runner-Ups 
  • Two Outstanding Datasets and Benchmark Track Papers  

This year’s organizers received a record number of paper submissions. Of the 13,300 submitted papers that were reviewed by 968 Area Chairs, 98 senior area chairs, and 396 Ethics reviewers 3,540 were accepted after 502 papers were flagged for ethics reviews. 

 

We thank the awards committee for the main track: Yoav Artzi, Chelsea Finn, Ludwig Schmidt, Ricardo Silva, Isabel Valera, and Mengdi Wang. For the Datasets and Benchmarks track, we thank Sergio Escalera, Isabelle Guyon, Neil Lawrence, Dina Machuve, Olga Russakovsky, Hugo Jair Escalante, Deepti Ghadiyaram, and Serena Yeung. Conflicts of interest were taken into account in the decision process.

Congratulations to all the authors! See Posters Sessions Tue-Thur in Great Hall & B1-B2 (level 1).

Outstanding Main Track Papers

Privacy Auditing with One (1) Training Run
Authors: Thomas Steinke · Milad Nasr · Matthew Jagielski

Poster session 2: Tue 12 Dec 5:15 p.m. — 7:15 p.m. CST, #1523

Oral: Tue 12 Dec 3:40 p.m. — 4:40 p.m. CST, Room R06-R09 (level 2)

 

Abstract: We propose a scheme for auditing differentially private machine learning systems with a single training run. This exploits the parallelism of being able to add or remove multiple training examples independently. We analyze this using the connection between differential privacy and statistical generalization, which avoids the cost of group privacy. Our auditing scheme requires minimal assumptions about the algorithm and can be applied in the black-box or white-box setting. We demonstrate the effectiveness of our framework by applying it to DP-SGD, where we can achieve meaningful empirical privacy lower bounds by training only one model. In contrast, standard methods would require training hundreds of models.

 

Are Emergent Abilities of Large Language Models a Mirage?
Authors: Rylan Schaeffer · Brando Miranda · Sanmi Koyejo

Poster session 6: Thu 14 Dec 5:00 p.m. — 7:00 p.m. CST, #1108

Oral: Thu 14 Dec 3:20 p.m. — 3:35 p.m. CST, Hall C2 (level 1) 

 

Abstract: Recent work claims that large language models display emergent abilities, abilities not present in smaller-scale models that are present in larger-scale models. What makes emergent abilities intriguing is two-fold: their sharpness, transitioning seemingly instantaneously from not present to present, and their unpredictability, appearing at seemingly unforeseeable model scales. Here, we present an alternative explanation for emergent abilities: that for a particular task and model family, when analyzing fixed model outputs, emergent abilities appear due to the researcher’s choice of metric rather than due to fundamental changes in model behavior with scale. Specifically, nonlinear or discontinuous metrics produce apparent emergent abilities, whereas linear or continuous metrics produce smooth, continuous, predictable changes in model performance. We present our alternative explanation in a simple mathematical model, then test it in three complementary ways: we (1) make, test and confirm three predictions on the effect of metric choice using the InstructGPT/GPT-3 family on tasks with claimed emergent abilities, (2) make, test and confirm two predictions about metric choices in a meta-analysis of emergent abilities on BIG-Bench; and (3) show how to choose metrics to produce never-before-seen seemingly emergent abilities in multiple vision tasks across diverse deep networks. Via all three analyses, we provide evidence that alleged emergent abilities evaporate with different metrics or with better statistics, and may not be a fundamental property of scaling AI models.

Outstanding Main Track Runner-Ups

Scaling Data-Constrained Language Models
Authors: Niklas Muennighoff · Alexander Rush · Boaz Barak · Teven Le Scao · Nouamane Tazi · Aleksandra Piktus · Sampo Pyysalo · Thomas Wolf · Colin Raffel

Poster session 2: Tue 12 Dec 5:15 p.m. — 7:15 p.m. CST, #813

Oral: Tue 12 Dec 3:40 p.m. — 4:40 p.m. CST, Hall C2 (level 1)  

 

Abstract: The current trend of scaling language models involves increasing both parameter count and training dataset size. Extrapolating this trend suggests that training dataset size may soon be limited by the amount of text data available on the internet. Motivated by this limit, we investigate scaling language models in data-constrained regimes. Specifically, we run a large set of experiments varying the extent of data repetition and compute budget, ranging up to 900 billion training tokens and 9 billion parameter models. We find that with constrained data for a fixed compute budget, training with up to 4 epochs of repeated data yields negligible changes to loss compared to having unique data. However, with more repetition, the value of adding compute eventually decays to zero. We propose and empirically validate a scaling law for compute optimality that accounts for the decreasing value of repeated tokens and excess parameters. Finally, we experiment with approaches mitigating data scarcity, including augmenting the training dataset with code data or removing commonly used filters. Models and datasets from our 400 training runs are freely available at https://github.com/huggingface/datablations.

 

Direct Preference Optimization: Your Language Model is Secretly a Reward Model
Authors: Rafael Rafailov · Archit Sharma · Eric Mitchell · Christopher D Manning · Stefano Ermon · Chelsea Finn

Poster session 6: Thu 14 Dec 5:00 p.m. — 7:00 p.m. CST, #625

Oral: Thu 14 Dec 3:50 p.m. — 4:05 p.m. CST, Ballroom A-C (level 2)  

 

Abstract: While large-scale unsupervised language models (LMs) learn broad world knowledge and some reasoning skills, achieving precise control of their behavior is difficult due to the completely unsupervised nature of their training. Existing methods for gaining such steerability collect human labels of the relative quality of model generations and fine-tune the unsupervised LM to align with these preferences, often with reinforcement learning from human feedback (RLHF). However, RLHF is a complex and often unstable procedure, first fitting a reward model that reflects the human preferences, and then fine-tuning the large unsupervised LM using reinforcement learning to maximize this estimated reward without drifting too far from the original model. In this paper, we leverage a mapping between reward functions and optimal policies to show that this constrained reward maximization problem can be optimized exactly with a single stage of policy training, essentially solving a classification problem on the human preference data. The resulting algorithm, which we call Direct Preference Optimization (DPO), is stable, performant, and computationally lightweight, eliminating the need for fitting a reward model, sampling from the LM during fine-tuning, or performing significant hyperparameter tuning. Our experiments show that DPO can fine-tune LMs to align with human preferences as well as or better than existing methods. Notably, fine-tuning with DPO exceeds RLHF’s ability to control sentiment of generations and improves response quality in summarization and single-turn dialogue while being substantially simpler to implement and train.

 

Outstanding Datasets and Benchmarks Papers

 

In the dataset category: 

ClimSim: A large multi-scale dataset for hybrid physics-ML climate emulation

Authors:  Sungduk Yu · Walter Hannah · Liran Peng · Jerry Lin · Mohamed Aziz Bhouri · Ritwik Gupta · Björn Lütjens · Justus C. Will · Gunnar Behrens · Julius Busecke · Nora Loose · Charles Stern · Tom Beucler · Bryce Harrop · Benjamin Hillman · Andrea Jenney · Savannah L. Ferretti · Nana Liu · Animashree Anandkumar · Noah Brenowitz · Veronika Eyring · Nicholas Geneva · Pierre Gentine · Stephan Mandt · Jaideep Pathak · Akshay Subramaniam · Carl Vondrick · Rose Yu · Laure Zanna · Tian Zheng · Ryan Abernathey · Fiaz Ahmed · David Bader · Pierre Baldi · Elizabeth Barnes · Christopher Bretherton · Peter Caldwell · Wayne Chuang · Yilun Han · YU HUANG · Fernando Iglesias-Suarez · Sanket Jantre · Karthik Kashinath · Marat Khairoutdinov · Thorsten Kurth · Nicholas Lutsko · Po-Lun Ma · Griffin Mooers · J. David Neelin · David Randall · Sara Shamekh · Mark Taylor · Nathan Urban · Janni Yuval · Guang Zhang · Mike Pritchard

Poster session 4: Wed 13 Dec 5:00 p.m. — 7:00 p.m. CST, #105 

Oral: Wed 13 Dec 3:45 p.m. — 4:00 p.m. CST, Ballroom A-C (level 2)

 

Abstract: Modern climate projections lack adequate spatial and temporal resolution due to computational constraints. A consequence is inaccurate and imprecise predictions of critical processes such as storms. Hybrid methods that combine physics with machine learning (ML) have introduced a new generation of higher fidelity climate simulators that can sidestep Moore’s Law by outsourcing compute-hungry, short, high-resolution simulations to ML emulators. However, this hybrid ML-physics simulation approach requires domain-specific treatment and has been inaccessible to ML experts because of lack of training data and relevant, easy-to-use workflows. We present ClimSim, the largest-ever dataset designed for hybrid ML-physics research. It comprises multi-scale climate simulations, developed by a consortium of climate scientists and ML researchers. It consists of 5.7 billion pairs of multivariate input and output vectors that isolate the influence of locally-nested, high-resolution, high-fidelity physics on a host climate simulator’s macro-scale physical state. The dataset is global in coverage, spans multiple years at high sampling frequency, and is designed such that resulting emulators are compatible with downstream coupling into operational climate simulators. We implement a range of deterministic and stochastic regression baselines to highlight the ML challenges and their scoring. The data (https://huggingface.co/datasets/LEAP/ClimSim_high-res) and code (https://leap-stc.github.io/ClimSim) are released openly to support the development of hybrid ML-physics and high-fidelity climate simulations for the benefit of science and society.  



In the benchmark category:

DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Models

Authors: Boxin Wang · Weixin Chen · Hengzhi Pei · Chulin Xie · Mintong Kang · Chenhui Zhang · Chejian Xu · Zidi Xiong · Ritik Dutta · Rylan Schaeffer · Sang Truong · Simran Arora · Mantas Mazeika · Dan Hendrycks · Zinan Lin · Yu Cheng · Sanmi Koyejo · Dawn Song · Bo Li

Poster session 1: Tue 12 Dec 10:45 a.m. — 12:45 p.m. CST, #1618 

Oral: Tue 12 Dec 10:30 a.m. — 10:45 a.m. CST, Ballroom A-C (Level 2)

 

Abstract: Generative Pre-trained Transformer (GPT) models have exhibited exciting progress in capabilities, capturing the interest of practitioners and the public alike. Yet, while the literature on the trustworthiness of GPT models remains limited, practitioners have proposed employing capable GPT models for sensitive applications to healthcare and finance – where mistakes can be costly. To this end, this work proposes a comprehensive trustworthiness evaluation for large language models with a focus on GPT-4 and GPT-3.5, considering diverse perspectives – including toxicity, stereotype bias, adversarial robustness, out-of-distribution robustness, robustness on adversarial demonstrations, privacy, machine ethics, and fairness. Based on our evaluations, we discover previously unpublished vulnerabilities to trustworthiness threats. For instance, we find that GPT models can be easily misled to generate toxic and biased outputs and leak private information in both training data and conversation history. We also find that although GPT-4 is usually more trustworthy than GPT-3.5 on standard benchmarks, GPT-4 is more vulnerable given jailbreaking system or user prompts, potentially due to the reason that GPT-4 follows the (misleading) instructions more precisely. Our work illustrates a comprehensive trustworthiness evaluation of GPT models and sheds light on the trustworthiness gaps. Our benchmark is publicly available at https://decodingtrust.github.io/.

Test of Time

This year, following the usual practice, we chose a NeurIPS paper from 10 years ago to receive the Test of Time Award, and “Distributed Representations of Words and Phrases and their Compositionality” by Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, and Jeffrey Dean, won. 

Published at NeurIPS 2013 and cited over 40,000 times, the work introduced the seminal word embedding technique word2vec. Demonstrating the power of learning from large amounts of unstructured text, the work catalyzed progress that marked the beginning of a new era in natural language processing.

Greg Corrado and Jeffrey Dean will be giving a talk about this work and related research on Tuesday, 12 Dec at 3:05 – 3:25 pm CST in Hall F.  



Announcing NeurIPS 2023 Invited Talks

Presenting Eleven Speakers for Six Keynotes and One Panel

By Amir Globerson, Kate Saenko, Moritz Hardt, Sergey Levine and Comms Chair, Sahra Ghalebikesabi 

NeurIPS Program Chairs are delighted to announce this year’s keynote speakers and schedule – all times listed are in CST and all invited talks will be held in Hall F. There are two invited talks during each day of the main program, with one on Monday during the opening of the conference. All keynotes will be recorded and will be available to the public in late January. 

Schedule-at-a-Glance:

  1. Monday – Björn Ommer at 5:25 – 6:15 pm 11 Dec
  2. Tuesday – Lora Aroyo at 8:30 – 9:20 am  12 Dec
  3. Tuesday – Linda Smith at 2:15 – 3:05 pm 12 Dec
  4. Wednesday – Jelani Nelson at 8:30 – 9:20 am 13 Dec
  5. Wednesday – Beyond Scaling Panel at 2:15 – 3:15 pm 13 Dec
  6. Thursday – Christopher Ré at 8:30 – 9:20 am 14 Dec
  7. Thursday – Susan Murphy at 2:15-3:05 pm 14 Dec

Alexander Rush will be moderating the Wednesday afternoon panel discussion, “Beyond Scaling,” with Aakanksha Chowdhery, Angela Fan, Percy Liang and Jie Tang. For more information about who is presenting and what the different invited talks will be about, read on for session abstract details and speaker bios. 

Invited Speakers

Monday – Björn Ommer at 5:25 – 6:15 pm 11 Dec

On Monday at 5:25 pm CST, the invite talk track will kick off with the opening plenary presented by Björn Ommer who will discuss “NextGenAI: The Delusion of Scaling and the Future of Generative AI.”  

Abstract: The ultimate goal of computer vision and learning are models that can understand our (visual) world. Recently, learning such representations of our surroundings has been revolutionized by deep generative models. As this paradigm is becoming the core foundation for diverse novel approaches and practical applications it is profoundly changing the way we interact with, program, and solve problems with computers. However, most of the progress came from sizing up models – to the point where the necessary resources started to have profound detriments on future (academic) research, industry, and society.

This talk will contrast the most commonly used generative models to date and highlight the very specific limitations they have despite their enormous potential. We will then investigate mitigation strategies such as Stable Diffusion and recent follow-up work that significantly enhance the efficiency of democratizing AI. Subsequently, the talk will discuss lessons learned from this odyssey through model space, the interesting perspectives this casts on the future of generative modeling, and its implications on society.

Bio: Björn Ommer is a full professor at University of Munich where he is heading the Computer Vision & Learning Group. Before he was a full professor in the department of mathematics and computer science at Heidelberg University and a co-director of its Interdisciplinary Center for Scientific Computing. He received his diploma in computer science from University of Bonn, his PhD from ETH Zurich, and he was a postdoc at UC Berkeley.

Björn serves as an associate editor for IEEE T-PAMI. His research interests include semantic scene understanding and retrieval, generative AI and visual synthesis, self-supervised metric and representation learning, and explainable AI. Moreover, he is applying this basic research in interdisciplinary projects within neuroscience and the digital humanities. His group has published a series of generative approaches, including “VQGAN” and “Stable Diffusion”, which are now democratizing the creation of visual content and have already opened up an abundance of new directions in research, industry, the media, and beyond.

Tuesday – Lora Aroyo at 8:30 – 9:20 am  12 Dec

8:30 am CST Tuesday morning, Lora Aroyo will present “The Many Faces of Responsible AI.”  

Abstract: Conventional machine learning paradigms often rely on binary distinctions between positive and negative examples, disregarding the nuanced subjectivity that permeates real-world tasks and content. This simplistic dichotomy has served us well so far, but because it obscures the inherent diversity in human perspectives and opinions, as well as the inherent ambiguity of content and tasks, it poses limitations on model performance aligned with real-world expectations. This becomes even more critical when we study the impact and potential multifaceted risks associated with the adoption of emerging generative AI capabilities across different cultures and geographies. To address this, we argue that to achieve robust and responsible AI systems we need to shift our focus away from a single point of truth and weave in a diversity of perspectives in the data used by AI systems to ensure the trust, safety and reliability of model outputs. 

In this talk, I present a number of data-centric use cases that illustrate the inherent ambiguity of content and natural diversity of human perspectives that cause unavoidable disagreement that needs to be treated as signal and not noise. This leads to a call for action to establish culturally-aware and society-centered research on impacts of data quality and data diversity for the purposes of training and evaluating ML models and fostering responsible AI deployment in diverse sociocultural contexts.

Bio: I am a research scientist at Google Research NYC where I work on Data Excellence for AI. My team DEER (Data Excellence for Evaluating Responsibly) is part of the Responsible AI (RAI) organization. Our work is focused on developing metrics and methodologies to measure the quality of human-labeled or machine-generated data. The specific scope of this work is for gathering and evaluation of adversarial data for Safety evaluation of Generative AI systems. I received MSc in Computer Science from Sofia University, Bulgaria, and PhD from Twente University, The Netherlands.

I am currently serving as a co-chair of the steering committee for the AAAI HCOMP conference series and I am a member of the DataPerf working group at MLCommons for benchmarking data-centric AI. Check out our data-centric challenge Adversarial Nibbler supported by Kaggle, Hugging Face and MLCommons. Prior to joining Google, I was a computer science professor heading the User-Centric Data Science research group at the VU University Amsterdam. Our team invented the CrowdTruth crowdsourcing method jointly with the Watson team at IBM. This method has been applied in various domains such as digital humanities, medical and online multimedia. I also guided the human-in-the-loop strategies as a Chief Scientist at a NY-based startup Tagasauris. Some of my prior community contributions include president of the User Modeling Society, program co-chair of The Web Conference 2023, member of the ACM SIGCHI conferences board. For a list of my publications, please see my profile on Google Scholar.

Tuesday – Linda Smith at 2:15 – 3:05 pm 12 Dec

2:15 pm CST Tuesday, Linda Smith will present, “Coherence statistics, self-generated experience and why young humans are much smarter than current AI.”

Abstract: The world presents massive amounts of data for learning.  However much of that data is latent, only made manifest by physical action on a physical world. The structure of those actions and revealed data are tightly constrained by the continuity of time and space.  In this talk, I will present evidence that the statistics of infant and child daily-life experiences at multiple timescales have a natural coherence structure that yields rapid learning and innovative generalization from sparse data and one-time experiences.  A common simplifying assumption in AI is that the quantity of the data is all that matters:  if enough data is amassed and aggregated, it will contain the latent structure necessary for optimal performance.  The findings on the statistics of human ego-centric experience suggests that is assumption may be seriously off-the mark. 

Bio: Linda B. Smith, Distinguished Professor at Indiana University Bloomington, is an internationally recognized leader in cognitive science and cognitive development. Taking a complex systems perspective, she seeks to understand the interdependencies among perceptual, motor and cognitive developments during the first three years of post-natal life. Using wearable sensors, including head-mounted cameras and motion sensors, she studies how the young learner’s own behavior creates the statistical structure of the learning environments with a current focus on developmentally changing visual statistics at the scale of everyday life and their role in motor, perceptual, and language development. The work has led to novel insights extended through collaborations in artificial intelligence and education. The work also motivates her current efforts on defining and promoting a precision (or individualized) developmental science, one that determines the multiple causes and interacting factors that create children’s individual developmental pathways. Smith received her PhD from the University of Pennsylvania in 1977 and immediately joined the faculty at Indiana University. Her work has been continuously funded by the National Science Foundation and/or the National Institutes of Health since 1978. She won the David E. Rumelhart Prize for Theoretical Contributions to Cognitive Science, the American Psychological Association Award for Distinguished Scientific Contributions, the William James Fellow Award from the American Psychological Society, the Norman Anderson Lifetime Achievement Award, and the Koffka Medal. She is an elected member of the National Academy of Sciences and the American Academy of Arts and Science. 

Wednesday – Jelani Nelson at 8:30 – 9:20 am 13 Dec

8:30 am CST Wednesday morning, Jelani Nelson will present “Sketching: core tools, learning-augmentation, and adaptive robustness.”

Abstract: ‘Sketches’ of data are memory-compressed summarizations that still allow answering useful queries, and as a tool have found use in algorithm design, optimization, machine learning, and more. This talk will give an overview of some core sketching tools and how they work, including recent advances. We also discuss a couple newly active areas of research, such as augmenting sketching algorithms with learned oracles in a way that provides provably enhanced performance guarantees, and designing robust sketches that maintain correctness even in the face of adaptive adversaries.

Bio: Jelani Nelson came to Berkeley in 2019 from Harvard University. He is a professor in the Department of Electrical Engineering and Computer Sciences. He was an Alfred P. Sloan Research Fellow and received a 2017 Presidential Early Career Award for Scientists and Engineers.  “I find it fulfilling to work on algorithmic problems that are both practically relevant and simultaneously mathematically beautiful.”

Research Focus: Sketching and streaming algorithms for big data, and dimensionality-reduction techniques for high-dimensional data.

Wednesday – Beyond Scaling Panel at 2:15 – 3:15 pm 13 Dec

Join us at 2:15 pm Wednesday for the Beyond Scaling Panel moderated by Alexander Rush where a discussion with Aakanksha Chowdhery, Angela Fan, Percy Liang, and Jie Tang will take place in Hall F. 

Thursday – Christopher Ré at 8:30 – 9:20 am 14 Dec

8:30 am CST Thursday, Christopher (Chris) Ré will discuss “Systems for Foundation Models, and Foundation Models for Systems.”  

Abstract: I’m a simple creature. I fell in love with foundation models (FMs) because they radically improved data systems that I had been trying to build for a decade–and they are just awesome! This talk starts with my perspective about how FMs change the systems we build, focusing on what I call “death by a thousand cuts” problems. Roughly, these are problems in which each individual task looks easy, but the sheer variety and breadth of tasks make them hard.

The bulk of the talk is about understanding how to efficiently build foundation models. We describe trends in hardware accelerators from a perhaps unexpected viewpoint: database systems research. Databases have worried about optimizing IO – reads and writes within the memory hierarchy – since the 80s. In fact, optimizing IO led to Flash Attention for Transformers.

But are there more efficient architectures for foundation models than the Transformer? Maybe! I’ll describe a new class of architectures based on classical signal processing, exemplified by S4. These new architectures: are asymptotically more efficient than Transformers for long sequences, have achieved state-of-the-art quality on benchmarks like long range arena, and have been applied to images, text, DNA, audio, video. S4 will allow us to make mathematically precise connections to RNNs and CNNs. I’ll also describe new twists, such as, long filters, data-dependent convolutions, and gating, that power many of these amazing recent architectures including RWKV, S5, Mega, Hyena, and RetNet, and recent work to understand their fundamental limitations to hopefully make even more awesome foundation models! A github containing material from is under construction at https://github.com/HazyResearch/aisys-building-blocks. Please feel free to add to it!

Bio: Christopher (Chris) Re is an associate professor in the Department of Computer Science at Stanford University. He is in the Stanford AI Lab and is affiliated with the Machine Learning Group and the Center for Research on Foundation Models.  His recent work is to understand how software and hardware systems will change because of machine learning along with a continuing, petulant drive to work on math problems. Research from his group has been incorporated into scientific and humanitarian efforts, such as the fight against human trafficking, along with products from technology and companies including Apple, Google, YouTube, and more. He has also cofounded companies, including Snorkel, SambaNova, and Together, and a venture firm, called Factory.

His family still brags that he received the MacArthur Foundation Fellowship, but his closest friends are confident that it was a mistake. His research contributions have spanned database theory, database systems, and machine learning, and his work has won best paper at a premier venue in each area, respectively, at PODS 2012, SIGMOD 2014, and ICML 2016. Due to great collaborators, he received the NeurIPS 2020 test-of-time award and the PODS 2022 test-of-time award. Due to great students, he received best paper at MIDL 2022, best paper runner up at ICLR22 and ICML22, and best student-paper runner up at UAI22.

Thursday – Susan Murphy at 2:15-3:05 pm 14 Dec

2:15 pm Thursday, Susan Murphy will discuss “Online Reinforcement Learning in Digital Health Interventions.”

Abstract: In this talk I will discuss first solutions to some of the challenges we face in developing online RL algorithms for use in digital health interventions targeting patients struggling with health problems such as substance misuse, hypertension and bone marrow transplantation. Digital health raises a number of challenges to the RL community including different sets of actions, each set intended to impact patients over a different time scale;  the need to learn both within an implementation and between implementations of the RL algorithm; noisy environments and a lack of mechanistic models.    In all of these settings the online line algorithm must be stable and autonomous.   Despite these challenges, RL, with careful initialization, with careful management of bias/variance tradeoff and by close collaboration with health scientists can be successful.   We can make an impact!

Bio: Susan A. Murphy is Mallinckrodt Professor of Statistics and of Computer Science and Associate Faculty at the Kempner Institute, Harvard University.  Her research focuses on improving sequential decision making in health, currently, online real-time learning algorithms for personalizing digital health interventions.  She is a member of the US National Academy of Sciences and of the US National Academy of Medicine.  In 2013 she was awarded a MacArthur Fellowship for her work on experimental designs to inform sequential decision making.  She is a Fellow of the College on Problems in Drug Dependence, Past-President of the Institute of Mathematical Statistics, and a former editor of the Annals of Statistics.

Reflections on the NeurIPS 2023 Ethics Review Process

By the NeurIPS 2023 Ethics Review Co-Chairs

The NeurIPS 2023 Ethic Review process began with the publication of the Code of Ethics. This step formally codified a foundation for ethics within the conference framework.

The Ethics Guidelines for Reviewers were also updated to reflect incorporation of the Code of Ethics. 

This year, 502 papers (3.77% of all submissions) were flagged for ethics review. This represents an increase in the number of papers flagged last year (474) but a decrease in the overall rate of papers being flagged (from 4.37% in 2022).

In particular, we are heartened that fewer submissions in the Datasets and Benchmarks track were flagged for ethics review, despite the number of submissions more than doubling since last year. We believe that these trends mean that the ethics of ML research is being taken more seriously by the research community.

20232022
Paper statistics (preliminary)Main trackD&B trackTotalMain trackD&B trackTotal
Papers submitted12,34597613,32110,41144710,858
Total papers flagged for ethics review395107502362112474
Flag rate3.20%10.96%3.77%3.48%25.06%4.37%

This year, reviewers in the main track were required to flag at least one specific area for ethics review. Note that papers can be flagged for multiple areas, so the percentages below do not add up to 100%.

Notably, discrimination, bias and fairness concerns have been flagged at nearly triple the rate this year compared to last year; the rate of papers flagged for legal compliance or responsible research practice also doubled.  

Papers flagged by area (main track only)20232022
Discrimination / bias / fairness concerns12233.7%4713.0%
Privacy and security9125.1%6317.4%
Legal compliance3910.8%174.7%
Inadequate data and algorithm evaluation8322.9%359.7%
Responsible research practice6116.9%308.3%
Inappropriate potential applications and Impact7621.0%4713.0%
Research integrity issues6317.4%5615.5%
Failure to comply with NeurIPS Code of Ethics5314.6%New in 2023
I don’t knowDeprecated3810.5%
No area flaggedDeprecated8022.1%
Total395362

This year, we invited everyone contacted for ethics reviews from last year to serve again as ethics reviewers, and also issued an open call for new reviewers. A total of 396 ethics reviewers answered the call.

In addition, we also asked reviewers if they were available as emergency ethics reviewers, to provide missing reviews and reviews for papers flagged for review on short notice. 76 people volunteered as emergency ethics reviewers. 

We are immensely grateful to everyone who answered the call to service, which allowed us to handle the increased workload for ethics reviews, while broadening the community and reducing the average number of reviews per reviewer.

Thank you all for a wonderful and successful ethical review process.

Yours truly,

Jiahao Chen, Lester Mackey, and Cherie Poland
NeurIPS 2023 Ethics Review Co-Chairs

NeurIPS Newsletter – November 2023

,

Welcome to the November edition of the monthly NeurIPS Newsletter!

The NeurIPS Newsletter aims to provide an easy way to keep up to date with NeurIPS events and planning progress, respond to requests for feedback and participation, and find information about new initiatives. Notably, this newsletter will focus on NeurIPS 2023, which will return to New Orleans from Sunday, December 10th – Saturday, December 16th, 2023.

You are receiving this newsletter as per your subscription preferences in your NeurIPS profile. As you prepare to attend NeurIPS, we hope that you will find the following information valuable. To remove yourself from receiving the NeurIPS newsletter, unselect the “Subscribe to Newsletter” checkbox in your profile:https://neurips.cc/Profile/subscribe. For more information on email preferences, visit: https://neurips.cc/FAQ/EmailPreferences

Topics this month include

  1. Poster printing
  2. Updates from our Diversity, Inclusion, and Accessibility Chairs
  3. Updates from our Tutorial Chairs
  4. Updates from our Social Chairs
  5. Updates from our Affinity Workshop Chairs
  6. Updates from our Workshop Chairs
  7. And the announcement of our collaboration with AIhub
  8. Programme Agenda

1) From our Organizers: Poster Printing

You can use any service you want to print your poster. The NeurIPS service deadline was 9 Nov, but you can print your poster and bring it to New Orleans or use the FedEx service provided at the Hilton New Orleans Riverside before the 2 Dec order deadline. Read NeurIPS poster printing information page for additional insight and information about templates, virtual poster and paper thumbnails, poster pickup instructions, poster sizes and printing.

2) From our Diversity, Inclusion, and Accessibility Chairs

Expanding the NeurIPS invitation letter

Visas continue to be a barrier to in-person attendance this year. We have expanded the visa guidance page (https://neurips.cc/Conferences/2023/Visa) and now provide more details (such as attendance at a prior year’s NeurIPS) in the automatically generated invitation letter (available after registration).

The invitation letter is an important piece of evidence in a visa application to attend NeurIPS in person. Building on our experience writing more detailed invitation letters in targeted cases (important details missing from the basic letter; expedited appointment requests), we have released a tool for registered attendees to request that the DIA Chairs augment their NeurIPS invitation letter; please see https://neurips.cc/visa/assistance for more information on who is eligible and what details we can include. In the first week that the tool was available, the DIA Chairs received and processed 19 requests, with at least 1 successful case of getting an expedited appointment so far!

Survey on barriers to in-person participation

The DIA Chairs have received many emails about visa refusals and extended wait times for visa interview appointments. Inspired by ACL’s visa survey(https://x.com/nedjmaou/status/1676994799490957317), we are launching a survey on barriers to in-person participation at NeurIPS to document and inform potential solutions to the difficulties of prospective in-person attendees. If you have faced difficulties attending NeurIPS in person (visa-related or otherwise), please complete the survey at https://forms.gle/oNFPfFxeZnLZfDoWA.

3) From our Tutorial Chairs

Conference tutorials will be held on Monday, Dec 11. Check out the full schedule here.

4) From our Socials Chairs

NeurIPS Socials are an excellent opportunity for all the members of our community to meet up, discuss, collaborate, debate, or celebrate around a common interest.

The Socials are scheduled at the end of the day on December 12th, 13th and 15th. Check out the full schedule here.

5) From our Affinity Workshop Chairs

As we approach the final weeks leading up to NeurIPS 2023, nine affinity groups are actively organizing workshops and socials, dedicated to creating inclusive spaces for marginalized communities at the conference. Most of these groups are progressively updating their pages on the NeurIPS website with details about events and papers. While most workshop acceptance notifications have closed, preparations are underway for visa invitation letters and travel arrangements. For further clarification or guidance, please check the respective group websites.

To showcase these valuable contributions, we are coordinating an in-person and virtual joint affinity poster session, emphasizing cross-learning among affinity groups. In addition to the workshops and poster session, many affinity groups will host receptions and socials at the venue or local restaurants/bars (more details coming soon). Collaborating with NeurIPS staff and DIA chairs, we are committed to providing childcare/lactation/quiet/work rooms and compiling FAQs about accessibility and gender inclusivity. Register for childcare before Nov 30 here https://neurips.cc/Conferences/2023/ChildCare.

This year NeurIPS is offering to stream affinity group events for free. No virtual pass will be needed to watch the stream, but note that some affinity groups might decide against streaming their event.

6) From our Workshops Chairs

With NeurIPS 2023 just around the corner, we couldn’t resist sharing a sneak peek. We’ve rounded up a few key highlights from a variety of workshops to offer you a glimpse into the upcoming program. Thank you to the workshop organizers who shared their highlight reels and exciting topics to be explored, summarized below:

7) Our new collaboration with AIhub

Would you like to learn how to communicate your AI research to a general audience? AIhub staff and trustees are giving a tutorial at NeurIPS on science communication for AI researchers. You will learn how to turn your research articles into blog posts, how to use social media to promote your work, and how to avoid hype when writing about your research. The session will take place on Monday 11 December (room 235/236) and will comprise a talk from 12:45 – 13:45, followed by an informal, open drop-in session from 14:00 – 16:00 for one-on-one support with your sci-comm questions, ideas and stories. You can find out more at  https://aihub.org/sci-comm-intro-neurips-2023/.

8) Programme agenda

The NeurIPS 2023 schedule is live. To plan your days, check out the agenda here.

Thanks and looking forward to seeing you at the conference,

Alice Oh & Tristan Naumann

NeurIPS 2023 General Chairs

NeurIPS Newsletter – October 2023

Welcome to the October edition of the monthly NeurIPS Newsletter! This newsletter was sent out by email to all subscribers on October 26, 2023, but we reposting on our blog as a monthly digest. To receive this newsletter in your email inbox directly, subscribe at: https://neurips.cc/Profile/subscribe. For more information on email preferences, visit: https://neurips.cc/FAQ/EmailPreferences

The NeurIPS Newsletter aims to provide an easy way to keep up to date with NeurIPS events and planning progress, respond to requests for feedback and participation, and find information about new initiatives. Notably, this newsletter will focus on NeurIPS 2023, which will return to New Orleans from Sunday, December 10th – Saturday, December 16th, 2023.

Newsletter includes:

  1. Keynote Speakers
  2. Journal Track
  3. Competition Track
  4. Workshops
  5. Affinity Groups
  6. Accepted Papers
  7. Town Hall
  8. Financial Aid
  9. Childcare and Other Amenities

1. Keynote Speakers

We are delighted to announce this year’s keynote speakers: 

  • Björn Ommer,  Mon 11 Dec
  • Lora Aroyo, Tue 12 Dec
  • Linda Smith, Tue 12 Dec 
  • Jelani Nelson, Wed 13 Dec 
  • LLM Panel (Sasha Rush, Angela Fan, Aakanksha Chowdhery, Jie Tang, Percy Liang), Wed 13 Dec
  • Christopher Ré, Thu 14 Dec
  • Susan Murphy, Thu 14 Dec 

More information about the seven invited talks will be coming soon.

2. Journal Track

This year we have a total of 45 papers for the Journal track. Out of these, 21 papers have been sourced from the Journal of Machine Learning Research (JMLR), while the remaining 24 papers are contributions from the ML Reproducibility Challenge (MLRC) 2022. We eagerly await your presence at the poster sessions.

3. Competitions Track

NeurIPS Competitions promote innovative research to solve open problems in Machine Learning and foster collaboration across different scientific disciplines by providing an opportunity to showcase research and compete with other leading researchers. The Competition track workshops will be held on Friday Dec 15th and Saturday Dec 16th 2023. Some of the sessions will be held in a hybrid format, while others will be purely virtual.  Be sure to check them out to see what approaches rise to the top of the leaderboard!

Check out the draft program currently up here: https://neurips.cc/virtual/2023/events/Competition. 

The Competition Track chairs want to thank everyone who organized and participated in this year’s competitions. 

4. Workshops

Each of our 58 workshops is associated with its own dedicated website, and is linked to the NeurIPS 2023 virtual website https://neurips.cc/virtual/2023/events/workshop. We thank all authors for contributing workshop submissions, and all organizers and reviewers for their hard work. Workshops are now entering the workshop submission reviewing stage. The decisions are expected to be announced before October 27, 2023 AoE. Stay tuned for updates!

While the workshop organizers are super busy preparing top-notch program agendas, we invite attendees and participants to read our blog post to understand more about the workshop preparation process: https://blog.neurips.cc/2023/09/12/your-neurips-workshop-was-accepted-now-what/

In 2023, we have strongly encouraged all NeurIPS workshops to prioritize having in-person speakers (except for exceptional circumstances), while still welcoming and facilitating both in-person and virtual attendance. As in previous years, virtual attendees will be able to watch the livestreams on the NeurIPS virtual website. We invite you to join our stellar workshops – many of which include outstanding speakers and roundtable discussions as well as mentoring and networking opportunities – regardless of whether you plan to attend in-person or virtually!

5. Affinity Groups

Nine affinity groups are organizing workshops/socials at NeurIPS 2023 on Monday, and are working hard to create spaces for marginalized communities at the conference. They are progressively populating their pages on the NeurIPS schedule with events and papers. Most workshops’ call for contributions have closed or are closing soon; please check their websites or get in contact with any relevant groups if you would still like to submit your work. To showcase these contributions, we are organizing an in-person and a virtual joint affinity poster session, which will emphasize learning from and across affinity groups. Beyond the workshops and poster session, many affinity groups are hosting receptions and socials at the venue or local restaurants/bars (more details coming soon). We are working with the NeurIPS staff and DIA chairs to provide childcare/lactation/quiet/work rooms and compile answers to FAQs about accessibility and gender inclusivity.

6. Accepted Papers 

The list of 3586 accepted papers and poster schedule are now available on the website: https://neurips.cc/virtual/2023/papers.html?filter=titles. 

These include papers from both the main track and the datasets and benchmarks track. At the conference there will be daily poster sessions (one morning and one afternoon session), as well as oral presentations. The schedule is available at https://neurips.cc/virtual/2023/calendar. 

For authors of accepted papers, please note that the SlidesLive video deadline is Nov 13.

7. Town Hall

On Thursday during NeurIPS the Communications Chairs will host the annual Town Hall between 7:00-9:00 p.m. as an opportunity to connect with and ask questions to the NeurIPS organizers and the Board. If you have questions for this year’s organizers please submit them to townhall@neurips.cc  so we can discuss these publicly during the conference in New Orleans. 

8. Financial Aid

Financial aid applications have closed. The NeurIPS DIA Chairs and Logistics Team have begun sending out rolling notifications. For the latest info about financial aid, visit this updating page: https://neurips.cc/Conferences/2023/FinancialAssistance

9. Child Care

NeurIPS is proud to provide free on-site child care. Please register before the November 30 deadline and familiarize yourself with the cancelation policy to avoid no-show fees. For details and how to register visit: https://neurips.cc/Conferences/2023/ChildCare

Other amenities will include a prayer room and quiet rooms.

Thanks and looking forward to seeing you at the conference,

Alice Oh & Tristan Naumann

NeurIPS 2023 General Chairs

Introducing the NeurIPS 2023 Tutorials

by Saadia Gabriel, Andrew Gordon Wilson, Marzyeh Ghassemi 

We are excited to announce the tutorials selected for presentation at the NeurIPS 2023 conference! We look forward to an engaging program, spanning many exciting topics, including human-AI collaboration, diffusion models, automated theorem proving, efficient deployment of large language models, AI governance for accountability, and others. In this blog post, we detail the upcoming program and our selection process. 

Tutorial Program

There will be 14 tutorials this year. To encourage active participation and facilitate community-building, we restricted tutorials to an in-person format. Another important feature of this year’s tutorials is the inclusion of a panel discussion. These panels open up the conversation to allow for diverse perspectives on the tutorial topic and lively exchange of ideas. The 2023 tutorials are:

Machine Learning for Theorem Proving
Speakers: Emily First, Albert Q. Jiang, Kaiyu Yang

Governance & Accountability for ML: Existing Tools, Ongoing Efforts, & Future Directions
Speakers: Emily Black, Hoda Heidari, Dan Ho 

Application Development using Large Language Models
Speakers: Andrew Ng, Isa Fulford 

Data-Centric AI for reliable and responsible AI: from theory to practice
Speakers: Mihaela van der Schaar, Isabelle Guyon, Nabeel Seedat

Do You Prefer Learning with Preferences?
Speakers: Aditya Gopalan, Aadirupa Saha

How to Work With Real Humans in Human-AI Systems
Speakers: Elizabeth Bondi-Kelly, Krishnamurthy (Dj) Dvijotham, Matthew E. Taylor

Language Models Meet World Models
Speakers: Zhiting Hu, Tianmin Shu

Exploring and Exploiting Data Heterogeneity for Prediction and Decision-Making
Speakers: Peng Cui, Hongseok Namkoong, Jiashuo Liu, Tiffany Cai

Recent and Upcoming Developments in Randomized Numerical Linear Algebra for ML
Speakers: Michał Dereziński, Michael Mahoney

Reconsidering Overfitting in the Age of Overparameterized Models
Speakers: Spencer Frei, Vidya Muthukumar, Fanny Yang

Contributing to an Efficient and Democratized Large Model Era
Speakers: James Demmel, Yang You

Latent Diffusion Models: Is the Generative AI Revolution Happening in Latent Space?
Speakers: Karsten Kreis, Ruiqi Gao, Arash Vahdat

What can we do about NeurIPS Reviewer #2? Challenges, Solutions, Experiments and Open Problems in Peer Review
Speaker: Nihar Shah 

Data Contribution Estimation for Machine Learning
Speakers: Stephanie Schoch, Ruoxi Jia, Yangfeng Ji

Selection Process

We received 54 proposal submissions this year. Each submission was reviewed by the Tutorial chairs, with the chair reviewing assignments based on expertise and avoiding conflicts of interest. Each chair gave a score between 1 (strong reject) to 10 (strong accept) to encapsulate their overall impression of the proposal. We then shortlisted the submissions that had received an average score of 5 or higher and discussed when there were disagreements between the initial reviews. A third review was then obtained from a different Tutorial Chair to finalize the decision to accept or reject a proposal. We accepted 14 proposals with this process.

Some relatively common reasons for low scores included (but were not limited to):

  • The topic was too niche for a very broad audience.
  • There were stronger submissions covering the same topic. 
  • Tutorial was not focused enough on core skills, or was formulated more as a workshop or talk.
  • The tutorial was too focused on the work of a particular speaker or panelist. 
  • Low diversity, especially gender diversity.
  • Guidelines were not followed (e.g. no panel included).

In the call for proposals, we emphasized the need for diversity in the teaching teams and panels. This was met with an overwhelming positive response. We look forward to tutorials that are representative of many NeurIPS community members. 

Stay tuned for a post-conference retrospective blog post with reflections on the tutorials and thoughts for next year. You can share your tutorial experiences for a potential feature in the blog here. 

NeurIPS Newsletter – September 2023

,

NeurIPS Logo

Welcome to the September edition of the monthly NeurIPS Newsletter! This newsletter was sent out by email to all subscribers on September 20, 2023, but we reposting on our blog as a monthly digest. To receive this newsletter in your email inbox directly, subscribe at: https://neurips.cc/Profile/subscribe. For more information on email preferences, visit: https://neurips.cc/FAQ/EmailPreferences

The NeurIPS Newsletter aims to provide an easy way to keep up to date with NeurIPS events and planning progress, respond to requests for feedback and participation, and find information about new initiatives. Notably, this newsletter will focus on NeurIPS 2023, which will return to New Orleans from Sunday, December 10th – Saturday, December 16th, 2023.

Newsletter includes:

  1. Registration and Visa Guidance
  2. Upcoming Calls for Participation
  3. Event Week – Schedule at a Glance
  4. Program and Dataset & Benchmarks Updates
  5. Affinity Workshops
  6. Tutorials
  7. Workshops
  8. Competitions
  9. Ethics Review
  10. Outreach
  11. Call to Register for Childcare
  12. Support to Attend NeurIPS

1. Registration & Visa Guidance

Registration is open! https://neurips.cc/Register/

This year NeurIPS opened registration April 11th to provide early access to visa materials (https://neurips.cc/Conferences/2023/Visa) in section five of the registration process. The Cancellation Policy (https://neurips.cc/FAQ/CancellationPolicy) contains considerations for visa denials, and the conference will grant a full refund before November 14th, 2023.

NeurIPS recognizes that this step may not solve all visa-related issues, but we hope it makes the process smoother for a significant number of attendees.

2. Upcoming Calls for Participation

NeurIPS is welcoming a second round of proposals for socials, due October 18th 2023 AoE (Anywhere on Earth). Socials are events focused on the social aspects of the conference, rather than the technical ones, creating space to meet up, discuss, or celebrate around a common interest. For more information on how to submit a proposal, read: https://neurips.cc/Conferences/2023/CallforSocials

NeurIPS is soliciting applications for the EXPO from Diamond & Platinum exhibitors. The NeurIPS EXPO is a one day (Sunday December 10th 2023), multitrack, interdisciplinary section of the conference focused on ideas being developed in an industrial context. Deadline for expos is October 13th 2023 AoE. For more details, read: https://neurips.cc/Expo/CallForTalksPanels

3. Event Week – Schedule at a Glance

Curious how the conference week will look? See the daily events:

  • Sunday December 10th
  • NeurIPS Expo 
  • Monday December 11th
  • Affinity Workshops
  • Tutorials 
  • Opening Remarks, First Keynote, and Reception
  • Tuesday December 12th and Wednesday December 13th
  • Main Conference: Keynotes, Posters, and Oral Sessions
  • Socials (Evening)
  • Thursday December 14th
  • Main Conference: Keynotes, Posters, and Oral Sessions
  • Townhall 
  • Closing Reception
  • Friday December 15th
  • Workshops
  • Socials (Evening)
  • Saturday December 16th
  • Workshops

4. Program and Datasets & Benchmarks Updates

Reviews are underway and nearing an end! Poster, Oral and Spotlight decisions will be announced for both the Main Program and Datasets & Benchmarks tracks September 21st at 9:00 PM EDT. Reviewing at scale presents many challenges. This year, we received 13,330 submissions, coordinated over 13,500 reviewers, 1,100 Area Chairs, and 100 Senior Area Chairs across both tracks. On behalf of the Program Chairs and Datasets & Benchmarks Chairs we thank our reviewers, ACs, and SACs for making this possible.

5. Affinity Workshops

At NeurIPS, affinity groups play a crucial role in advocating for and amplifying the ideas and voices of marginalized communities. This year, we’re excited to host eight Affinity Workshops on Monday. Many Affinity Workshops have released calls for contributions, and they welcome your work:

Read our blog post “Announcing the NeurIPS 2023 Affinity Workshops” for details: https://blog.neurips.cc/2023/07/18/announcing-the-neurips-2023-affinity-workshops/

6. Tutorials

Tutorials are self-contained events covering emerging ML & AI topics, led by a small number of speakers with panels to broaden discussion. This year, we’re excited to announce a total of 14 tutorials – stay tuned for an upcoming blog post next month with the full list of accepted tutorials. 

7. Workshops

NeurIPS Workshops provide a structured means of bringing together attendees with shared interests to form communities. This year, we’re excited to host 58 workshops that will take place on December 15th & 16th. Many of these workshops will have a submission deadline near September 29th, 2023 AoE (Anywhere on Earth), so mark your calendar!

For planning and more details, including the list of accepted workshops, see: https://neurips.cc/virtual/2023/events/workshop 

In addition, read our blog post, “Announcing the NeurIPS 2023 Workshops”:
https://blog.neurips.cc/2023/07/13/announcing-the-neurips-2023-workshops.

Lastly, new this year, we are releasing our guidance on how to organize an accepted workshop, as a means to improve transparency about the process. Read our blog post:
Your NeurIPS Workshop was Accepted – Now What? – NeurIPS Blog

8. Competitions

NeurIPS Competitions promote innovative research and foster collaboration across different scientific disciplines by providing an opportunity to showcase research and compete with other leading researchers. We’re excited to host 20 competitions in this year’s Competition Track on Friday Dec 15th and Saturday Dec 16th 2023.

Read more about this year’s competitions and how you can participate in this blog announcement, “Announcing the NeurIPS 2023 Competitions”: https://blog.neurips.cc/2023/06/14/announcing-the-neurips-2023-competitions/

9. Ethics Review

This year saw 502 papers (3.77% of all submissions) flagged for ethics review, up in absolute number from last year (474), but down in relative terms (4.37%). We thank our 396 ethics reviewers and especially the 76 who also volunteered as emergency reviewers. Please watch for our detailed report on the ethics review process, coming soon!

10. Outreach

At NeurIPS, we’re always looking for ways to grow our community, with focus on demographics underrepresented in ML/AI. This year in NeurIPS Outreach, we are planning to invite undergraduate and graduate students from local colleges and universities including HBCUs. Stay tuned to hear more about the details in the coming newsletters!

11. Call to Register for Childcare

NeurIPS is proud to be providing free on-site child care at the conference this year. Registration deadline for child care is November 30th 2023, and is on a first-come first-serve basis. For more information on how to register, please read:
https://neurips.cc/Conferences/2023/ChildCare

12. Support to Attend NeurIPS

Applications for financial assistance and volunteering will be open soon after the conference paper decision notification date. We are aiming for a rapid turnaround, especially on applications from authors of accepted conference papers, to give attendees sufficient time to organize travel. Keep an eye on the conference homepage for further updates this month!

Thanks and looking forward to seeing you at the conference,

Alice Oh & Tristan Naumann

NeurIPS 2023 General Chairs

Introducing the College Outreach Program at NeurIPS 2023

by Jihun Hamm, Edward Choi

Summary: NeurIPS is hosting a college outreach program in New Orleans on Monday, Dec. 11th. Our vision is to empower students to engage with AI and learn how it can impact their lives.

Hello! We’re Jihun and Ed, the outreach co-chairs for NeurIPS 2023.  We’d like to introduce our plan for the college outreach program at this year’s conference! 

Last year, we piloted a high school outreach program, attracting widespread interest with over 240 students from 12 high schools in New Orleans attending NeurIPS. The scale of engagement and impact from this program reaffirms how important it is to give junior researchers an opportunity to participate in machine learning and artificial intelligence. 

This year, to ensure that we are giving opportunities to students at various stages of education, we are focusing our outreach opportunities on college students, who will be given unrestricted access to the full NeurIPS program, allowing them to choose sessions based upon their own preferences and pacing. 

We are working with local colleges in Louisiana and NOLA in particular, with a focus on HBCU students and with a goal of contributing back to the local community. With the partnership of these local institutions, we’re organizing an outreach event on Dec. 11th to introduce students to NeurIPS and the world of AI. Students participating in our outreach program will get to participate in a special outreach session with a keynote and lunch, receive recommendations for sessions, and be given full access to the entire conference! 

Our Vision

NeurIPS plans to invite students from local colleges to visit the conference. Since we understand navigating a giant conference for the first time can be overwhelming, our aim is to provide special sessions and some schedule recommendations for students, while still giving them flexibility and freedom to dictate what main conference sessions they prefer. Students will benefit from:

  • An outreach session with a keynote and lunch to get introduced to NeurIPS
  • Invitations to affinity workshops and the dinner reception to socialize with the community
  • Access to tutorials and the main conference program to engage with AI experts from around the world

Throughout our planning process, we will be partnering with members of the New Orleans community. We recognize that we don’t have all the answers. Our job as co-chairs is to create a program that benefits students first – and that involves bringing educators, community leaders, and students to the table. 

What’s Happening Next

As co-chairs, we’re currently working on two responsibilities

  1. Gathering feedback from the New Orleans community and iterating on event specifics
  2. Working with our logistics partners on food and transportation.

Expect another update from us with more specifics soon.

We’re excited to run this iteration of NeurIPS outreach. If you have any suggestions, questions, or feedback, please let us know! 

Jihun Hamm, Edward Choi

Outreach Chairs, NeurIPS 2023

Email: outreach-chairs@neurips.cc

Appendix: more about us!

Jihun here! I’m an associate professor at Tulane University located in the heart of New Orleans, one of the most unique cultural and historical cities in the US.  My main research interests are robust ML, optimization and medical applications of ML. I have taught ML and Deep Learning classes in and outside of computer science. I am glad to have the opportunity to invite Louisiana college students to the outreach program in my hometown NOLA.  

Ed here! I’m an assistant professor at KAIST, South Korea. My main research area is AI for healthcare, specifically interested in machines that can understand medical data. I have been teaching an AI programming course for several years, where I had the pleasure to engage with the newcomers of the AI and help them acquire the programming skill to build their own AI models. I’m excited to serve as an outreach chair this year to interact with students of Louisiana!

This year’s outreach program would not have been possible without helpful input from Matt Wang and Jessica Forde, who were last year’s outreach chairs!

Your NeurIPS Workshop was Accepted – Now What?

,

By Rowan McAllister and Yarin Gal

This post offers guidance for the workshop organization process post-workshop-acceptance: 

– how to organize a successful workshop

– some pitfalls to be aware of

A version of this post was communicated with this year’s NeurIPS workshop organizers, but we are sharing this post more widely in order to increase transparency about the typical workshop organization process; and to encourage diversity within workshop organizers’ teams, specifically to encourage junior organizer’s participation. 

Timeline

So your workshop was accepted, congratulations on putting together a successful proposal! Your first task is to meet with all your workshop co-organizers. The goal of the meeting is to finalize co-organizer role assignments and synchronize everyone’s consensus on the important dates in your workshop’s timeline. 

Further, consider establishing regular meetings with your co-organizers; either fortnightly, monthly, or prior to events like sending speaker invitations and starting the review phase, keeping co-organizers involved. Without follow up meetings, the lead organizer might get stuck doing most of the work, unbeknownst to the other organizers. 

Milestones

Once accepted, the workshop organizers will verify that your workshop is making progress towards basic milestones. Dates for future NeurIPS workshops will need adjusting. Here are some example milestones and dates for the NeurIPS 2023 organization process. 

– August 15th: Website URL active, including:

  – Submission link,

  – Easily-found contact info, ideally top or bottom of the landing page

  – URL added to the NeurIPS site

– October 27th: Mandatory accept/reject notification

– November 3rd: Import accepted paper list to NeurIPS 

– November 10th: Author deadline for Poster Printing Service

– November 20th: Schedule inputted into Schedule Editor on NeurIPS site (schedule between 8:15am and 5:30pm, with the schedule editor link that can be seen after an organizer logs in to the neurips.cc site).

– November 29th: Mandatory Slideslive upload deadline, if deciding to do any pre-recording

Dates for you to decide

– call for papers date (we recommend as soon as possible following workshop acceptance)

– submission due date (we recommend September 29th for NeurIPS 2023)

– review dates

– emergency review dates

– decision notification date (no later than October 27th for NeurIPS 2023)

– camera ready due date

We recommend all deadlines be scheduled at 23:59 (11:59PM) “Anywhere on Earth” time zone (AoE) to reduce confusion. You may also wish to email your speakers, confirming that their workshop is accepted.

Call for papers:

Send out a call for papers as soon as you’ve updated your website with your workshop’s timeline, paper format, and submission portal. Post about your workshop ideally sometime Tuesday–Thursday so more people notice it, and then follow up periodically with a few more online reminder posts in the lead up to the deadline (in case people miss the first one).

Submission date:

The suggested submission date for workshop contributions is September 29th for NeurIPS 2023, but this is really up to you. Late submission deadlines help attract more papers and more recent papers. Early submission deadlines allow more time for reviewing; and/or also allows you to pull your notification date earlier too, to give authors more time to organize their visas and plan for their trips.

Whatever you choose, consider setting your submission deadline to be after the conference’s author notification date September 22nd to attract conference rejections too (which hopefully improve when submitted to your workshop). Avoid setting deadlines that fall on a weekend or common holiday.

Notification date:

Make sure to set your notification date so authors have enough time to get visas. The latest possible date to notify authors of your accept-or-reject decision of their paper is October 27th for NeurIPS 2023. However, consider notifying authors of your decision before the Early Registration Deadline October 21st to reduce your authors’ registration costs. Soon after the notification date you’ll also want to decide on contributed or spotlight talks (if any) so authors have time to prepare.

Outreach:

Schedule a set of announcements and reminders about your workshop using social media to remind people about your workshop beyond a one-and-done call for papers that people might forget about. Consider assigning the role of social outreach to one of your co-organizers so you know it will get done reliably.

Speakers

Keep Speakers Updated

It’s important to keep speakers updated with relevant developments (like when your workshop is selected, and when the conference selects which date your workshop will be). Writing update emails to speakers can be a good excuse to remind them of your call for papers (if it’s been called yet) and invite them or their lab to submit works too. 

Speaker Cancellations

Speaker cancellations are normal. When faced with a cancellation, you can either replace the speaker with another speaker or not. Opting for fewer speakers means you can expand the interactive elements of your schedule; like posters (ideally 2+ hours), panel discussions (45+ mins), or each speaker’s question time (10 mins). So if you have an excessive number of speakers left (9+), consider using the time for more attendee interaction.

Papers

Call for Papers

There are multiple ways to call on the community to submit papers to your workshop:

– Social Media

  – Adding Images with social media posts will make them more noticeable. You could use a graphic that represents your workshop’s theme or an image of your confirmed speakers with the session title.

  – Tag your speakers and co-organizers, they often want to share and promote it.

  – Tag NeurIPS in your call for papers post using the hashtag #NeurIPS2023 or handle @NeurIPSConf.

– Mailing lists

– Institutions: Share with your lab, university department or company newsletter and invite participation.

– Individuals you know would be interested.

Paper Format

Consider copying the NeurIPS style files, editing slightly with a footer that says it’s your workshop. Please be clear to your authors about what you expect, such as additionally stating “we welcome papers up to 9 pages (max) not including references or appendix, as a single PDF”.

Extended abstracts: 

Whatever format you choose, consider allowing 4-page submissions (“extended abstracts”). This is because some computer vision conferences consider peer-reviewed workshop papers exceeding 4 pages as prior publications. Extended abstracts (4 pages or less) therefore allow authors to submit preliminary results to your workshop before a conference submission without later violating a dual/double submission policy, which are the types of papers you want: exciting up-and-coming ideas that might not even be on arXiv yet.

Submissions

Archival or non-archival?

An archival or formally published workshop proceedings often precludes authors from submitting an extended version of the same work to other venues such as a conference or journal. Some authors therefore ask about this, so organizers usually specify on the website that “no submission will be indexed nor have archival proceedings”. In some situations, you may invite submission for both proceedings and non-proceedings. For example, full-length papers go to proceedings, extended abstracts go to non-proceedings. Note that NeurIPS will not itself publish proceedings for workshops: workshops will need to set up their own proceedings if desired.

Reviewing

Reviewing submitted papers is optional. Some workshops provide reviews since feedback is helpful to authors, but not all workshops do. Some workshops prefer lightweight reviewing: largely checking for fit and correctness and a selection of stronger papers for longer presentations. Such lightweight reviewing is more lenient, only rejecting poor or off-topic submissions, corresponding to a very high acceptance rate. Detailed reviews are more valuable to authors but place more burden on reviewers and can be less practical for larger workshops to provide reviews of consistent quality. If you wish to provide reviews, consider recruiting a program committee (aka reviewers) if you expect 15+ papers, otherwise the organizers can review the papers themselves. The organizers can choose between single blind and double blind reviews considering additional workload and fairness.

Program Committee

Be courteous:

To invite people to help review, people are much more likely to respond to a personalized invite from a human rather than an automated email from your submission portal. So consider reaching out to people first to ask, and add them into a reviewing system once they accept and are expecting automated emails. When writing the email, explain what you’re trying to do, and ask if they’d like to help review. To keep the workload low, not more than 3 reviews per reviewer are expected. You can also cc your co-organizers on the email to increase the recognition each reviewer receives, though if sending many reviewer invites then consider scheduling the emails to send at the same specific time to avoid distracting your co-organizers with a slow trickle of emails.

How many reviewers?

Aim for 3 reviews per paper so that the majority of papers receive 2–3 reviews. That means inviting about as many reviewers as expected submissions. Tell them the reviewing dates, and follow up with 1–2 polite reminders as the due date approaches to those yet to review. If you do this, only a minority of submissions will receive 0–1 reviews. Many reviewers only do reviews close to the actual deadline, so if you schedule a couple days between the reviewing due date and the notification date, then you can give late reviewers a little more time if they ask for it.

Emergency reviewers:

In anticipation of some papers with 0–1 reviews, you can recruit some “emergency reviewers” in advance to be “on call” for the couple days prior to paper decisions, so that every paper can have at least 2 reviews by the time you made your acceptance/rejection decisions. And if all else fails, you review the papers with fewer than 2 reviews.

Set expectations:

You want to attract novel ideas that are scientifically interesting, on topic, and would create interesting discussions at the workshop. No need to be super critical of experimental results. Workshop papers are not conference papers so communicate to reviewers the criteria you’re looking for. 

Reviewer questions:

Reviewing shouldn’t be laborious for your program committee, so consider limiting the amount of long-answer fields and add instructions to each field. You can even add the reviewing rubric to your website to help answer submitters’ questions like “my paper contains ABC, but not XYZ, is this good enough for a submission?”  Consider including a score on reviewer confidence too.

Paper matching:

To match reviewers to papers, several tools are available. OpenReview has its own matching mechanism. Proper matching benefits are nice for authors and reviewers alike, especially if your topic is broad, or is an application (not method) of AI. For small workshops, bidding is probably overkill and takes time, so you can also manually match reviewers, by looking up their google scholar to classify their expertise in a spreadsheet, then classify the papers into clusters of subtopics on the same spreadsheet, then match like-to-like. Systems like OpenReview help avoid certain conflicts of interest by preventing you from accidentally matching an author and reviewer from the same institution.

Decisions

You can set the bar for acceptance however you like. Generally workshop decision thresholds are more lenient than conferences, resulting in the acceptance of 50–90% of submissions. You can consider aiming for three reviews per paper, and then make decisions based on the three reviewer ratings as follows:

– No accept ratings = reject

– 1/3 accept ratings = investigate, read reviews thoroughly, organizer makes decision

– 2/3 accept ratings = probably accept (check review that rated submission as reject)

– 3/3 accept ratings = accept

Keep in mind some reviewers will review in “conference mode” as if these were conference papers, despite any expectations you set initially. Workshop papers are not conference papers (yet). You want to attract new, scientifically interesting ideas; even if the experimental results are premature, or don’t outperform the state of the art, or even compare to it yet. So if the idea is on-topic and novel, consider dismissing any reject ratings that were based on lackluster experimental results and comparisons.

Notification:

Once you’ve decided which papers to accept and reject, then notify the authors by your promised notification date (which should be on your website). If papers were reviewed, make sure the reviews are viewable by the authors and remind them of the link so they can use the feedback to improve their camera-ready paper if accepted or some other venue if rejected.

Event

Before the Workshop

Checklist:

  1. Organizer presence: Schedule which organizers will attend the workshop when. Ensure you have 2+ organizers actively monitoring things at any one time concurrently.
  2. Organizer communication: Setup a private messaging system for communication between organizers, e.g., with slack, so that on the day of the workshop organizers can coordinate quickly resolving issues as they arise. You can also simply use the slack account that the NeurIPS workshop chairs have invited some of you to already and chat in your specific workshop channel.
  3. Calendar invite for speakers: Make it impossible for your speakers to be confused with locations and time zones by sending them each a calendar event for their talk. Include room+location for the event and the Zoom link if they are remote. This helps if they are running late, and don’t have time to find such info buried in their email somewhere. You could even remind your speakers traveling to New Orleans to set their time zone on the website to America/Chicago.  
  4. Prepare introductions for each speaker (their bio) for a warm introduction and smooth transitions between speakers. If you are unsure how to pronounce their name, ask them privately in advance.
  5. Obtain any consent and release forms (permission to record) from all speakers (keynotes and author talks) supplied to you by any recording groups the conference uses (if any) like SlidesLive.
  6. Print messages: To stay on schedule, help speakers avoid going overtime by bringing boldly printed numbers that you can hold up to inform them how many minutes they have left. Pack a Sharpie!  
  7. Print schedules to pin up outside the room (so conference attendees walking outside can check when they might want to join).
  8. Spare stuff: Bring spare AV adaptors including Apple adaptors, since AV issues often arise on the day, and spares are useful. Also bring your laptop, USB drives; and spare tape and push pins for posters. Pack dongles for laptop to HDMI connections.
  9. Backup questions: Prepare “backup questions” for each keynote or spotlight speaker. When each speaker concludes their talk, you’ll invite questions from the audience, but sometimes they cannot think of anything to ask initially. In such cases, it’s great to have some questions pre prepared. You can prepare by watching speakers’ talks in advance if recorded, or reading their recent on-topic papers in advance.
  10. Attend at least one Workshop Organizer meeting hosted by conference support staff
  11. Enter schedule into the neurips.cc website
  12. Some workshops will be using new remote cameras. This does require that the workshops provide an organizer to select the stream using a SlidesLive stream box.  Plan to identify this organizer to SlidesLive and they will need to attend a training session for the stream box (about 20-30 minutes). Recommended that more than one organizer attend this training so that the job can be shared.
  13. Identify any special needs for room setup several weeks before the conference. A service kit will be provided for additional hardware needed or special room sets. Be sure to pay attention to the deadline dates. i.e., extra mics, room set other than theater, sponsor materials, catering, etc.

Day of the Workshop

It is likely that something will go wrong on the day of your workshop. Redundancy helps here. So have multiple organizers attending the workshop to deal with issues together. Carry a spare laptop, AV adapters, and USB drive in case speakers have equipment issues. Consider having a private schedule with the other organizers who will be the master of ceremonies (MC) when, and who will be watching on standby as “backup MC” ready to take over if anything goes wrong.

Checklist: arrive early and:

  1. Reserve front seats for organizers + speakers (staff office will have “reserved for speaker” seat signs for pick up).
  2. Greet audio-visual (AV) staff, if any, to understand how everything will work, and explain any signals they’ll be gesturing toward you from the back when you’re up at the front talking during the day. There’s often AV issues on the day (a speaker doesn’t have the right adapter etc) and the AV team is critical here, so it’s good to get to know how the AV team (fully-dedicated or rotating) would support you, and thank them at your workshop’s conclusion.

Poster Sessions

Attend your own workshop! Go chat to authors during the poster sessions and learn about their work. As an organizer, you are recommended to visit posters that are not receiving as much attention. People are social, and crowds assume social proof that certain posters are better if other people are already there, creating unequal distributions of crowded posters and lonely posters. You can counteract this bias by visiting less populated posters, which will attract others.

Talks

Even if you prepared “backup questions” for each speaker in advance, for non-recorded talks, you’ll want at least one organizer (the MC) to be paying attention to each speaker during the event to think of backup questions live. If no one asks a question, then you can ask your backup question. Questions beget questions that will warm up the audience. 

After the Workshop

After the event, there’s several options to consider:

  1. Thank any audio-visual staff: After the workshop, thank the audio-visual staff and consider adding their names to the website. They do much critical work behind the scenes.
  2. Get feedback, e.g., via a link on your website or email to improve next year (example).
  3. Dinner: You can organize a dinner for the speakers and (if you have them) sponsors. This is nice for the speakers, and a good opportunity to get to know them better. It can be a bit pricey to buy a dinner for ~10 people, but one of your sponsors may be happy to pay the bill. Have some time between the workshop end and dinner though, people often want a break, or want to chat to others after the workshop instead of rushing off to dinner straight away.
  4. Link recordings to the website. It’s nice for speakers to have public links to their talks and for others who couldn’t make your event to see. E.g., SlidesLive might make links public a month after the workshop, or if you have the Zoom recording, you could upload this to YouTube with time tags for different speakers (example).
  5. Summarize submissions: Some organizers like writing up a paper or blog or website to summarize the workshop, something easily spread on social media. This helps give further publicity to workshop papers and the workshop itself. Most workshops do not take this extra optional step, but this could set the workshop apart.  

Website

You’ll need a webpage, to summarize the speakers, schedule, and how to submit papers.

Accessibility: 

Not all websites are accessible to everyone. Google Sites is very easy to use, but not accessible in China. GitHub Pages is a little harder to use, but accessible in more countries. 

– Custom URLs: Alternatively, you can provide custom URLs to make certain sites accessible. For example https://www.icra2022av.org (accessible in China, and since expired) was actually this website under the hood https://sites.google.com/view/icra2022av/home (inaccessible in China), so the workshop organizers just communicated the .org domain version. You can do this by going to your google site, then settings→ custom domains → add → buy a domain (~$12 a year).

– Github or Google Sites? An advantage of google sites: quick to create + upkeep, for anyone in a team. Advantage of github: get more control on appearance and enable javascript. For example, people can get confused with time zones, so you can always use the “Anywhere on Earth” (AoE) time zone for deadlines, and include a [javascript countdown](#submission-countdown) so people know exactly when the deadline is, or linking to https://time.is/Anywhere_on_Earth. For github you can use this template which looks like this initially, which you can change to look like this. Or use GitHub Pages which is simpler, looks like this; or http://jekyllrb.com/ which looks like this. An advantage of github is you can upload and host papers directly on your website.

– Other: You could also host and manage websites using Amazon S3 buckets

Helpful Tips / FAQ:

You can add suggestions where authors can print their posters near the venue (easier for one person googling this than 50). Some workshops even include poster templates.

Sponsorship

Workshop sponsorship is optional. You do not need sponsorship. Some workshops use sponsorship to fund best paper awards (e.g., cash, cloud credits, or hardware). Although prize amounts not exceeding the cost of travel may not provide extra motivation to submit. Researchers are already motivated to submit papers and often best-papers are won by well-funded labs already. In our opinion, it’s better to fund student travel awards based on financial need, to enable more people to attend your workshop or at least reduce the financial stress of doing so (example application). It helps create a more equal research community and can increase physical attendance. However, workshop sponsorship can raise new challenges, for reasons below. 

Warning: handling funds can be difficult. 

If you have multiple sponsors, then holding funds together can be difficult. You generally cannot hold funds in your own personal bank account. You can open a new bank account, or you can ask a trusted third party like a university to hold funds for you, for a fee (e.g. Toronto might if you’re a student/staff there, Berkeley may not unless the event is on campus). The easiest option could be asking sponsors to transfer to recipients directly without involving third parties. When companies want to sponsor using their corporate credit card, use PayPal. PayPal is the easiest way to make international card transactions to individuals, and works in almost every country (Venmo is US only). If the sponsor requests an invoice for internal accounting purposes, an example is like this template. Please note that NeurIPS does not handle sponsorship for your workshop. You need to handle it on your own.

Warning: hardware gifts can be difficult. 

Certain countries unfortunately have export bans against other countries, especially for computing hardware. If you intend for a sponsor to give hardware to the “best paper award” authors as the prize, ask the sponsors if there are any constraints on authors’ nationalities. If constraints do exist in any country you’d expect to attend at NeurIPS, then you could consider not offering such a prize. 

Other considerations. 

Sponsors might also request proof that prize recipients are in fact the competition or best-paper winners. Emails and websites might not count as proof, but event video recordings might. Find out in advance!

Acknowledgements

This post was adapted from https://rowanmcallister.github.io/post/workshops/ with the help of the original author Rowan McAllister. Original acknowledgements carry (Xinshuo Weng, Erin Grant, Andrea Bajcsy, Thomas Gilbert, Roberto Calandra, Yarin Gal, Pieter Abbeel, Wei-Lun Chao), and additional advice about pre-workshop acceptance is available in the original post.

Announcing the NeurIPS 2023 Affinity Workshops

by Arjun Subramonian, Konstantina Palla, Ogbuokiri Blessing

We are excited to announce this year’s affinity workshops co-located with NeurIPS.

What are affinity groups?

At NeurIPS, affinity groups play a crucial role in advocating for and amplifying the ideas and voices of marginalized communities, bringing attention to the various issues that affect their members. In addition, they provide members of these affinity groups with increased opportunities to showcase their work, engage in discussions, and build connections during NeurIPS events, promoting diversity and inclusion at NeurIPS

The primary goal of workshops is to encourage individuals marginalized in the machine learning community to present their research and become active contributors to the NeurIPS community. In contrast to regular workshops, affinity group workshops adopt a more open and flexible approach around topic area, focusing on forming research communities within the affinity groups and bringing attention to the unique challenges faced by their members.

Additionally, affinity group socials serve as safe and welcoming spaces, allowing members to interact, socialize, and network in an inclusive environment. By fostering such social gatherings, NeurIPS supports the creation of a supportive community that values diversity and ensures that all voices are heard and valued.

What kind of events do affinity groups offer at conferences?

Affinity groups organize workshops and social events for their members at conferences, for which the D&I committee provides support. The main aim of the workshops is to encourage people to showcase their work and step into the NeurIPS community. Most affinity events at NeurIPS this year will take place on Monday, December 11 in-person. NeurIPS will offer streaming support for all the in-person components of the affinity groups. However, individual groups also have the option to organize additional virtual components if they wish to. The specific timings will be communicated by the affinity groups as the conference dates draw nearer.

What can NeurIPS attendees do to support affinity groups?

Participate! Actively attending and participating in panels and sessions for discussions about issues that are relevant to the affinity groups goes a long way in helping promote their cause. We request all members of our community to participate actively!

Black in AI workshop

Black in AI exists to create a space for sharing ideas, foster collaborations, and discuss initiatives to enhance the representation of Black individuals in the field of AI. NeurIPS, being one of the largest AI research conferences, offers a valuable opportunity for researchers in the community to stay updated on the latest research and expand their network. Since its inception in 2017, the Black in AI workshop has had a profound impact. Many members of Black in AI have walked away from their experience at NeurIPS with internship offers, graduate program advisors, mentorship, and renewed energy to continue working in the field. The workshop remains committed to bridging the gap between Black researchers and the AI community at large. For more information, please visit the Black in AI website.

Global South in AI workshop

The Global South in AI workshop is organizing an affinity session focused on Generative AI, with an emphasis on the languages and cultural history of the global south. The session aims to explore both the potential benefits and challenges of Generative AI in a creative and engaging manner. Participants are encouraged to showcase innovative use cases of Generative AI in global south countries using LLMs and to also address issues related to access difficulties or negative impacts arising from its use. For more information, please visit the Global South in AI website.

Queer in AI workshop

The Queer in AI workshop at NeurIPS 2023 aims to gather diverse perspectives from queer students, researchers, and practitioners worldwide to assess the impact of AI on the LGBTQIA* community. The workshop seeks to build a community of queer individuals, foster discussions at the intersection of AI and queer identities, and address the discrimination faced by the LGBTQ+ community in the tech sphere and beyond. The workshop highlights the need for a more ethics-driven approach in AI development and challenges the exclusion of critical voices in AI discussions. It also aims to create an inclusive space for young queer students in academia, where their identities are recognized and celebrated rather than marginalized. By using NeurIPS as a platform, the workshop intends to promote equitable AI and contribute to a future where technology benefits people of all identities and backgrounds. For more information, please visit the Queer in AI website.

LatinX in AI workshop

The LatinX in AI research workshop is a full-day event featuring invited speakers, oral presentations, and posters. It provides an excellent opportunity for faculty, graduate students, research scientists, and engineers to come together, fostering connections and idea exchange. The workshop includes a panel discussion and mentoring session to explore current research trends and career paths in artificial intelligence and machine learning, with a focus on the distinct challenges faced by LatinX identifying researchers. The main objective is to create a platform for showcasing the work of Latinx researchers, and we warmly invite everyone to join us at this event. For more information, please visit the LatinX in AI website.

Women in ML workshop

The goals and activities of WiML are to

  1. Highlight the work of researchers in machine learning who are women or nonbinary,
  2. Give them a chance to meet, encourage technical discussion, and future collaboration,
  3. Give junior researchers an opportunity to present their work to their peers as well as senior members of the community.

The workshop plans to feature speakers who are women or nonbinary to give talks on their research, organize mentorship sessions to discuss relevant topics, and encourage networking. It will encourage and foster research conversations and help participants develop collaboration opportunities. There will also be poster sessions for participants to present a broader array of work and exchange feedback. For more information, please visit the Women in ML website.

North Africans in ML workshop

The North Africans in ML workshop aims to foster collaboration, networking, and skill development and create a supportive community that celebrates and amplifies the contributions of North Africans to the world of machine learning. We aim to create a sense of community among North African researchers in ML, increase the visibility of North Africans within the AI community, highlight their accomplishments, and acknowledge and discuss the hardships faced by researchers based in North Africa.

The workshop will feature various activities such as talks, panel discussions, poster sessions, and a social event. Distinguished experts in machine learning will present cutting-edge research and its applications in the North African context, offering valuable insights and inspiration. The panel discussions will specifically address challenges faced by North African communities in ML research. For more information, please visit the North Africans in ML website.

Muslims in ML workshop

The Muslims in ML workshop aims to explore the intersection of machine learning, fairness, and the global Muslim community. It recognizes the potential of AI and ML to bring positive changes but also acknowledges existing barriers and biases that may perpetuate unfairness against Muslims. The workshop aims to bring together diverse experts and perspectives to examine challenges and opportunities in integrating AI/ML in the lives of Muslims and those in Muslim-majority countries. It will showcase research by Muslim scholars in ML and highlight work addressing challenges faced by the Muslim community. The workshop provides a safe space for open discussions and social activities. It adopts an inclusive approach that considers cultural association and proximity to the Muslim identity, emphasizing the importance of understanding the complexity and diversity within the Muslim community. The workshop seeks to promote awareness, collaboration, and mitigation strategies to ensure fair and equitable implementation of AI/ML technologies for Muslims worldwide.  For more information, please visit the Muslims in ML website

New in ML workshop

The New in ML workshop aims to welcome new researchers in the community and provide them with some guidance to contribute to Machine Learning research fully and effectively. In addition to catering to newcomers, the workshop recently extended its audience to include senior PhDs undergoing role changes, transitioning to academia or industry, thereby embarking on new paths within their ML career. The workshop plans to host presentations offering guidance on how to conduct research from experienced researchers and broadcast them to attract local and remote participants. Attendees will also have the opportunity to be mentored by senior researchers to refine their academic writing and oral presentation skills and boost their chances to have their papers published at the future top conferences. For more information, please visit the New in ML website.

.

Load More