2026 Conference

Ethics in ML research is an important part of ensuring the integrity and impact of scientific work, and NeurIPS 2026 is looking for Ethics Reviewers to support this process. If you are able and willing to participate in the review process, please sign up at this form. Feel free to share this call with your colleagues.

Key Dates

We ask that ethics reviewers

• Review up to 5 papers each,
• Provide ethics reviews during at least one of the following periods:
  • July 6 – 20, 2026 (main ethics review period), * 
  • July 22 – August 13, 2026 (emergency review period – review requests may arrive at any point during this period)

* Note that ICML 2026 takes place July 6-11.

A full list of relevant dates for the conference is available here.

Again, the sign-up form is here: NeurIPS 2026 Ethics Reviewer Self-Nomination 

About the ethics review process

The main reviews conducted through the program committee (reviewers, program chairs, and area chairs) are, and continue to be, the sole decision-making process for accepting or rejecting papers for publications at NeurIPS. Reviewers are expected to review submissions not just for pure technical merit, but also in the context of the NeurIPS Code of Ethics.

The ethics review is a second round of review that takes place mainly when the program committee flags any potential concerns during the main review phase that merit further attention. Ethics reviewers provide feedback to the program committee regarding risks and harms of the work in line with the NeurIPS Code of Ethics, and recommend potential mitigations for authors to incorporate when revising their submissions.

The ethics review process is not a disciplinary or punitive process. However, in rare situations, the NeurIPS program committee may decide to reject submissions that have grossly violated the NeurIPS Code of Ethics, taking into account recommendations from the ethics reviews. In past instances where this occurred, the authors were provided with substantial guidance and relevant citations and were invited to revise and resubmit to NeurIPS.

As detailed in the Ethics Guidelines for Reviewers, the ethics reviews generally follow the double-blinded review process of the main reviews. ​However, additional steps are taken in order to minimize exposure risks. During the ethics review process, any submissions flagged for ethics review will not be publicly labeled as such. During the author response period, ethics reviews will be anonymized when made visible to authors and main reviewers. When the final accept or reject decision has been made, authors who have accepted papers may, at their discretion, choose to make their ethics review public.

Thank you for your consideration,2026 ETHICS REVIEW CHAIRS
Stephanie Hyland, Principal Researcher, Microsoft Research
Emanuel Moss, Senior Research Scientist, Intel
ethics-review-chairs@neurips.cc

When Joelle Pineau launched the first Machine Learning Reproducibility Challenge at ICLR 2018, it was a small community experiment: could we systematically invite researchers to reproduce published results and share what they found? 

Eight years and eight editions later, we are delighted to announce that MLRC 2026 will be an official track at NeurIPS 2026 –  the first time in MLRC’s history that reproducibility science has a dedicated home inside a major ML conference.

This milestone reflects the community’s growing relationship with reproducibility. NeurIPS wanted to send a meaningful signal to the field that reproducibility has become a scientific question worthy of its own rigorous study – this official track allows that to happen.

From challenge to venue

The early iterations of MLRC (v1, v2, v3) were structured as challenges: pick a paper, try to reproduce it, and report what happened. They were enormously valuable as an educational exercise, especially for early-career researchers. Courses like FACT AI at the University of Amsterdam built entire curricula around the challenge, and the quality of the work that came out of those courses showed just how seriously students took the opportunity.

As we reflected on what we learned across those iterations, it became clear that MLRC should be more. Reproducibility is not a binary outcome: a paper is never simply “reproducible” or “not.” The most insightful submissions were always the ones that engaged deeply with the specific claims of a paper, pushed those claims into new settings (generalization), tested their limits, discovered novel insights on top of the paper’s claims, and reported back with nuance. We wanted to create a venue that actively sought out that kind of contribution. Over the years, we improved the program to incentivize these submissions. 

Reproducibility studies are also not easily rewarded by the ML community’s standard metric of novelty: they do not propose new architectures, beat state-of-the-art numbers, or introduce new datasets. Getting researchers to invest serious effort in this kind of work required building a publication and recognition path that made that investment worthwhile. We have updated this systematically over the years: early iterations published through ReScience, a respected open journal for reproducibility across computational science; then, in 2023, we transitioned to TMLR, bringing MLRC papers into a high-prestige, well-indexed ML venue with a rigorous open review process. 

While the initial MLRC operated primarily as satellite workshops to conferences, in 2022 (and 2023), we partnered with NeurIPS in the Journal to Conference track, where the reproducibility papers were presented in poster sessions alongside the conference papers. In 2025, we ran MLRC’s first in-person conference at Princeton University, further elevating the incentive in the form of a dedicated conference and physical stage, inviting keynote speakers to present, and having a full-day event with orals, posters, and networking sessions. Each step raised the incentives, and the NeurIPS track is the next step in that progression.

What this means in practice

MLRC 2026 accepted reproducibility papers will be presented in person at NeurIPS 2026 in Sydney, Australia (December 6–13, 2026), alongside papers from the Main Track and the Evaluations & Datasets Track. The submission and review process remains anchored in TMLR. Reproducibility papers must first be accepted at TMLR within the eligibility window, and then undergo a light compatibility review by the MLRC committee to confirm suitability for the track.

This TMLR-first model is deliberate. TMLR’s open, continuous reviewing cycle allows authors to refine their work and get expert feedback before it is considered for presentation. It also means that accepted papers carry the full weight of TMLR’s review standards, independent of MLRC. The MLRC committee’s role is then to identify papers among accepted TMLR submissions that represent the best of reproducibility science and would benefit from the visibility of a NeurIPS venue.

What we are looking for

MLRC has always welcomed a broad range of reproducibility work, and that continues this year. We are looking for papers that take reproducibility seriously as a scientific question — not just as a means to an end, but as a contribution in its own right. This includes:

• Reproductions and replications that rigorously test specific claims from published papers, whether they confirm, partially replicate, or fail to reproduce prior results
• Generalizability studies that extend original findings to new settings, datasets, or model architectures, adding insights that the original paper could not offer
• Meta-reproducibility studies examining reproducibility patterns across a body of related work
• Methods and tools that make reproducibility research more accessible or rigorous
• AI-assisted reproducibility, including studies that use or critically evaluate automated approaches to replicating research papers
• Reproducibility of AI systems and agents as subjects of study in their own right

An important point to note: negative results and partial failures to reproduce are as valuable as confirmations. Science advances by understanding where claims hold and where they do not. A careful, well-documented failure to reproduce a result — with a clear account of what was tried and what was found — is a genuine contribution to the literature.

Note: Work focused on evaluation methodology more broadly may also be a good fit for the Evaluations & Datasets NeurIPS 2026 track. We encourage authors to consider both venues when deciding where to submit.

How to submit

To be eligible for MLRC 2026, your reproducibility paper must be accepted as is / with minor revisions to TMLR and must have been submitted to TMLR between June 20, 2025 23:59 AOE and September 30, 2026, 23:59 AOE. Please check our CFP for more details.

We accept submissions through three paths:

Path 1: Expression of interest (EOI) before acceptance. If your reproducibility paper is currently under review at TMLR and was submitted within the eligibility window, you may submit this EOI form to express your intent to be considered for MLRC. The “intent to submit” deadline is June 4, 2026, AOE – this is primarily a soft deadline to allow you to submit your paper to TMLR well in advance, so that you get your decisions in time. If your reproducibility paper is subsequently accepted to TMLR, you will be asked to update the form with your acceptance details and camera-ready materials. The hard deadline by which we need your TMLR paper’s acceptance decision is September 30, 2026, AOE – we cannot accommodate your paper into MLRC any later than this date, even if you are still waiting for your TMLR decisions.

Path 2: Self-nomination after acceptance. If your reproducibility paper has already been accepted to TMLR and was submitted within the eligibility window (start date June 20, 2025 23:59 AOE), you may submit this form to be considered for MLRC. Once accepted to MLRC, you will be asked to update the form with your camera-ready details. Similar to Path 1, the hard deadline to submit this form is September 30, 2026, AOE. Ensure you have not submitted your accepted paper to the NeurIPS 2026 Journal to Conference track, as our dual submission policy restricts dual presentation.

Path 3: Area Chair nomination. TMLR Area Chairs may nominate accepted reproducibility papers within the submission window for consideration at MLRC. No action is required from authors unless they are contacted. Area Chairs can use the same form to submit their nomination. The deadline for TMLR Area Chairs to nominate reproducibility papers is also September 30, 2026, AOE.

For all paths, please read the TMLR author guidelines and submission guidelines for formatting and review process details.

The hard deadline to have your TMLR acceptance in our system is September 30, 2026 AOE.

Important Dates

• Earliest date of the acceptance of your TMLR reproducibility paper for consideration to this year’s MLRC: June 20, 2025 23:59 AOE
• Soft deadline (EOI / intent to submit): June 4, 2026, 23:59 AOE
• Deadline to have TMLR decisions in our system: September 30, 2026, 23:59 AOE
• Author notifications: October 7, 2026
• NeurIPS 2026: December 6–13, 2026

Closing Thoughts

Reproducibility has always been foundational to science. What MLRC has tried to do, across eight editions, is make reproducibility a first-class research activity in machine learning — one that is worth investing in, publishing, and being recognized for. Having MLRC as an official NeurIPS track is an affirmation that the community values this work, and we hope it encourages more researchers to take reproducibility seriously as a scientific contribution.

We look forward to seeing the community’s work at NeurIPS in Sydney. Please visit the MLRC 2026 website for full details, and do not hesitate to reach out at  at reproducibility-chairs@neurips.cc with any questions.

This year, to improve transparency and responsible use of datasets, we are introducing a new requirement for dataset submissions in the NeurIPS 2026 Evaluations and Datasets Track: all dataset submissions must now include Responsible AI (RAI) metadata as part of the dataset’s Croissant file.

Dataset submissions to the track are already required to include a Croissant file, a standardized, machine-readable metadata format for describing datasets and their structure. We extend this requirement to also include RAI metadata within that file.

RAI metadata helps ensure datasets are used appropriately in AI research. Providing standardized information on how a dataset was created, its limitations, and its intended use helps researchers understand when and how the data can be used, reducing the risk of misuse, biased results, or misleading conclusions. This requirement also advances a core goal of the track: making datasets more transparent, comparable, and reusable across the community.

To make this as smooth as possible, we collaborated with the Croissant RAI team to establish a minimal set of RAI fields describing key aspects such as dataset limitations, potential biases, intended use, and other responsible AI considerations. Authors must include the RAI information directly within the Croissant metadata wherever possible, or clearly link to relevant sections of the paper when needed.

To support authors, we provide two tools to streamline this process:

• An online RAI editor to help complete the required RAI metadata fields
• A Croissant validator to ensure files are complete and correctly formatted

We recommend completing these steps as early as possible, and inform authors that dataset submissions missing RAI metadata from their Croissant file will be flagged during review.

For full details on the dataset hosting and submission requirements, please see the Call for Papers and the data hosting guidelines. 

Joaquin Vanschoren, Konstantina Palla, Jessica Schrouff, Alexandre Drouin, Lijun Wu

Evaluations and Datasets Track Chairs 2026

NeurIPS Program Chairs: Marc Deisenroth, Finale Doshi-Velez, Nika Haghtalab, David Rolnick, Jenna Wiens

Google Research: Rajesh Jayaram, Vincent Cohen-Addad, Drew Tyler, David Woodruff

Google Cloud: Jinsung Yoon, Mihir Parmar, Palash Goyal

Google Sponsors: Corinna Cortes, Vahab Mirrokni, Tomas Pfister, Burak Gokturk

NeurIPS Communication Chair: Jean Kossaifi

Following positive feedback from other venues, like STOC and ICML, NeurIPS is pleased to announce a new initiative, in partnership with Google, that will provide authors access to their Paper Assistant Tool (PAT) and support authors in improving their submissions to NeurIPS 2026. 

This program offers authors a limited opportunity to receive free, automated, and actionable feedback on their manuscripts before the final deadline, private to the authors. The feedback the authors will receive from PAT through the NeurIPS program will not be used in the review process. Reviewers, area chairs, and program committee members will not have access to the PAT feedback. 

What is PAT?

PAT is a specialized, experimental tool powered by Google’s Gemini models, utilizing a “reasoning”-focused pipeline. It is similar to those that have achieved high-level performance on mathematical problem-solving benchmarks. The model is designed to help authors identify issues that human reviewers might flag, including (but not limited to) experimental and methodological rigor, narrative clarity in English, and technical correctness.

In a pilot at the Annual ACM Symposium on Theory of Computing, STOC, (see blog post), 94% of participants found the pre-submission feedback generated by an AI assistant to be helpful, and 85% reported that the feedback resulted in improved clarity of their paper. Following this pilot, PAT was expanded and launched in partnership with ICML (blog), which had a similarly positive reception. Notably, 35.4% of responding authors with theoretical results reported the tool identified significant theory gaps that took more than an hour to fix, and 31% of responding authors with experimental results said the feedback prompted them to run new experiments. 

While the program at STOC focused heavily on theoretical correctness, the ICML program was then specifically tuned to address the needs of the machine learning community. After considering the feedback received during the ICML program, including pain points raised by authors, the Google Research team augmented PAT by integrating components of the ScholarPeer system into the PAT feedback.  As a result of this integration, compared to the ICML iteration, this version of PAT has:

• Improved search and tool capabilities: allowing for fact checking and reducing the hallucination rate.
• Improved comparison with related work: resulting from the ScholarPeer deep literature review agent. This empowers PAT to take into account related work in its analysis of the paper.
• Strengths / Weaknesses Analysis: the feedback produced by PAT will contain an analysis of the potential strengths and weaknesses of the paper. This will enable authors to improve their paper at a higher level, rather than just addressing individual errors.

Like the ICML and STOC programs, the goal is to help authors improve the quality of their papers, not to replace human peer review. By fixing clarity issues and potential technical gaps before the work is submitted to the conference, we hope to give authors actionable feedback before their paper enters the review process.

Logistics and Eligibility

The program is entirely optional. It operates inside OpenReview, but completely outside the official review process. The program will run for a 7 day window, ending on the Abstract Deadline (May 4th, Anywhere on Earth). 

To manage resources fairly, each eligible author is granted one virtual “voucher” to have a single paper run through the AI feedback system. In addition, each paper can be run through the system at most once. In alignment with the system ultimately used during ICML 2026, we define an eligible author as any author whose OpenReview Account was created prior to April 1, 2026.

To redeem this voucher, authors will select a checkbox “Ready for LLM Feedback” on the OpenReview paper submission form to flag the manuscript for AI review. This feature will only work after a PDF has been successfully uploaded to the OpenReview server. The author who checks the box redeems their voucher for the paper once the edit to the submission is submitted, and the version of the document at the time of the edit is then sent to the pipeline for automated feedback. If an ineligible author or an author who has already used their voucher attempts to select the “Ready for LLM Feedback” button, an error message will appear and the paper will not be sent out for review. Eligible submissions will typically receive the feedback within 12 hours of being submitted. 

Submission Timing: To ensure system stability and incentivize early submissions, papers submitted earlier in the feedback window are guaranteed the full compute budget of the PAT pipeline. Submissions made very close to the deadline (1-2 days) may be subject to throttling of their overall compute allocation depending on the demand. 

The technical staff will be able to provide limited, best-effort support on the program, such as answering questions or checking for failed paper delivery. Please direct such questions, as well as any feedback you may have on the program, to paper-assistant@google.com.  

Privacy and Data Safety

We recognize the sensitivity of unpublished research. Trust is the cornerstone of this experiment, and we have implemented strict protocols to ensure author safety:

1. Strict Separation from Peer Review: The AI Feedback is entirely independent of the NeurIPS review process. It is visible only to the authors. Reviewers, Area Chairs, and Program Chairs will have no access to this feedback. Furthermore, the PAT system will not be used in any part of the NeurIPS review process, including the NeurIPS 2026 AI Assisted Reviewing Experiment. 
2. Stateless Inference (No Training): Submissions will not be used to train, fine-tune, or improve Google’s models. The model operates in a stateless “inference-only” mode; it processes the text to generate feedback and retains no memory of the specific content for future learning.
3. Data Destruction: To minimize data exposure, Google will employ a strict deletion policy. All PDFs and feedback submitted to Google are stored in a restricted access environment and are scheduled for permanent deletion within 7 days after the feedback is delivered and the program is completed.
4. Restricted Access: No one has access to this data unless there is a technical difficulty, in which case Google staff (Rajesh Jayaram and Drew Tyler from the Google Research Organizing Committee) will only inspect the data (submission PDFs and generated feedback) with explicit author approval.

Caveats and Disclaimers

Like all LLMs, the models used by the PAT pipeline are not infallible. Authors should treat the generated feedback with the same critical eye they would apply to a human review.

• The model may occasionally flag correct statements as errors or miss actual flaws. It is the author’s responsibility to verify the validity of the feedback.
• Note that the model may make suggestions for the paper that you disagree with. This is not necessarily a bug, and may even be desirable as such behavior could also occur with reviewers. By considering why you disagree with the suggestion, you may be able to add justification to the paper which preempts certain reviewers’ comments, thereby increasing the strength of the paper.

Outcomes

Our primary objective is to empower authors to elevate the quality of their submissions by acting as a high-precision filter. This tool is designed to help authors catch nuanced errors—in proofs, experimental setups, or reasoning—that human reviewers might miss or lack the time to detail.

We believe this tool represents a promising opportunity to test the use of AI to elevate the standard of our own scientific submissions. After the full paper deadline, an anonymous author survey will be sent to authors who used PAT to request feedback so that Google can improve PAT. Submitting feedback via the survey is optional. We look forward to seeing the results of the program, which will be shared with the broader community via the NeurIPS blog once the program is over.

FAQ

PDF Size: Due to context limitations, very large PDF’s containing, for example, multiple high resolution images or plots, may have their images stripped before processing. This would result in PAT running only on the extracted text portion of the paper. In extreme cases, the pipeline may fail altogether. We recommend submitting PDFs no larger than 20MB (ideally less than 10) to the PAT system to ensure success of the pipeline. 

Turnaround Time: We expect that feedback will be posted within 1-2 hours of submission to the PAT system. During times of high demand, such as closer to the deadline, the latency may be longer. We strive to have all feedback posted within 12 hours of the submission to PAT.

How do I know my paper was successfully submitted?: After clicking the “Ready for PAT Review” button, your PDF should be picked up by the PAT system within a few minutes. At that post, a private comment (visible only to authors) will be posted to your paper notifying you that the paper has entered the PAT processing queue.

Contact (for feedback and best-effort assistance): paper-assistant@google.com 

The NeurIPS community benefits from a wide diversity of papers and ideas, which can arise and be developed in many different ways. To help cultivate the diversity of papers that belong at NeurIPS, this year the Main Track is asking authors to select a Contribution Type:

• General: We expect that most submissions will fall into this type.
• Theory: The main contribution is via theoretical analyses and proofs.
• Use-Inspired: The main contribution is in framing or designing approaches to meet the needs of a specific real-world application. (This often involves, e.g., engaging with domain experts.).
• Concept & Feasibility: The main contribution is a highly novel, high potential reward idea with scope beyond what can be validated in a single paper. (The significance and originality bar for these contributions is high.)
• Negative Results: The main contribution is in understanding a negative result. (The significance and originality bar for these contributions is high.)

The review form used is the same across all Contribution Types, but the way that reviewing criteria are interpreted will differ across Types. We encourage authors to look at the reviewing guidelines for each Contribution Type in deciding which to select. (By contrast, the Evaluations & Datasets Track and Position Paper Track have different review forms from the Main Track.)

Note that because reviewers will base their recommendations on the Contribution Type, it is not possible for this to be changed after submission, either by the authors or by the reviewers / PCs. It is also, of course, not permitted to submit the same or highly similar papers to multiple Contribution Types or Tracks – such papers will be desk rejected.

To help authors select the right Track and Contribution Type for their paper, we include a number of example papers below.

Example papers

Main Track – General

Segment Anything (Kirillov et al. 2023)

This paper introduces a model (SAM) for segmenting images, and a dataset used for training it. The focus of the paper is on the particular model being developed, so the paper would be a better fit for the Main Track (General) than the Evaluations & Datasets Track. The paper does not motivate or evaluate SAM with respect to the specific needs of a real-world use case – therefore the Use-Inspired contribution type would not be a good match.

The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks (Frankle and Carbin 2019)

This paper empirically investigates the efficacy of training pruned subnetworks of a neural network. While this area of work is related to deep learning theory, the contributions are not theoretical and so Main Track (General) would be the best choice.

Main Track – Theory

Neural Tangent Kernel: Convergence and Generalization in Neural Networks (Jacot et al. 2018)

This paper formally connects neural network training to kernel methods with the introduction of the Neural Tangent Kernel (NTK). While the authors do conduct some illustrative experiments, the focus is on proving theoretical properties of the NTK and analyzing the implications of these results.

A Universal Law of Robustness via Isoperimetry (Bubeck and Sellke, 2021)

This paper is motivated by the significant overparameterization of deep learning models in comparison to the number of datapoints being fit. The authors prove theoretical results on the number of parameters a model needs to smoothly interpolate between datapoints, showing this is significantly more than the number required for interpolation alone, and consider how these results may align with behavior observed in practice.

Main Track – Use-Inspired

Estimating Canopy Height at Scale (Pauls et al. 2024)

This paper considers the application of ML models to the problem of canopy height estimation. While the core models are not novel, the authors contribute an analysis of what makes this problem and data special and introduce a novel loss function to account for these properties.

Probabilistic Emulation of a Global Climate Model with Spherical DYffusion (Rühling Cachay et al. 2024)

This paper considers the problem of emulating the Earth’s climate over long time horizons. The authors introduce Spherical DYffusion, an approach combining diffusion models with Spherical Fourier Neural Operators, in order to address several challenges particular to atmospheric data. Via extensive domain-informed experiments on a single, well-chosen dataset, the authors compare Spherical DYffusion to other models, both from machine learning and from physics.

Main Track – Concept & Feasibility

Capsule networks (Sabour et al., 2017)

This paper introduces a novel type of neural network architecture, capsule networks, which permit dynamic routing of information via a type of attention computed between different capsule units in the network. The paper shows that capsule networks have promising performance on a range of MNIST-based tasks, but explicitly does not attempt to explore the full range of properties they demonstrate on different tasks and datasets, nor to consider all ways that capsule networks may be instantiated. The authors draw parallels between the early stage of capsule network research and that of recurrent networks several years previously – emphasizing that such promising results in early-stage research suggest the value of more work in this area.

Stochastic Variational Inference (Hoffman et al. 2013)

This paper introduces a scalable approximate algorithm for posterior estimation that allows for analyzing large datasets through a Bayesian lens. This is an example of a paper with very broad applicability. The experiments could obviously not test on all Bayesian models, but provided concrete evidence that the ability to approximately analyze larger datasets provides performance benefits over using traditional inference on smaller datasets.

Main Track – Negative Results

Inherent Trade-Offs in the Fair Determination of Risk Scores (Kleinberg et al. 2017)

This paper investigates the impossibility of satisfying multiple fairness criteria simultaneously. While it relies heavily on a theoretical framing, its main contribution is that of a negative, rigorously demonstrated and surprising result that is not demonstrated via empirical evaluations.

Understanding deep learning requires rethinking generalization (Zhang et al., 2016) 

At the time this paper was written, the common wisdom was that generalization in deep learning was similar to generalization in classical models. Through a combination of experimentation with carefully chosen semi-synthetic data and a theoretical analysis of a specific neural architecture, the authors make a compelling argument that the sources of generalization in classical models – limited expressivity and explicit regularization – cannot explain the generalization properties of neural networks. This was their core (negative) result. While they speculate that implicit regularization via SGD may be a critical element, the negative result is the main argument of the paper.

Evaluations & Datasets Track

ImageNet: A large-scale hierarchical image database (Deng et al., 2009)

This paper offers a dataset for computer vision applications with a demonstration of its value in three tasks. The primary contribution relates to the dataset.

Fairness Through Awareness (Dwork et al., 2011)

This paper gives a definition and implementation of individual fairness, with a secondary contribution that provides an algorithm to improve on this metric. The paper relies heavily on a theoretical framework. While the authors could consider different options (main/general due to algorithm development or main/theory), we believe the Evaluations & Datasets Track is most appropriate given that the main contribution is about the definition of a new fairness metric.

The Illusion of Readiness in Health AI (Gu et al., 2025)

This paper focuses on evaluations for a use-case inspired application in healthcare providing negative results through experimentation. While this is a use-case inspired, negative result, the primary focus of this paper is on experimental evaluations.

Position Paper Track

Position: Bayesian Deep Learning is Needed in the Age of Large-Scale AI (Papamarkou et al., 2024)

This paper revisits the role of Bayesian deep learning in the modern AI landscape. It argues that uncertainty-aware approaches should play a central role. While it is close to a review paper due to its broad synthesis of existing methods, it advances a clear normative stance on the field’s future direction rather than merely summarizing prior work.

Position: Graph Learning Will Lose Relevance Due To Poor Benchmarks (Bechler-Speicher et al., 2025)

This paper examines the current state of graph learning, arguing that progress in the field is hindered by flawed benchmark datasets and evaluation practices. The position is about benchmarking and evaluation practices in graph learning, arguing that they must be fundamentally rethought. While being close to an Evaluations and Datasets paper due to the introduction and analysis of concrete benchmarking setups, it focuses on a broader critique and agenda for the field, which is characteristic of a Position Paper.

Position: Probabilistic Modelling is Sufficient for Causal Inference (Mlodozeniec et al., 2025)

This paper challenges the common view that causal inference requires specialized frameworks such as structural causal models or do-calculus. The position is about the foundations and methodology of causal inference. While lying close to a Main Track paper, it does not introduce a concrete algorithmic pipeline as its primary contribution, but instead advances a conceptual stance on how causal inference should be framed.

We are asking community members to self-nominate as Area Chair for the 2026 Position Paper Track! In line with the track’s aim of expanding topical scope, we are looking for experts from a diverse range of disciplines. NeurIPS relies upon the active participation of the community to evaluate submissions and uphold scientific quality. Area chairs in particular help manage reviewers for submissions, making sure that their reviews are high-quality and on-time, and provide accept/reject recommendations based upon reviewer assessment and discussion. 

Criteria

We welcome self-nominations from anyone who is or wants to be part of the NeurIPS community, and believes they have the experience to act as an area chair. Although we provide some guidance below, especially because we are soliciting self-nominations from a wide range of backgrounds, we recognize that many nominees may qualify as strong area chairs without meeting all (or many) of the criteria below. Thus, we also provide a free-form text box for nominees to justify their readiness. 

To provide some guidelines, we generally expect area chairs to:

• Be relatively senior, meaning they have a mature published research portfolio and experience in a supervisory capacity in projects.
  • A rule-of-thumb is 3-5 last-authored peer-reviewed publications, and 8-10 peer-reviewed publications in any capacity
• To have at minimum two years of experience either as a reviewer or area chair for a major machine learning or related conference (e.g. NeurIPS, ICLR, ICML, CVPR, ICCV/ECCV, ACL, EMNLP, KDD, COLT, FAccT, ACM)

To self-nominate, please fill out our Google forms here. We only accept self-nominations, but we would appreciate anyone who would forward this form to colleagues who would be well-suited: https://docs.google.com/forms/d/1IQWm78nsVFJW7UXU7QgmKXtA2JVSipnSbbCXy609rvU

Alex Lu, David Rügamer, and Seth Lazar
Position Paper Chairs 2026

We are excited to announce the Position Paper Track is back at NeurIPS 2026 for the second year. Last year was a crucial experiment that created a forum for discussion on hot topics in our field. As a newer track at NeurIPS, we made several experimental decisions last year intended to promote discussion and improve positive interactions during review. We have been carefully listening to the community’s feedback, both positive and critical, and are excited to announce in this blog post new directions for the track this year.

Aligning timeline and standards, broadening scope, and providing clearer definitions of rigor

As we learn more about community expectations, we continue to evolve the track accordingly, and this year we are seeking to better align our paper acceptance process and timelines for submission, review, and notification with the other conference tracks. Working towards standardised practices for acceptance decisions wherever feasible (informed by post-conference reports from last year’s Main and Datasets and Benchmarks chairs) will ensure more consistent standards for quality and  acceptance rates across NeurIPS tracks. We also heard feedback from the community that the lag in review and decision timeline relative to other tracks was disappointing, as it caused authors to miss deadlines for re-submission to other venues. We are therefore planning to adhere closely to timelines adopted by the other tracks this year.

We’re also aiming to broaden the scope of papers accepted to the Position Paper Track. Many community members perceived a bias against papers with technical topics in 2025. While part of this balance occurred because some papers submitted to the Position Paper Track were better suited for the Main Program (we desk rejected many papers that were effectively “our method is superior and the community should adopt or investigate it”), we recognize that it may also reflect a reviewing pool skewed towards certain kinds of expertise. To address this, we’re coordinating with the other tracks to recruit reviewers from a shared pool. Not only will this broaden the range of expertise for reviewers on the Position Paper Track, but it will also reduce burden on reviewers by ensuring that they do not get assigned to a full review load for multiple tracks simultaneously.

We hope this broadening of reviewer expertise will better equip us to address both topics that are of more widespread interest for the entire community, and topics that may be specific to some subcommunities. We ask the community to please submit papers engaging with a wide range of topics. We welcome everything from more technical topics like “Bayesian learning is crucial for LLMs”, to topics with more widespread accessibility like “Deployment of agentic AI is hurting society”, to topics for specific application areas like “Virtual cell models must predict more than gene expression” – and obviously, this list is not exhaustive. We look forward to your perspectives!

Finally, if you go to our Call for Papers, you’ll notice that we’ve substantially updated our text this year. Our aim was to better define what high-quality, rigorous submissions look like for the Position Paper Track, and we encourage all authors to carefully read our new criteria. We are putting greater emphasis on how well-argued the position is, not just the likelihood that a position will create discussion in the community. We make it clearer that position papers should support their positions with evidence. We also ask that authors make a case that topics are interesting for contemporary discussion in their papers, not just assume that the topic is widely held to be “hot”. 

Because position papers are relatively new for machine learning conferences, standards for quality and track boundaries are still forming. To help refine this, we are asking authors to prepare a short statement explaining why their submission is best suited to the position paper track. This statement will be used to assess that submissions are indeed aligned with the criterion that we are putting out this year, but also to refine criterion for future years. 

Improving author-reviewer interaction and review

Last year, we experimented with review processes, moving rebuttals to an author survey and an adjudication process. While we noticed some positive dynamics from both and the adjudication process, although noisy, was instrumental in identifying some cases of misconduct in the review process, we also heard feedback from the authors that they were frustrated at not being able to discuss opportunities for changes with the reviewers. Hence, this year, recognizing that interaction between authors and reviewers is important, we intend to align with the main conference in creating a period for author-reviewer discussion.

In addition, the Position Paper Track will be participating in the Responsible Reviewing initiative with the Main Program and Evaluation and Datasets track. Last year, around 35% of reviewers were not responsive in their reviews. This created a huge need for emergency reviewers and delayed our timeline in releasing decisions. As last year was our first year in hosting this track, we were unprepared for the high rate of missing reviews. We are regretful this happened as it meant some authors missed deadlines to resubmit rejected papers to other venues. Learning from experience, this year we intend to adopt the same policies by other tracks at the conference to safeguard reviewing quality and timeline. Crucially, the Responsible Reviewing initiative will apply across tracks: for example, reviewers who do not submit reviews on time for the Position Paper Track can expect to have reviews for their papers withheld even if those papers are under the Main Program or Evaluation and Datasets track.

Please provide feedback!

The Position Paper Track still remains a new track for NeurIPS, so the goal of releasing this blog post is to declare our intentions this year on how we will refine and improve it. However, we remain committed to learning throughout the process this year, and while some decisions may be fixed early on, others may have flexibility as conference planning proceeds. For this reason, we are especially keen on receiving feedback from the community. Do you think our directions this year are going the right way? Do you disagree? Do you have ideas? Please reach out by emailing us at position-track-chairs@neurips.cc! 

We are excited to announce that the Datasets & Benchmarks Track at NeurIPS 2026 has been officially renamed the Evaluations & Datasets (ED) Track. While the track continues to align with the main conference (see the call for paper) in terms of requirements and timeline, this year we introduce an important refinement; an expanded scope that explicitly positions evaluation as a scientific object of study. The goal is to improve clarity and rigor in how we assess progress in the field. 

This post accompanies the ED Track’s Call for Papers and outlines the rationale behind this update. 

From “Datasets & Benchmarks” to “Evaluations & Datasets” 

The Datasets & Benchmarks Track has become a leading venue for work on dataset creation and benchmarking. As the field matures, it has become increasingly clear that evaluation plays a central role in shaping scientific conclusions. What we measure, under which assumptions, and how we interpret results often determine conclusions as much as model architecture does. 

Datasets and benchmarks are tightly connected to this process: they define what can be measured, compared, and ultimately claimed. Ongoing concerns around reproducibility and comparability have further highlighted that disagreement frequently stems from differences in evaluation design, assumptions, and reporting practices. As models become more standardized and widely available, evaluation methodology more frequently determines the conclusions we draw.

For NeurIPS 2026, we refine the track to reflect this evolution; evaluation itself becomes an object of scientific study.  The new name places evaluation first, while datasets remain central and receive renewed focus within broader evaluative practices rather than as endpoints in themselves.  This shift broadens the scope of the track, allowing works that were previously submitted to the main track to be more clearly aligned with the ED Track’s reframed focus.

Defining evaluation as a scientific object

We explicitly define evaluation as:

Processes, practices, tools, and resources for making evaluative claims about AI/ML systems, including – but not limited to – datasets, benchmarks, user studies, simulators, auditing, red-teaming methods, interaction protocols, metrics, and experimental or qualitative study designs.

This definition also includes datasets and related resources when they are intended for use in any part of the AI/ML lifecycle (e.g. training, fine-tuning, testing etc.) and are accompanied by a clear description of their scope, underlying assumptions, and explanation of how models or systems studied using them are expected to be meaningfully assessed.

By making this definition explicit, we clarify the motivation behind the track’s new name. Evaluation is not merely a procedural step, but a scientific object in its own right – one that can be studied, stress-tested, reproduced, audited, and improved. It shapes how we interpret performance, robustness, fairness, and reliability, and ultimately determines which conclusions and comparisons are justified. Placing “evaluations” first reflects this central role.

What This Means for Submissions

Under this reframing:

Contributions extend beyond introducing new datasets or benchmarks. Works whose primary contribution is the analysis, critique, redesign, or stress-testing of evaluation practices or methodologies are fully in scope. Advancing evaluation science is itself a central goal of the track.

Datasets remain central. They may be used at any stage of the AI/ML lifecycle, including training, fine-tuning, auditing, or testing. Submissions introducing datasets are welcome, including those whose primary contribution is enabling new capabilities, applications, or problem settings. To support meaningful use and comparison, authors should clearly explain what claims the dataset is intended to support (e.g., improved model performance, fairness, robustness, safety, or other model characteristics), under what assumptions those claims are valid, and what limitations constrain them. 

The key requirement is not the stage at which a dataset is used, but that its relationship to evaluative claims is clearly articulated; authors should clarify how models developed or studied using the dataset are intended to be assessed and what kinds of claims such assessments can support. For instance, if a dataset enables the creation of better models, the submission should explain how those models should be meaningfully evaluated (e.g. how improvements are to be assessed and what constitutes meaningful evidence of progress). Papers that present data without clarifying the intended problem formulation, evaluation setup, or interpretive boundaries are unlikely to provide sufficient context for review and may therefore not fall within the intended scope of the track. For example, simply releasing a large collection of data without specifying the problem setting it addresses, how it shapes evaluation, or what kinds of claims it is meant to enable  would not provide a sufficient basis for review under the track’s criteria.

Submissions need not introduce a new model or outperform prior work. Rather than results showing that one model outperforms another, we seek contributions that advance our understanding of model performance and how evaluative claims are constructed, supported, and interpreted. Negative results, critical analyses, and use-case-inspired evaluations are welcome. A submission need not “beat a baseline”; its primary contribution should be to deepen and refine our understanding of evaluation practices.

Benchmarks remain in scope. They are a subset of evaluations, standardized, reusable setups that serve as shared comparison points. Removing “benchmarks” from the title does not signal exclusion; rather, it reflects that benchmarks are one form of evaluation. This framing allows the track to welcome both the development of new benchmarks (when scientifically justified) and rigorous analyses of when and why existing benchmarks fail.

In-Scope Submissions

Under the revised scope of the track, we welcome work that:

• Analyze strengths, limitations, or failure modes of existing benchmarks or evaluation practices
• Study benchmark saturation or overfitting and their impact on scientific conclusions
• Compare evaluation designs and demonstrate how different assumptions lead to different results or conclusions
• Provide rigorous reproduction, auditing, and stress-testing of prior evaluations
• Develop documentation methodologies (e.g., Data Cards, Model Cards, evaluation cards) that improve how evaluative claims are made, interpreted, and compared
• Refine existing evaluation setups
• Propose new evaluation protocols, practices, or methodologies
• Conduct human- or interaction-centered evaluations (e.g., user studies, red-teaming)
• Introduce datasets and clearly explain their scope, assumptions, limitations, and how they are intended to support or shape evaluative claims across the AI/ML lifecycle
• Contribute tools, analyses, or frameworks that improve how evaluative claims are constructed or interpreted
• Present negative results, critical analyses, and use-case-inspired evaluations

Moreover, data-centric and benchmarking submissions historically welcomed by the track remain fully in scope. These include, but are not limited to:

• New datasets and dataset collections
• Data generators and reinforcement learning environments
• Data-centric AI methods and tools
• Advanced data collection and curation practices
• Responsible dataset development frameworks
• Audits of existing datasets
• Benchmarks on new or existing datasets, benchmarking tools, and methodologies
• Systematic analyses of systems on novel datasets
• In-depth analyses of machine learning challenges and competitions (by organisers and/or participants) that yield important new insights
• Competition papers from prior NeurIPS competitions 

For 2026, we encourage all submissions, including those in the categories above, to clearly explain their contribution in light of the revised scope.

We welcome your feedback

As the Evaluations & Datasets Track continues to evolve, we welcome input from the community. Our goal is to improve clarity and rigor, and we value perspectives on how to best achieve that. Please feel free to reach out to the ED Track Chairs at evaluationsdatasets@neurips.cc with feedback or suggestions.

ED Track chairs: Konstantina Palla, Jessica Schrouff,  Alexandre Drouin, Lijun Wu, Joaquin Vanschoren

As NeurIPS continues to grow, we recognize that Area Chairs (ACs) are, in many senses, the most essential people in the NeurIPS program creation process. Your judgment about how to synthesize reviews towards an accept or reject decision largely determines our program.

Recognizing the pivotal role that ACs play, we are introducing three synergistic initiatives this year to enhance the role of ACs as the definers and upholders of standards: 

Initial Meta-Reviews Prior to Author Response

This year, to better guide authors, we will pilot early AC meta-reviews to provide context before the rebuttal period begins. As is common in many other computer science conferences, the goal of these initial meta-reviews will be to help authors identify the key issues to focus on in their responses. We anticipate that meta-reviews will generally fall into three primary categories:

• A clear signal that this paper is headed toward acceptance. In some cases, all reviewers may be strongly aligned toward acceptance and have no substantive concerns. In this case, you would tell the authors that there are no major concerns and that they should respond to minor questions and corrections that the reviewers have brought up. 
• A clear signal that this paper is headed toward rejection, unless the reviewers have grossly misinterpreted the paper. In some cases, all reviewers may be strongly aligned toward rejection and you agree with their deep concerns. In this case, you would briefly summarize the major concerns and ask the authors if the concerns stem from a gross misinterpretation of the paper. Importantly, you should communicate that the authors will not succeed by pointing out additional strengths if the deep concerns are true – the deep concerns, if true, represent fundamental shortcomings that preclude publication.
• A clear signal about what concerns would have to be addressed for the paper to increase the odds for acceptance (most common). Going through the initial reviews, you decide what concerns are critical (e.g. a potentially incorrect proof, an important missing baseline, placing the work in context) and what concerns are not critical (e.g. novelty with respect to a method you believe is largely unrelated, an additional experiment you believe is not relevant). In this case, you would list these critical concerns to the authors and tell them that they will have to address them to the satisfaction of yourself and the reviewers. What you are communicating is that the authors do not need to worry about concerns you have not listed, and that there is a real opportunity for acceptance if listed concerns are sufficiently addressed. Note: even if the authors respond to the concerns, you may still choose to reject the paper if you and the reviewers find the response insufficient. 

These meta-reviews will be available to both the authors and reviewers. We emphasize that these initial meta-reviews do not need to be long, but they do need to be thoughtful because you are effectively making a promise to the authors regarding the next steps of the process.

We recognize the increased effort required at this stage. However, we hope that these initial meta-reviews will streamline the subsequent stages for everyone involved: the author response and reviewer discussion will be focused on the salient points, and your final meta-review will in most cases be a quick edit to your initial meta-review regarding if concerns were addressed as well as any expectations you have on simpler fixes (e.g., “authors should be sure to include references to xx and yy in the related work for the camera ready”).

In return for this investment of effort, we aim to foster a more thoughtful, humane, and focused review process: authors will know from the start what are the key concerns, and reviewer discussion will also be focused on these key concerns. 

Responsible AC’ing Initiative

NeurIPS 2025 piloted a responsible reviewing initiative in which reviewer-authors were asked to match their commitment to their reviewer role as their author role. Following a successful pilot, we are expanding that initiative to AC-authors as well. We believe that all our community members responsibly engaging in their program-related duties is essential to maintaining the quality of the conference as we grow. 

The 2025 responsible reviewing initiative had two components: First, reviewer-authors were required to submit their reviews before they could see their own, encouraging punctuality. Second, we reserved the option to desk reject, at the meta-review stage, papers submitted by reviewers with particularly low-quality reviews as flagged by the AC, encouraging quality. Together, these policies were also designed to support ACs to spend more effort on managing discussion rather than chasing reviews. 

As we expand this initiative to ACs, we recognize that while most ACs are highly thoughtful, there are instances where authors feel like feedback lacks synthesis or clear communication, sometimes an unintended signal of insufficient engagement. 

Following the model of the responsible reviewing program, reinforces that professional participation is vital to the high standards of NeurIPS, we are now piloting a similar initiative for ACs. This pilot introduces two measures designed to ensure the same level of quality and accountability at the Area Chair level:

Withholding Mechanism. ACs that are also authors (along with their co-authors) will not have access to the reviews or meta-reviews of their own submissions until they have completed all assigned meta-reviews. As NeurIPS 2026 introduced pre-response meta-reviews, these must be submitted on time to maintain access. If these are delayed, access will be suspended until the AC’s tasks are complete (at latest, until two days before the author rebuttal period concludes). It is the ACs responsibility to coordinate with their SAC regarding extenuating circumstances including non-responsive reviewers.

Desk Rejection, Future Ban, or other Sanctions.  Our goal is to support ACs as they navigate this new workflow. To that end, SACs will act as mentors throughout the pilot. That said, we also reserve the option to address highly inadequate professional engagement. In instances where an AC, despite attempts at support and guidance, submits meta-reviews of demonstrably poor quality or fails to engage with reviewers, the Program Chairs reserve the right to implement sanctions. Such cases will be reviewed by the Scientific Integrity Chair and Ethics Board to determine appropriate actions, which may include desk rejection of the AC’s own submissions or future participation bans.

With these two new safeguards, we expect a significant portion of papers to receive more thoughtful and consistent consideration. We view the AC role as a vital partnership with NeurIPS, and we thank you for your commitment to a process that improves outcomes and fosters a more supportive environment for the entire community.

AC Discussions on Subcommunity Standards

To better support our ACs, we will endeavor to assign papers within a single subject area, enabling chairs to better calibrate contributions across their track. Additionally, prior to the submission deadline, we will host both synchronous and asynchronous forums for ACs to discuss and define the core standards for their respective subcommunities. Aligning these expectations early, we aim to foster more consistent evaluation standards and improve calibration across the entire program.