Özer Tanrısever, Aligning Reviewer Guidelines and Reviewer Feedback: A Data-Driven Study
This study proposes a framework for investigating the alignment between institutional reviewer guidelines and peer review practice. A dataset was constructed from the ICLR 2024 venue on OpenReview. Large Language Model (LLM) pipelines were utilized to extract reviewer inquiries, perform topic modeling, and apply soft classification. These outputs were compared against the ICLR 2024 reviewer guideline to quantify the guideline-practice gap. Generated topics are also mapped to the guidelines from the top ten AI venues. The proposed framework provides a data-driven approach to identify implicit evaluation norms for venue organizers and gives authors and reviewers a data-driven roadmap of implicit evaluation norms.
Date: 14.01.2026 / 14:30 Place: A-212









