arXiv:2109.09774 ?[ pdf , other ]? cs.DL cs.LG Inconsistency in Conference Peer Review: Revisiting the 2014 NeurIPS Experiment Authors: Corinna Cortes , Neil D. Lawrence Abstract : In this paper we revisit the 2014 NeurIPS experiment that examined inconsistency in conference peer review. We determine that 50\% of the variation in reviewer quality scores was subjective in origin. Further, with seven years passing since the experiment we find that for \emp...