ICLR 2027 Review Atlas

Mean	Papers
1.00	855
1.17	1
1.20	6
1.25	109
1.33	3,131
1.40	14
1.50	335
1.60	31
1.67	7,304
1.71	1
1.75	545
1.80	53
1.83	3
2.00	12,079
2.17	4
2.20	70
2.25	725
2.33	8,563
2.40	52
2.50	605
2.60	24
2.67	4,228
2.75	216
2.80	10
3.00	1,388
3.20	4
3.25	10
3.33	238
3.50	6
3.67	28
3.75	1
4.00	3

What the data shows

Reviewers often disagree. Say two reviewers rate a paper 3 on average. The third reviewer gives it about 2.3 on average. If reviewers agreed, the third would also give about 3. Fig. 1
Luck plays a big part. Which reviewers a paper gets can move its average by about 0.36 points. That is almost as big as the normal spread between papers, 0.45. Fig. 1
More confident reviewers gave lower ratings. Reviewers say how sure they are of each review. The least sure reviews average 2.27, and the most sure average 1.88. Fig. 3
Papers added to OpenReview later got lower ratings. The first 5% of papers average 2.13, and the last 5% average 1.94. The effect is small: timing explains less than 1% of the differences in ratings. Fig. 5
Papers with fewer keywords rated higher. Papers with 2 keywords average 2.13, and papers with 8 or more average 1.90. That doesn't mean dropping keywords would raise a rating. Fig. 8
Topic makes a difference. Averages by area run from 1.92 to 2.15, a gap of 0.23. That is bigger than the gap between early and late papers, though within any area papers still go from 1 to above 3. Fig. 18
Soundness and informativeness line up with the rating most. Reviewers score these separately. Together with presentation and confidence, these scores explain 43% of the differences in ratings. Fig. 14
Most reviews are lukewarm. 57.4% of reviews give a 2 out of 4, which reviews label weak rejection. Only 23.5% give a 3 or 4. Fig. 27

How much reviewers agree

Keep this in mind for every ranking on this page: reviewers agree very little. A standard agreement score, the intraclass correlation (ICC), is only 0.16, where 1 is full agreement and 0 is none. So the average of three reviews is not very reliable, about 0.36 on the same scale.

Fig. 1

If two reviewers rate a paper highly, does the third?

For papers with three reviewers, this compares one review with the average of the other two. If reviewers agreed, the dots would follow the straight diagonal line. When the other two average 3, the third averages only 2.29.

Fig. 2

Most common rating patterns

Share of three-reviewer papers. A split such as 1, 2, 3 is about as common as 2, 3, 3.

Fig. 3

Rating by reviewer confidence

Reviewers also say how confident they are in each review. The more confident they were, the lower they rated.

Fig. 4

Gap between highest and lowest rating on a paper

About 18% of papers have a gap of 2 or more.

Later submissions rated lower

OpenReview numbers submissions as they are created, so a higher number means the paper entered the system later. Half of all submissions appeared in about the final two days. Each dot below holds 5% of reviewed papers.

Fig. 5

Mean rating by submission number

The earliest 5% average 2.13, and the last 5% average 1.94. Submission number explains less than 1% of the differences in ratings (Spearman correlation -0.09).

Fig. 6

Papers at 2.5 or above by submission number

The share falls from 21.0% in the earliest 5% to 11.6% in the last 5%. The rating trend also holds within primary areas (Spearman -0.07).

This shows association, and these are review ratings, not final decisions. Later papers also list more keywords, 3.75 on average in the earliest 5% against 4.97 in the last 5%, so timing may stand in for other differences between papers.

Titles and metadata

These are associations across 40,642 papers. They describe what kinds of paper rated well, and changing a title would not change a rating.

Fig. 7

Title length

Mean rating by words in the title, in groups of four.

Fig. 8

Number of keywords

Ratings peak at 2 keywords and fall steadily after that.

Fig. 9

Title style

Acronym-led titles rate slightly higher, and question titles slightly lower.

Fig. 10

Paper status

Fig. 11

Title words linked to higher ratings

Words in at least 200 titles.

Fig. 12

Title words linked to lower ratings

Audit and study papers sit at the bottom.

What moves a rating

Reviewers score soundness, informativeness, presentation, and their own confidence alongside the rating. These charts show how those scores relate to the rating and each other.

Fig. 13

Correlation matrix

Rating ties most closely to informativeness and soundness. Confidence runs slightly negative.

Fig. 14

Regression on the rating

Fig. 15

Mean rating by soundness and informativeness

Each cell is the mean rating for that pair of scores, with its review count on hover or tap. Both must be high for a high rating.

Fig. 16

Mean rating by informativeness and presentation

Good presentation lifts the rating at every level of informativeness.

Fig. 17

Rating mix at each score level

Each bar shows how ratings split among reviews giving that score. A soundness of 1 rarely comes with a 3 or 4.

Topics

Primary area comes from the submission form. Keywords come from authors, with common spellings merged. Dots show means with 95% confidence intervals.

Fig. 18

Mean rating by primary area

The line marks the ICLR mean of 2.06.

Fig. 19

Each area's profile across review scores

Cells show how far an area sits above or below the ICLR mean, in standard deviations across areas. This shows what reviewers liked or disliked in each area.

Fig. 20

Keyword landscape

Fig. 21

Highest-rated keywords

Keywords on at least 120 papers.

Fig. 22

Lowest-rated keywords

Reproducibility and evaluation methodology sit lowest.

Explore any paper or topic

Search titles and keywords across all reviewed papers. The chart compares your selection with ICLR overall.

Search titles and keywordsPrimary area

Fig. 23

Selection against all of ICLR

How papers were rated

How ratings fell across all reviewed papers, on a scale of 1 to 4.

Fig. 24

Mean rating per paper, all ICLR

Count of papers at each exact mean. Hover or tap a bar for its share of papers.

Fig. 25

Distribution by area

All papers against the highest-rated and lowest-rated primary areas, in bins 0.25 wide.

Fig. 26

Papers at or above each mean

Read this as a ladder: 16.6% of all papers reach 2.5. The two areas are the same as in Fig. 25.

Fig. 27

Individual review ratings

Share of reviews at each rating.

Fig. 28

All five review scores

Reviewers score four other dimensions. Among the three quality scores, presentation runs highest and informativeness lowest. Those three stop at 3.

Method and limits

The data covers official reviews visible on OpenReview on 6 November 2026, including 361 reviewed papers that were later withdrawn. Papers with no reviews are left out of the charts. There are 1,682 desk-rejected and 308 unreviewed withdrawn submissions.

The rating scale is inferred from the data, where only 1 to 4 appear. Reviews label 1 as clear rejection, 2 as weak rejection, and 3 as weak acceptance. No label for 4 appeared in the reviews.

Agreement is a one-way intraclass correlation (ICC) for a single reviewer. Reliability for three reviewers uses the Spearman-Brown formula. Regression uses standardized variables, so the coefficients compare directly. Confidence intervals use a normal approximation.

Everything here shows association. Nothing here shows that a feature causes a rating.

Submission numbers were read from OpenReview on 7 November 2026 and rounded to the nearest 256 for the submission order charts, so the numbers on those charts are approximate. Dates are estimated from the creation times of nearby submissions.