CHAIR HOW-TO: Manage Review Aggregates

Note

The Chairs may want to see an aggregate of answers to valued-review questions in the Chair Console.

OVERVIEW

A valued-review question is a review question with a weight assigned to multiple choice answers.
The Reviewer answers these questions when giving their review of their assigned paper. The answers, in turn, show up as a numeric value in the Chair Console.
These aggregate values can also be configured to show in the Meta-Reviewer Console as well.

Single-Track Conference

Review Settings

• Go to the Review settings page and select 'Show aggregate columns.' Save the changes.

Review Questions

Important

Create review questions with a value.

Related: Review Questions

Reviews Completed

Once the Reviewer completes the review of a paper, the Chair will be able to see the review values in the Chair Console. In this example we have three valued-questions.
All three questions show in the Chair Console.

Limitations

There is a maximum of eight review-aggregated columns that are able to be shown in the Chair Console. These columns are shown in alphabetical order according to the review question.
This means if you have more than 8 value added questions, you will need to decide which ones will be shown on the Chair Console. Additionally, if the 10th question you want is at the end of the alphabet, even if it is question number 2 in the Review, it will not be visible in the Chair Console.

Workaround

The workaround is quite easy. You can use grouping symbols to mitigate the column-order. Grouping symbols are parentheses, brackets and braces and are used as they have a hierarchy.
For example, you have 10 value-questions with the last question, alphabetically, titled ‘Years Active.’
Since there are only 8 columns, this will not be seen in the Chair Console.
If you want this question to appear in the columns, wrap the title in parentheses so it looks like this – (Years Active).

Here is the hierarchy of the grouping symbols for CMT:

1. Parenthesis ( ) come first: The title will appear in the beginning columns, in alphabetical order.
2. Brackets [ ] will be second: All titles with brackets will appear after all titles with parentheses, but before braces, again in alphabetical order.
3. Braces { } will be third: This will put the questions after the first two grouping symbols, but ahead of ones without any grouping symbols – also in alphabetical order.

Note

If you use brackets and no parentheses, then those titles with brackets will be in the beginning columns.

You can see the hierarchy in these 8 questions from left to right. Also, each grouping is in alphabetical order.

Multi-Track Conference

Follow the same steps for a Multi-Track Conference as a Single-Track Conference.

Scenarios

The below scenarios are for viewing aggregates in the Chair Console.

1. If you have six review aggregate questions in Track 1, and two in Track 2, and they are not duplicate questions in both tracks, then they all will show in the Chair Console.

2. If you have six review aggregate questions in Track 1, and five in Track 2, again all different questions, then the last three questions at the end of the alphabet will not be visible. It doesn't matter if the last three questions are in Track 1, Track 2, or spread across both.

3. If you have six review aggregate questions in Track 1, and five in Track 2, but all of them are the same in both Tracks, except Track 2 is missing one, then six questions will show.

4. If you have six review aggregate questions in Track 1, and five in Track 2, with three questions the same in both Tracks, then you will have eight columns shown. This will be all of the questions.

5. If you have six review aggregate questions in Track 1, and six in Track 2, with three questions the same in both Tracks, then the question that was at the end of the alphabet will be the one not visible, regardless of Track.

For Track Chairs

In Scenarios 2 and 5, the Track Chair can see those questions not visible to the main Chair Console.
This is because the main Chair Console shows everything for every Track, while the Track Chairs only see what is in their particular Track.

Duplicate Columns

Important

It is very important in a multi-track conference to make sure there is no white space when duplicating the questions, otherwise there will be two of the same column in the Chair Console.

The following is an example of why a duplicate column may show in your Chair Console, and how to fix it.

Track 1 has this question:

Track 2 has the same question, albeit with white space (gray area):

The result is the Chair Console will show 2 different columns for what is perceived to be the same question:

To rectify that, just remove the white space from the Edit Question text field in Track 2.