Area Chair Guidelines

Thank you for serving as an Area Chair (AC) for the first ever CCN Proceedings track!

In order to create a review process that yields the highest quality, as a grass-roots organization, we have adopted a system that is more common at machine learning conferences.

As this is the first CCN Proceedings track, the timeline and guidelines may evolve. We appreciate your understanding and patience as we develop this new process for CCN.

Role

As AC, your job is to ensure that all the submissions you are assigned have high quality reviews and good discussions. You should become familiar with the contents of all your submissions and are responsible for making the initial decision recommendation in the form of a meta-review, with guidance from your senior area chair (SAC).

OpenReview Console

The OpenReview console provides an overview of all papers assigned to you. Use the console to track review progress, deadlines, and discussion status. The console allows you to filter papers by status (e.g., "Missing Reviews," "Ready for Decision").

The console points to the forum page for individual submissions, where you can view details, post comments, and engage in discussions specific to a single submission. For a quickstart on how to use the OpenReview forum page, see Using the New Forum Page.

Setting comment visibility

To set comment visibility in OpenReview, use the "Readers" dropdown when posting your comment. Check carefully that the appropriate people can see the comment before posting. For example, to engage in private discussion with an AC or SAC, the comment should be set visible to the AC or SAC and "Program Chairs" only. (The "Program Chairs" on OpenReview are the CCN TPC.)

Timeline

Assignment

review-period

We have done our best to match ACs with the most appropriate submissions. However, if you find that a manuscript assigned to you is too far outside of your area of expertise, or if you recognize a potential conflict of interest; please let your SAC or the TPC know straight away so that we can re-assign the manuscript by posting a comment on the OpenReview forum for the submission with the readers set to your Senior Area Chair and the TPC (see Setting comment visibility).

In addition, please make sure that your OpenReview profile has your latest email listed and keep an eye on emails possibly landing in spam.

Also, make sure that every submission in your batch is matched with suitable reviewers whom you can trust on this submission, ideally with a diverse set of opinions.

Reviews

review-period

As an area chair, you will oversee the review process for the submissions assigned to you and address any potential concerns raised by the reviewers.

To ensure the review process stays on schedule, please keep an eye out for any hold-ups or delays on the reviews. Please make sure to follow up with reviewers and find replacements if necessary.

Assist reviewers who may feel poorly matched to submissions. Address reviewer concerns and assign additional reviewers if necessary to ensure thorough evaluation.

Reviews are due reviews-due. Read all reviews carefully. If a review is substandard, you should ask the reviewer to improve their review. Please remember to be polite and provide concrete guidance. CCN expects the highest quality in its reviews. Please encourage this standard in the reviewers you oversee.

Please make sure reviewers complete their reviews on time.

Things to flag

Reviewers are instructed to look out for the following in their submissions. If you note any of the below, please escalate them with your SAC or the TPC by posting a comment on the OpenReview forum for the submission with the readers set to your Senior Area Chair and the TPC (see Setting comment visibility).

Breaches of anonymity

If you find that the identity of the authors is revealed (e.g., names or affiliations can be found in the text, or in included or linked supplementary material), please flag this to your AC or SAC.

Ethical concerns

This includes for instance harm, injury, or unfair bias. If you notice unethical behavior involving authors or reviewers assigned to you, please notify your SAC.

Dual submission

Submissions that are identical or substantially similar to papers that are under review, have been accepted to, or have been published in other archival conferences and journals should be deemed dual submissions.

Discussion

discussion-period

During the post-review discussion period, you will facilitate a constructive discussion between the authors and the reviewers through the OpenReview platform. Please moderate this discussion to ensure that it is productive and respectful of everyone’s opinion. Also, try to steer the discussion towards critical aspects of the submission that are helpful for its assessment. Please make sure reviewers update their reviews and assessments based on author responses. Also, make sure that authors address critical concerns raised by the reviewers. You can follow up with authors / reviewers publicly or privately by restricting the visibility of your comments in the OpenReview platform.

Meta-reviews

meta-review-period

After the discussion period, you will be responsible for writing meta reviews to summarize the reviews and the discussion that followed. You can reach out to the reviewers or the SAC for further comments and clarifications. Your meta review indicates your decision recommendation (accept/ reject) and your justification for the decision. Please address and interpret any disagreements within the reviewers in your meta reviews and deliver a decisive recommendation with the help of the SAC. Please ensure to deliberate with your SAC about the borderline cases as well as ones where you are making a recommendation that goes against the reviewer recommendations. The SAC will help you calibrate the decision criteria across other ACs.

Please take a holistic approach when writing meta reviews. Please take into account the strength of reviewer concerns and the quality of reviews in addition to reviewer scores or confidence estimates.

Acknowledge the author’s response, even if it didn’t change the reviewer’s concerns. If you find yourself wanting to overrule a unanimous opinion of the reviewers, the standards for your summary should be at the level of a full review. Other than papers where there is a genuine disagreement, much of our work will involve borderline papers where no one confidently expresses excitement, nor are any major problems identified. These are the tough decisions where we need your judgment!

Meta-review format

As an AC, we trust you to make an informed recommendation based on sufficient knowledge and justified analysis of the paper, and to clearly and thoroughly convey this recommendation and reasoning behind it to the authors. We ask you to provide the following in the meta review form on OpenReview:

1. Your recommendation: “Accept to Proceedings” (accept full paper) or “Invite to Extended Abstracts” (reject full paper; convert to 2 page format).

2. A concise summary of the engagement between the reviewers and authors (including the reviews, the author response, and the ensuing discussion), highlighting the key strengths and weaknesses raised by the reviewers surrounding Interest, Soundness, and Clarity, the responses provided by the authors, and any evolution in the reviewers’ evaluation that occurred through the exchange.

3. Your meta-review, where you describe if the reviewers’ points were sufficiently addressed by the authors and whether you as the AC found the reviewers’ points worthy of consideration in decision-making. The meta-review should end with a clear justification of your recommendation on the basis of the content in and the engagement around the submission (i.e., the points you described above).

Example meta-reviews

TPC has crafted two generic examples below to demonstrate the content and level of detail expected in a meta-review. However, we encourage you to be more specific about the details of the claims and evidence presented in the submission, as the below are deliberately vague (i.e., referring to “points”, “ideas”, “evidence”, rather than, for example, “… the absence of statistical tests for the main effect of stimulus complexity on reaction time, that weakens the central claim of the submission”). The latter is much preferred, both to make the meta-review more verifiable by your SAC and the TPC, and more actionable for the authors for the next revision.

Example 1

CCN Recommendation: Invite to Extended Abstracts

Summary:\ Both Reviewer 1 and 2 acknowledged the novelty and potential broader impact of the paper. However, they also indicated that a cohesive narrative was missing as well as some critical statistical evidence for the claims made, and made several suggestions to improve the soundness of the manuscript. The authors replied to most of these in a point-by-point fashion, rewriting certain sections to improve clarity and adding one new analysis, to which Reviewer 1 responded that the manuscript has been improved but that their rating remains the same. Reviewer 2 also noted that the revised paper had improved but reported that their initial concern regarding the packing in of too many ideas in one paper still stands. The authors replied again to further explain the main goal and indicated that they would address this in the discussion in a further revision. Reviewer 3 raised the same point as the other two reviewers regarding the clarity of the narrative, and was overall more critical of the paper’s soundness, while also mentioning several concrete examples of inconsistency in the results. The authors responded to some of these, but the reviewer indicated that the author’s response did not (fully) address their concern.

Metareview:\ While all reviewers acknowledged and appreciated the broad interest of the study, they also expressed a strong concern that the high number of ideas and analyses decreased the clarity and soundness of the manuscript. The AC had the same concern and believes this study is more appropriate for a longer format paper in which the authors have the space to fully explain and motivate all their different analysis steps. In its current form, it does not seem suitable for the CCN Proceedings format. This decision was discussed with and approved by the SAC.

Example 2

CCN Recommendation: Accept as Proceedings

Summary:\ Three out of four reviewers indicated that both the soundness and clarity of the paper was at least adequate and two reviews judged both of these as strong upon submission. In addition, all reviewers indicated that the interest of the paper was disciplinary or broad. One reviewer, with the lowest confidence (uncertain), indicated that the soundness of the paper needed improvement and suggested additional analyses to determine to what extent the results were specific to one experimental condition or all conditions in general, and to characterize the essential network involved in generating the effect. The authors replied by explaining why the suggested analysis on condition was not feasible, adding more discussion regarding this point in the text and by implementing a new analysis to address the reviewer’s second point. The reviewer thanked the authors for their responses and increased their soundness rating to adequate.

Metareview:\ There seems to be a general consensus amongst three out of four reviewers that the findings are sound and are communicated clearly. Several reviewers indicated that certain methodological details were missing, and in response these were added by the authors in the revision. The authors also added new analyses as well as additional discussion in response to comments made by the reviewers. Taken together, the overall assessment of the reviewers was positive and therefore, the AC is happy to recommend acceptance.

Policies

Availability

Respect deadlines and respond to emails as promptly as possible. Make sure that your preferred email address is accurate in your OpenReview profile and that emails from noreply@openreview.net don't go to spam. Please ensure your availability and engagement during your active periods of work. If you will be unavailable (e.g., on vacation) for more than a few days during important windows (e.g., decision-making), please let the TPC know as soon as possible.

If you are unable to meet these expectations, please let your SAC know of your constraints by posting an Official Comment.

Kindness

It is important to acknowledge that personal situations may, in rare instances, lead to late or unfinished work. If you find yourself unable to complete your work on time, please communicate this as soon as possible. If you oversee others who are unable to complete their work on time, we encourage you to be considerate of the personal circumstances and be ready to pick up slack in such cases. If necessary, make a back-up plan, and be flexible to the extent possible. In all communications, exhibit empathy and understanding.

Conflicts of interest

A conflict of interest arises when an author on one of your assigned submissions is a current or former advisor, family or a close personal relationship, a current or recent collaborator; or someone who works in your current or recent immediate organization, or when you have a financial interest in the work.

If you recognize a potential conflict of interest, please let your SAC or the TPC know straight away so that we can re-assign the manuscript by posting a comment on the OpenReview forum for the submission with the readers set to your Senior Area Chair and the TPC (see Setting comment visibility).

Confidentiality

Do not discuss, distribute or use ideas, content or code of the submissions. Reviews are double-blind; authors and reviewers do not know each others' identity. Maintain strict confidentiality for all review materials. Don't use or share submission content (ideas, results, code) until publicly available. Never distribute submissions outside the OpenReview platform.

For reviewers: The use of LLMs or other automated tools is prohibited for generating review text or summarizing submissions. Inputting a paper submission into such a tool is a violation of confidentiality.

Conduct

Abide by the CCN Code of Conduct. Take part in an active, polite and constructive manner.

Transparency

Please note that all reviews and meta-reviews of accepted papers will be made public.

Your reviewers and your SAC know your identity. In general, your primary point of contact for any discussions should be your SAC. Your SAC does not have any conflicts with any of the submissions that are assigned to you.

Flexibility

Despite our best efforts to plan this year's process, this is very much a new endeavor and therefore the timeline and guidelines may shift now and then. So please keep an eye on our communications, and we ask for your understanding and patience as we keep developing CCN.

Contact Info

If you encounter a situation that you are unable to resolve on your own, please contact the Technical Program Committee (TPC) at tpc@ccneuro.org.

If the issue is related to OpenReview technical issues, email the OpenReview support team directly at info@openreview.net.

If you have specific questions related to the handling of a particular paper, please leave a comment on the OpenReview forum for that paper with the readers set to the Senior Area Chair and the TPC.