Guidelines

Short paper track aka. Track 1

This page provides detailed guidelines to Area Chairs of ICML 2019 AISG workshop as to how the process will unfold and what their responsibilities are.

We wish to acknowledge the following as sources of inspiration for the following guidelines:

• CVPR 2014 guidelines for Area Chairs.

Important dates

April 26th, 11:59PM ET: Deadline for submission
April 23rd: Area Chairs and reviewers bidding periods begin
April 25th: Area Chairs and reviewers bidding periods end
April 26th: Short Papers Track Chairs assign short papers to Area Chairs
April 27th: Area Chairs assign short papers to their reviewers
May 7th: Review period ends
May 7th: Area Chairs meta-review starts
May 14th: Area Chairs give back their meta-reviews and decisions
May 18th: Final decision, notification of acceptance

Contact information

aisg2019.icml.contact@gmail.com

Role of the Area Chairs

The Area Chairs (AC) identify suitable reviewers for each paper, write meta-reviews summarizing discussion at least for short papers with diverging reviews, evaluate the reviews and suggest acceptance/reject decisions for each short paper. ACs are expected to play an active role in ensuring higher quality and more timely reviews.

Best Practices of Being an Area Chair

We want to remind all ACs of some of the basic (and obvious) practices of being an AC.
You have a strong influence on the decision for a short paper. Please, take your job very seriously and be fair. If you identify any short paper that you believe may be written by a current or former student, post-doc or supervisor, or by a collaborator in the last two years, please notify the Short Papers Track Chairs using CMT immediately so that the short paper can be assigned to another AC.
The Short Papers Track Chairs did not submit any papers, so they are not in direct conflict with any papers and they know all authors of all papers and also all reviewers/ACs. They will NOT be involved with any decisions of short papers from institutions with which they are in conflict. You can send them a question about any short paper and if one of them is conflicted, another one will respond to you without including the conflicted Short Papers Track Chairs.
DO NOT talk to any other AC about short papers assigned to you without prior approval from the PCs as there may be several other ACs conflicted with the short paper.
DO NOT talk to any other AC about your own short paper (the short paper you are an author on) or a short paper you have some conflict with, during this whole process.
Remember, it is not fair to dismiss any review without looking at other reviews and reading the short paper yourself to make an evaluation. Outright rejecting a review is not acceptable without a clearly articulated supporting argument.

Meta-reviews summarizing the short paper

Area Chair meta-reviews are the MOST CRUCIAL aspect of the review process of the ICML 2019 AISG workshop. This is where the Area Chair justifies his/her recommendation to accept/reject a paper. The Short Papers Track Chairs will read all meta-reviews and we have committed ourselves to ensure that these meta-reviews are representative of a good quality review process. These meta-reviews should highlight why the decision was reached. If all reviewers agree on a paper, this meta-review can be simple, but feel free to encourage authors and provide constructive feedback. If there is even a slight disagreement on the reviews, it is your job to clarify why and how the disagreement was resolved. Overruling all three reviewers to make a decision on a paper is not acceptable. If you strongly feel that the paper deserves that, make sure to discuss it with the Short Papers Track Chairs using CMT. In such cases, (a) you should read the short paper completely, and (b) provide a detailed meta-review. The Short Papers Track Chairs will review and confirm or reverse any such decisions.

Meta-Review Form

• Feedback to Authors
  For short papers with diverging reviews or short papers where you disagree with the reviewers, please summarize the key things you would like the raise. There is no need to summarize the paper or reviews. If you have additional concerns that were not included in the reviews, please be sure and include them as well.
• Preliminary Rating
  Please rate the paper according to the following choices:
  
  • Oral: These are short papers whose quality is in the top 10% of the papers at the workshop.
  • Poster: These are strong short papers, which have more than one weakness. For example, a well-written short paper with solid experiments, but incremental; a short paper on a well-studied problem with solid theory, but weak experiments; or a novel paper with good experiments, but poorly written.
  • Reject: These are papers that have some promise, but they would be better off by being revised and resubmitted, or have a major flaw. Reject decisions will be reviewed by an Short Papers Track Chairs.

Please also be on the lookout for one paper to be considered for the best paper award.

Project Proposals track aka. Track 2

Though the content of the submissions differ (proposal vs. paper), the Area Chairs guidelines for track 2 differs from track 1.

Important Dates

April 26th, 11:59PM ET : Deadline for submission
April 23rd: Open Proposal Reviews
April 29th: Screening reviews Due
April 30th: Proposal Pre-Acceptance
May 1st: Project Mentoring start
May 14th: Selection Reviews Due
May 18th: Notification of Acceptance

Project mentorship

• Though it is not the mentor’s responsibility to connect projects with volunteers, they should keep in mind how to make proposals accessible to potential volunteers drawn from the workshop participant pool.
• Since this review process involves speaking with project proposers, it is important to be courteous to those whose projects you are evaluated. Don't condescend to people who may not be machine learning specialists, and respect the time set aside for you to understand the project and whether it is a good fit for conference volunteers.
• Offer suggestions. If you can see opportunities to improve the impact or feasibility of a project, you should voice your perspective in a respectful and constructive way.