March 22nd, 11:59PM ET: Deadline for submission
March 23rd: Area Chairs and reviewers bidding periods begin
March 25th: Area Chairs and reviewers bidding periods end
March 26th: Short Papers Track Chairs assign short papers to Area Chairs
March 27th: Area Chairs assign short papers to their reviewers
April 7th: Review period ends
April 7th: Area Chairs meta-review starts
April 14th: Area Chairs give back their meta-reviews and decisions
April 15th: Final decision, notification of acceptance
Authors were asked to take reasonable efforts to hide their identities, including not listing their names or affiliations and omitting acknowledgments. This information will of course be included in the final version. Reviewers should also make all efforts to keep their identity invisible to the authors. ArXiv papers are not considered prior work since they have not been peer reviewed. Therefore, you should review your short papers independently as if the arXiv papers didn't exist. Citations to these papers are not required and failing to cite or beat performance of arXiv papers are not grounds for rejection. An important general guideline is to make every effort to treat papers fairly whether or not they know (or suspect) who wrote them. Reviewers should not search for the authors of a paper, and complain that the paper is not anonymous if they happen to find them.
As soon as you get your reviewing assignment, please go through all the short papers to make sure that (a) there is no obvious conflict with you (e.g., a short paper authored by your recent collaborator from a different institution) and (b) you feel comfortable to review the short paper assigned. If either of these issues arise, please let us know right away by contacting your the Area Chair using CMT.
Minor flaws can be corrected and shouldn't be a reason to reject a short paper.
Acceptance and rejection decisions should not be determined solely by the method's raw performance. Rather, it is important to weigh both the novelty and potential impact of the work on the society alongside the reported performance. Impact can be on the research community, but also on the specific problem being addressed.
Each short paper that is accepted should be technically sound and make a contribution to the field.
It is important to note that the potential impact of a proposition is one of the most important criteria. Some problems do not require technical novelty, but instead thoughtful application of existing methods. Papers in that category (i.e. impactful but not necessarily novel) should not be rejected for this reason only.
Please think carefully about your reviews. In particular, it's a good idea to avoid ad-hoc policy innovations, which can occur with the best of intentions. Here is an example. Author submits a short paper relying on a dataset that cannot be public. Reviewer takes the position that this means the results cannot be trusted, and rejects on these grounds. The problem with this review is that it's clearly about a matter of policy, rather than about the short paper's content. We don't have a policy about non-public datasets, and it's unfair for reviewers to invent one of their own.
When you have finished with your review, you should destroy any short paper manuscript and/or supporting material you received. See the Ethics guidelines below.
Here are some specific issues to keep in mind as you write your reviews:
1. Protect Ideas
You have the responsibility to protect the confidentiality of the ideas represented in the short papers you review. Protection of the ideas in the short papers you receive means:
The following provides further details for reviewing papers:
No. Each paper should be evaluated in its own right. If you feel that most of the papers assigned to you have value, you should accept them. It is unlikely that most papers are bad enough to justify rejecting them all. However, if that is the case, provide clear and very specific comments in each review. Do NOT assume that your stack of papers necessarily should have the same acceptance rate as the entire conference ultimately will.Can I review a paper I already saw on arXiv and hence know who the authors are?
Yes. See next bullet below for guidelines.How should I treat papers for which I know the authors?
Reviewers should make every effort to treat each paper fairly, whether or not they know who wrote the paper. For example: It is Not OK for a reviewer to read a paper, think "I know who wrote this; it's on arXiv; they're usually quite good" and accept paper based on that reasoning. Conversely, it is also Not OK for a reviewer to read a paper, think "I know who wrote this; it's on arXiv; they're no good" and reject paper based on that reasoning.How should I treat arXiv papers?
ArXiv papers are not considered prior work since they have not been peer reviewed. Therefore, you should review your AISG workshop short papers independently as if the ArXiv papers didn't exist. Citations to these papers are not required and failing to cite or beat performance of arXiv papers are not grounds for rejection. For example:
Though the content of the submissions differ (proposal vs. paper), the reviewer guidelines for track 1 continue to apply. We make some additions to those guidelines due to the different form of track 2 reviews.Important Dates
March 22nd, 11:59PM ET : Deadline for submission
March 23rd: Open Proposal Reviews
March 29th: Screening reviews Due
March 30th: Proposal Pre-Acceptance
March 31st: Project Mentoring start
April 14th: Selection Reviews Due
April 15th: Notification of Acceptance