General policy
Reviewers are at the center of the peer review process, and as such play a fundamental role in the community. In order to maintain the highest standard of reviewing, we encourage reviewers to read and follow the following recommendations:
- Reviewers should preserve their anonymity. The Conference Management System in which the authors uploaded their contribution (Conftool) already ensures that reviews are double-blind. Reviewers should avoid providing any hint about who they are: names, affiliations, or excessive references to specific papers and projects.
- Citations to preprints are not required and omitting such reference is not grounds for rejection (see ISPRS policy on preprints in public repositories are available at the following link) . Posting to arXiv is also not considered as a violation of anonymity. A reviewer who recognizes the authors of a paper because they had read the preprint should ask themselves whether they can still perform an unbiased evaluation. If the reviewer recognizes a possible conflict of interest (e.g., identifying the authors as close collaborators), they should immediately signal it to their Area Chair.
- The reviewer should treat the papers equally whether or not they know (or think they know) who are the authors. The reviewers should not try to find who the authors are by searching the Web for preprints.
- The reviewer has the responsibility to protect the confidentiality of the ideas exhibited in the papers he/she reviews. The work is considered new or proprietary by the authors. The reviewer is not allowed to show the paper, any results, code or supplementary material to anyone else, unless he/she has asked them to write a review, or to help with the review. The reviewer should not use ideas/code to develop his/her own ideas/code.
- If the reviewer thinks a paper may be a dual submission or suspected of plagiarism, please contact the Program Chairs as early as possible.
Reviews are available to other reviewers, AC, PC, and the authors of the papers.
Evaluation criteria
The evaluation sheet is composed of two main parts: (1) an overall recommendation score (50% of the total), and (2) a list of criteria (50% of the total), for which each reviewer should provide a score and a detailed evaluation (see below). The advantage of such a list is that all criteria are of similar importance. They are weighted equally so as to provide a final mark that is consistent whatever the scope of the paper (method-driven or application-driven).
The overall recommendation (and associated confidence score) and the technical evaluation of all reviewers are all taken into account in the AC’s decision for acceptance/rejection and for the selection between long oral, short oral or interactive presentation.
Five main questions compose the review form. The criteria slightly differ depending whether a full paper or an abstract is submitted.
- Innovation:
o Full paper: Does the paper contain original/innovative work (on theory and/or methods and/or applications)? Has related work been properly credited?
o Abstract: Does the abstract indicate original/innovative work (on theory and/or methods and/or applications)? Is there any mention of existing approaches?
- Scientific formulation:
o Full paper: Are the hypotheses clearly stated? Are the research questions correctly formulated? Are the employed methods appropriate? Are equations / methodological frameworks correctly presented / justified?
o Abstract: Is the abstract informative? Does it contain information about the hypotheses, the research questions, the methods employed and their relevance?
- Experiments and validation:
o Full paper: Do the authors properly reflect on the achieved performance of the proposed methods or procedures? Are facts and interpretation duly separated? Are results assessed in a qualitative and quantitative way? Are the interpretations and conclusions justified by the data?
o Abstract: Do the authors properly consider the expected performance of the proposed methods or procedures, and propose meaningful interpretation?
- Relevance:
o Full paper and Abstract: Does the subject fit within the call for papers? Do the results have a practical relevance? Will results impact other research? - Presentation:
o Full paper: Is the paper well written and well structured? Are figures well readable? Are the references complete? Is the paper formatted according to the author guidelines?
o Abstract: Is the abstract well written? Do the authors provide some references and an illustration of their work? Is the abstract formatted according to the author guidelines?
Full papers
In addition to rating the papers according to the five criteria, reviewers are asked to explain their evaluation in a detailed and clear manner and justify the scores above. The comments should be specific since a discussion is more helpful than scores, both for authors and for the final decision. We recommend pointing out strengths and weaknesses of the submitted contribution. In addition, reviewers should provide suggestions for improvement for the camera-ready paper and use an objective and constructive writing style. References to significant missing papers should be provided, excluding as much as possible papers signed by the reviewers themselves or their close collaborators.
The reviewers can signal borderline papers (i.e., leaning towards reject) to the ACs by checking the Major Revision Required checkbox. For such papers, the reviewer must list in a dedicated text field the major changes needed for the camera ready version of the paper to reach acceptance standard. This option should be used sparingly, and reserved for papers with serious but fixable flaws such as an improper evaluation protocol, significant mistakes in formulas or their interpretations, or failure to mention a previous research crucial for correctly framing the paper. All other comments should stay in the main body of the review.
Reviewers should also provide detailed comments regarding minor issues they have spotted, such as typos, erroneous sentences, missing references, or formatting problems Minor flaws that can be easily corrected should not be a reason to reject a paper.
Innovation and stimulation are core elements in an ISPRS event. Acceptance or rejection decisions should not be advocated solely by the performance of the proposed methods. It is not mandatory for the authors to propose methods that exceed the accuracy of the state-of-the-art or that have been tested on specific benchmark datasets. Nevertheless, papers should be technically correct and should propose a novel contribution to their field.
Major revision required checkbox
Abstracts
Once accepted, abstracts will not be reviewed again. Therefore, the degree of maturity of the work should be clearly assessed in the reviewer score sheet. This is absolutely necessary so as to build a consistent program. Suggestions on improvements, method comparisons or references to existing works should be provided in the review.
Finally, the reviewers should note that at the stage of the submission, experimental results may not be available and fully analyzed. Therefore, the reviewers should not focus their evaluation on the ”Experiments and validation” criterion.