My take on writing rebuttals.
by Niklas Elmqvist, University of Maryland, College Park
Many conferences (notably CHI) have a two-round review process where the authors get to see a first version of the reviews and are allowed to write a rebuttal to defend their paper against the comments of the reviewers. Writing a rebuttal that will make a difference and actually “save” a paper is challenging. In this document, I will explain my thoughts on how to write such a rebuttal.
What Is a Rebuttal? What Is It Not?
It is important to differentiate a rebuttal from a response letter for a journal paper review. A conference paper is submitted “as-is”. Even though the authors can fix a few problems in the paper for the final, camera-ready version of the paper, there is no way for the reviewers (at least the externals) to check on this. Therefore, authors can never propose to do major changes to a paper in the rebuttal. Reviewers will not accept this, and the paper will be rejected. You can propose to do a few things, and generally on the order of presentation or organization.
Instead, the mindset that you should have when writing a rebuttal should be “our paper already has all the information you are asking for, it is just a matter of looking.” This may not be strictly correct, but you cannot admit to a major problem in a rebuttal because that means your paper will inevitably be rejected (if you have a major problem, you probably should not be submitting a rebuttal anyway). Use this tone in the whole rebuttal. It is a fine line not to sound arrogant, which is why writing an effective rebuttal is so challenging.
Organizing Your Rebuttal
How do you structure your rebuttal? After poring over the reviews incessantly, you likely have your own views on the main concerns raised by reviewers, so one way to organize the rebuttal is to simply use your perception as a guide. A common temptation as an author is to write a self-contained essay about your paper that responds to all of the criticism in a neat and high-level manner. However, as a reviewer, I am usually involved in writing reviews for many papers, so when I receive a rebuttal I am often struggling to remember details of the paper that I read several weeks ago. I find that such “essay rebuttals” are difficult to penetrate because I often have a hard time seeing what the authors are responding to, and what my original feedback was. In the quest for elegance, the purpose of the rebuttal is easily lost.
Instead, I find that the easiest and most effective way is to start with the reviewers, and to make your rebuttal a pragmatic and directed document. A smart author will use the reviewers’ own words to help them remember their feedback, and will then respond to that feedback directly. If the primary reviewer (1AC) gave a list of concerns to respond to, that smart author will use those exact concerns, and in that exact order, to structure the rebuttal. If the primary numbered and/or named the issues, even better; use those names and numbers! Don’t be afraid to include headings, bullets, and direct references to reviewers. Favor brevity and directness in your responses.
What to Cover in Your Rebuttal?
To decide what to include in your rebuttal, you have to first realize that your space is limited—often only 5,000 characters (for CHI). For this small amount of space, you will have to make a decision on what to address and what not to address. Here is a useful rule of thumb:
Address all major problems in your rebuttal—except those that you cannot address.
This is a little controversial, but the message is clear: you need to address the main problems that reviewers had with your paper, but you cannot address the problems that are so dire that it threatens your whole paper. If you do not have a plausible reason why these problems are irrelevant, maybe your paper should not be accepted and you should not be writing a rebuttal in the first place.
Below I cover some additional guidelines in detail.
DO Propose Minor Changes
One of the ill-kept secrets of the CHI community is that rebuttals can be used to promise a limited set of specific changes. For example, if the reviewers are asking for a missing piece of information, you can furnish that information (at least in abbreviated form) in your rebuttal and then explain where in the paper you will insert it. Be specific. This is even possible for what could be seen as new research; for example, I have seen rebuttals where authors promise to add subjective spoken feedback from the experiment participants, another statistical test, or even a missing section in the related work. Every reviewer is different in what they will let you get away with, but it (usually) doesn’t hurt to try.
This should go without saying, but when explaining misunderstandings and promising changes in your rebuttal, remember to also actually make those changes in your camera-ready if your paper is accepted. Yes, there is no way of checking this for the reviewers, but it should be a given. What’s more, I have had situations where authors think that explaining misconceptions in the rebuttal is sufficient. It’s not—misunderstandings are often symptoms of greater problems in your writing, and you will likely have to improve your paper and not just explain it in the rebuttal. The rebuttal benefits only three or four reviewers, whereas your paper may be read by many more people. Don’t just do your reviewers a favor, do your entire readership a favor.
One more thing: it is easy to overlook that when you add something to an already full-length paper, your paper may go over the page limit. For this reason, in addition to describing what you want to add, reviewers will respond well to a clear plan of what you will remove to make space for the new content, and why the paper will still work. It shows that you are not just blindly trying to please your reviewers, but that you are thinking realistically about your paper.
Do NOT Insult the Reviewers
Be sure to never use language that suggests that the reviewers are at fault for misunderstanding your paper. It is your responsibility as an author to communicate your scientific findings clearly. At the same time, you should not be apologizing too much (that is defeatist language, see below). Instead, state clearly and succinctly that there is a problem due to misunderstanding, and then give the right reasoning in your rebuttal. You need to do this last step. It is not enough to say that the reviewers misunderstood the paper, you must also explain what they misunderstood.
Also, be sure to thank the reviewers (but don’t be creepy), and also to acknowledge good remarks.
Do Mind Your Language
Writing a rebuttal is often about getting into the head of the reviewers and to give them a feeling of your work being a worthwhile contribution. This will not work if you use defeatist language. Use powerful and positive language that suggests that there are no major problems with your work, and that the points that reviewers raise require only minor clarifications. More specifically, do not use words like “flaw”, “problem”, “mistake”, “unoriginal”, “incremental”, and so on. Instead, use “weakness”, “issue”, “mishap”, “limited”, “extending existing work”, and so on. This may sound like a joke, or even underhanded, but it is important. The tone of your rebuttal is vital in getting your reviewers to think positively about your paper.
Reviewers sometimes put forth statements that are demonstrably false or at least misguided (unknowingly or not). Do not respond in kind, do not fly off the handle. Avoid sarcasm or direct confrontation. Do not be snarky or passive-aggressive. Instead, answer all issues with facts. Cite references (preferably ones already in your paper) to answer problems raised by reviewers. Collect your arguments and present them one by one. Try to dispute claims about your work with overwhelming evidence to the contrary.
Sometimes it is not worth addressing all reviewers in your rebuttal. Some reviewers may be over the top, either with a high or a low score, and you know that you will not be able to address that reviewer (or addressing them would not be worthwhile). In such cases, save your ammunition for the reviewers who are on the fence. These are the ones who are likely to tip your paper in a specific direction, and they are thus the ones you need to convince. However, do not ignore any reviewer (see below).
Remember that your rebuttal is as important as your paper in terms of correct grammar and spelling. In fact, because of the limited space allowed for a rebuttal, as well as the important persuasive message you are trying to convey, a rebuttal must be even more carefully crafted than your paper. Don’t let a typo or grammatical error slip by your review.
Address Every Reviewer by Name (Number)
Reviewers are generally pleased when seeing their names (or numbers, as it were) in an author’s rebuttal. It means that you are paying attention to their feedback. Be sure that your rebuttal mentions each reviewer at least once (specifically, not just “all reviewers”) and discuss at least one of their points. Acknowledge the comment and discuss how your paper answers it. Your reviewers will be happy that you took the time to answer their concerns and may feel more positive about the paper.
Urging, not Begging
Lastly, I am always a little leery of authors who write a rebuttal and ask for the reviewers to change their rating. At the same time, if the authors did not want this, they are not serious about getting their paper accepted. In other words, as a reviewer, I expect the authors of papers that I reviewed to be passionate about getting their work accepted. If they are not passionate, maybe their paper should not be accepted, so I am not against authors urging reviewers to reconsider (especially if they have a valid reason for that).
However, at the same time, there is a difference between urging and begging. Never ever beg. It demeans you and your work, and there is no chance that your paper will be accepted. Use factual, clear, and forceful arguments why your work is important, but never appeal to someone’s compassion or sympathy. It will not work.
Other Rebuttal Guides
I am not the only one who has written about writing rebuttals. In fact, writing about writing rebuttals is a common blog topic! Here are few of these articles (let me know and I will add to this list):
- Writing CHI Rebuttals by Gene Golovchinsky
- SIGCHI Rebuttals: Some Suggestions How to Write Them by Albrecht Schmidt
- A CHI Rebuttal by Simone O’Callaghan
- How to Write SIGCHI Rebuttals by Hyunyoung Song
- Created 2009 – original version.
- Published November 19, 2016 – revived internal guide on my blog.
- Updated November 22, 2016 – added organizational notes.