My informal guide for how to join the IEEE InfoVis scientific community for new students, industry professionals, and established scientists from other fields.
I wrote this as a reflection of my experience reading, reviewing, and authoring IEEE InfoVis papers for more than 10 years (my first InfoVis paper was in 2003). As always, your mileage may vary, this is my own highly personal opinion and not the official policy of IEEE VIS or any other body, and following these guidelines will still not guarantee that your paper will be accepted to the conference (of course!). Nevertheless, comments and feedback are welcome!
Advice on Joining the InfoVis Community
by Niklas Elmqvist, University of Maryland, College Park
Practically all scientific disciplines as well as business sectors have large amounts of data that can benefit from being presented visually. Thus it follows that data visualization can be applied to any such discipline or sector, making visualization a truly interdisciplinary research area. Accordingly, the information visualization (InfoVis) community is nominally very welcoming to new contributors, both from other disciplines as well as industry. However, InfoVis, just like every other scientific community, has its own customs and traditions, and being aware of and following these will make it easier for new authors to get their submissions accepted. In this guide, I provide advice for people looking to break into the field from industry, grad school, or other disciplines. I also list some specific guidelines for writing successful InfoVis papers.
…for Established Scientists from Other Domains
Almost all scientific disciplines deal with a multitude of data in some form or another. For this reason, almost all scientific disciplines can benefit from data visualization. Accordingly, researchers from other disciplines regularly find their way to into the information visualization community, either as temporary visitors or as more regular contributors. This section deals with the unique challenges inherent in entering the InfoVis community from another field.
Some of the strengths that you as a representative of another domain bring to the table are the following:
- Scientific rigor: HCI and visualization research is often fairly applied, informal, and practical in nature, whereas many other scientific disciplines take a more rigorous and formal approach. Use this to your advantage by offering a well-described and rigorous method; many reviewers will recognize and appreciate this.
- Real domain expertise. Most InfoVis papers deal with data from a specific domain outside the InfoVis community. However, many InfoVis papers are written by researchers who are part of the core community, which means that they are often (at least) one step removed from the application area. You have the advantage of being an actual member of the application domain in question, which means that you have actual understanding of the domain and its challenges. Make sure to demonstrate this in the paper by summarizing the problem domain concisely and succinctly in your paper.
Meanwhile, be aware of the following weaknesses:
- Informality: As stated above, HCI and visualization papers are often more informal than in other domains. This manifests in several ways: (1) the tone is often more informal than in other domains, (2) many (although not all) InfoVis papers have a more engineering-centric flair, and (3) human subjects experiments often involve a smaller number of participants than in the social sciences or medicine (on the order of 10-20 rather than 100-200 or even thousands) with factorial designs, minimal individual differences (compared to effect sizes), and a large number of trials.
This only scratches the surface of the complexities of moving from an unrelated field to InfoVis. Here are some additional external resources on this topic (to be expanded as I find new ones):
- Tamara Munzner recently wrote a blog post on different experiment conventions across fields.
…for Industry and Business Representatives
As for any scientific discipline, the ultimate goal of the InfoVis community is for our results to matter and be applied in society as a whole. In computer science disciplines, this is most easily achieved through industry adoption and commercialization. For this reason, industry and business representatives are nominally welcome to contribute to the InfoVis community. However, because cultures in academia and industry are very different, it may seem to industry representatives that the InfoVis scientific community is not welcoming at all (in fact, sometimes even the opposite!). This is unfortunate. In this section, I give some pointers on how to counteract this phenomenon.
Some of the strengths that you as a representative of industry bring to the table are the following:
- Relevance: Unlike some academic projects, commercial products only exist because they have a real value to its customers. Exploit this to your advantage by explaining the market relevance and demand for your product in the academic paper. Your company probably already conducted some form of market research on your users and their needs; these can often be reworked into personas, use-cases, and task analyses in a scientific paper.
- Practicality: Where academic projects may sometimes be needlessly complex, most industry products will use common-sense, pragmatic, and straightforward solutions that work. While creative and/or crazy solutions have a value, most reviewers respond better to industry’s practical solutions.
- Deployment: Deployed and/or established products are already in the hands of your users. This can be used to validate your contribution, sometimes by sheer numbers alone (“500 people use our product for at least two hours each day”). A more invasive approach is to instrument your product with performance measurements and/or survey mechanisms, allowing you to collect specific data from use of the tool in the wild (this is often the most valuable and useful form of user data). You may also be able to use your company’s usability testing group and usability testing lab (if you have those resources) to perform controlled user studies. Note, however, that scientific user studies are often significantly different from typical usability testing; be sure to consult the appropriate literature.
- Resources: Where an academic project often consists of a professor and one or two graduate students, companies typically have significantly greater resources for product development and testing. Use this resource disparity to your advantage by building polished, well-designed, and powerful research contributions that go well beyond what academic teams can achieve.
Meanwhile, be aware of the following weaknesses:
- Market-speak: Academic reviewers are allergic to strong claims without proper backing, often discarding such writing as “marketing”. Avoid this practice by remembering that you are not writing promotional material intended to attract customers, but you are objectively describing your solution to a scientific audience consisting of your peers.
- Science, not (just) engineering: Most scientific contributions go beyond the low-level technical details that make up the actual solution, whereas the technical (engineering) details are often highly important and interesting in industry white papers. Resist the temptation to pad your scientific paper with technical details that do not contribute to the scientific contribution of your work, only its engineering aspects (these are important too, but include just enough to ensure replicability).
- Insufficient novelty: A direct consequence of using straightforward design solutions is that your scientific contribution may appear insufficient to an academic reviewer. Where you routinely try to eliminate complexity in favor of designs that work, academics sometimes pursue complexity as a goal in and of itself. Avoid this by carefully motivating your designs (perhaps by quoting formative evaluation that guided it) and highlighting the novel scientific contributions that do go beyond the state of the art.
Additional resources for practitioners (to be expanded):
- Visual Business Intelligence – Stephen Few’s blog on data visualization
- Data Stories – a podcast on data visualization with Enrico Bertini and Moritz Stefaner
…for New Students
It can be difficult to break into the InfoVis research area as a new student if you lack an advisor with experience in the research area (as it turns out, this is what I had to do as a Ph.D. student). While not strictly related to paper-writing, the below pieces of advice can be useful for new students attempting to break into the visualization field:
- Read, read, read: The best way to learn about the visualization community is to read accepted papers that have appeared in well-regarded scientific conferences and journals in the visualization field. Pay special attention to best papers, as these are presumably good representatives of high-quality research in the field.
- Steal presentation methods: Once you have read enough visualization papers, you will begin to have a good idea of what they should contain to be acceptable. When you write your own paper, mimic the presentation styles (not the content, that is plagiarism!) that these papers use for your own work. For example, if you think that a specific paper had a particular clear and crisp way to present results from a user study, do not hesitate to “steal” that approach (for your own data, project, and words, of course; never reusing writing verbatim).
- Doctoral colloquium: If your own advisor is not an expert in visualization, you should connect with someone who is. The IEEE VIS doctoral colloquium is co-located with the VIS conferences and annually assembles an expert panel of visualization researchers to give feedback to current Ph.D. students. The submission deadline is usually around the end of the Spring or early Summer. As a past VIS doctoral colloquium organizer, I can personally attest to how useful this is for our student participants.
- Research visits: The most involved approach to break into a new research area such as InfoVis—barring switching university and advisor—is to visit a faculty member at another university as part of your Ph.D. studies. This was the path I chose myself—I spent Spring 2006 in John Stasko’s research group at Georgia Tech—and it was very helpful (even if it can be disruptive to your studies).
Some resources for new students (to be expanded):
- IVMOOC – Katy Borner’s (Indiana University) MOOC on information visualization
Important InfoVis Paper Components
While there is no single format for a successful InfoVis paper, there exist several common properties that many published papers have. Many of the guidelines below are synthesized from Tamara Munzner’s paper “Process and Pitfalls in Writing Information Visualization Research Papers” (Information Visualization 2008). Also note that none of these guidelines are prescriptive; no research paper is required to follow all of the guidelines (even if most papers follow at least some), and the very best contributions often make their own rules. Furthermore, even if you follow all of these guidelines, there is no guarantee that your paper will be accepted; InfoVis papers are ultimately accepted on their scientific merits and novelty of their contribution, whereas many of the below factors deal with presentation and scientific rigor.
- Motivation: Science is the pursuit of new knowledge, but the InfoVis conference specifically solicits new knowledge in the area of information visualization. Include a specific motivation in your paper as to why your work is relevant to InfoVis research. This is particularly important for basic research, such as in perceptual and cognitive psychology.
- Design aspects: Since information visualization by its definition creates visual encodings where no natural such encoding exists, InfoVis research is partially a design discipline. This means that authors are encouraged to describe and motivate their design process, design rationale, and any specific design decisions they made.
- Human aspects: Similarly, visualization is poised at the intersection of the human and the computer, so successful papers commonly include (and are guided by) in-depth discussions of the user, including personas, goals, and tasks.
- Data aspects: The data is often central to an InfoVis paper. Be sure to explicitly describe your data model, characteristics, and scale. Synthetic or small, so-called “toy”, datasets are rarely useful except for demonstration purposes. Use datasets of the same size and complexity as competing approaches; in fact, many InfoVis contributions solely challenge competing approaches by increasing scale or complexity.
- Theory and practice: InfoVis has active scientific as well as industry communities. While basic and applied research on their own are important, the best papers maximize their impact by including both theoretical as well as practical solutions. Ben Shneiderman calls this approach applied and basic research combined (see his 2016 book on the topic). Even when combining both in one paper, it is often worth keeping the contributions separate: describe the theoretical idea first, and then explain how your practical solution implements or instantiates the idea.
- Validation: All scientific papers must include some form of validation to demonstrate the value, utility, or feasibility of the contribution. The form of validation in InfoVis depends on the contribution type (see Munzner’s “A Nested Model for Visualization Design and Validation“, IEEE TVCG 2009). However, if you do include statistical analysis of human performance data (i.e. in the style of cognitive psychology experiments), be sure to use correct statistical methods and interpretation of the results. Otherwise strong papers have been rejected due to incorrect statistical analysis; communicate with your organization’s statistical consulting service if you are unsure.
- Limitations: No scientific contribution is ever perfect, but the important thing in an InfoVis paper (in any scientific paper, for that matter) is that the authors demonstrate that they are aware of the limitations of their work. Do include an explicit section or paragraph in your paper where you enumerate the specific conditions, scale, and complexity under which your contribution works and does not work, respectively.
The best way to learn how to write InfoVis papers is to read a lot of papers that have appeared in well-regarded scientific conferences and journals; see my list of the top such venues here. Finally, note that this document represents my own opinion, and not that of the IEEE InfoVis conference. Accept and reject decisions are made by the collective reviewer community and not individual reviewers (or even papers co-chairs!).
- January 4, 2016 – document created.
- January 14, 2016 – added advice sections (unfinished), minor editing.
- January 15, 2016 – expanded advice sections.
- January 17, 2016 – settled on a title, minor editing.
- January 18, 2016 – reorganization, minor editing.
- January 20, 2016 – added illustration, added list of resources, discussed theory and practice.
- January 21, 2016 – created a separate document on top visualization conferences and journals.
- January 28, 2016 – need to add a section describing suitable research topics for InfoVis venues.