Visual Analytics Introduction: Research, Solutions, and Effectiveness

COMP 150-04 | Visual Analytics Lecture 01: Introduction COMP 250-02: Visualization Seminar September 08, 2020 Remco Chang 01 - Introduction 1/36

COMP 150-04 | Visual Analytics People Remco Chang Email: remco@cs.tufts.edu Office: 196 Boston Ave Pronouns: (he | him | his) Ashley Suh Email: Ashley.Suh@tufts.edu Office: 196 Boston Ave Pronouns: (she | her | hers) Ab Mosca Email: amosca01@cs.tufts.edu Office: 196 Boston Ave Pronouns: (they | them | theirs) Remco Chang 01 - Introduction 2/36

COMP 150-04 | Visual Analytics Course Structure Classes on Tuesdays and Thursdays (1:30-2:45) Course Website: www.cs.tufts.edu/comp/250VIS More on the debates later Requirements (grades, etc.) Reading papers Participating in 2 of the 4 debates Presentation (of papers) Class discussions Remco Chang 01 - Introduction 3/36

COMP 150-04 | Visual Analytics Motivation Motivation for this class Learn how to do research in VIS Build knowledge about research in VIS Goal is to have a common framework that is consistent with the rest of the community 1. 2. What is Research? Identify problem Come up with a solution Demonstrate that the solution works on the problem Remco Chang 01 - Introduction 4/36

COMP 150-04 | Visual Analytics Defining a Problem Identify Problem What is a problem ? Whose problem? Researchers Practitioners (e.g. programmers) Industry / Government (e.g. companies, governments) Everyone in the world (the masses) Maybe people didn t know that they had a problem/need (yet)? Remco Chang 01 - Introduction 5/36

COMP 150-04 | Visual Analytics Come up with a solution Before inventing your own What are the existing solutions? How well do they work? Do we need a new solution? Why? Two types of solutions Incremental (makes solution faster, more efficient, etc.) New approach (a different take on an old problem) Remco Chang 01 - Introduction 6/36

COMP 150-04 | Visual Analytics Demonstrate Effectiveness Evaluation: demonstrate that the solution works on the problem How to evaluate (and demonstrate improvement?) Depends on the problem statement If the problem is that an algorithm is slow, demonstrate that your solution shows quantitative improvement If the problem is that a domain user cannot do X effectively, show (in a qualitative study) that your solution helps the user do X easily. A nested model for visualization design and validation , Tamara Munzner, TVCG, 2009 https://ieeexplore.ieee.org/abstract/document/5290695 Remco Chang 01 - Introduction 7/36

COMP 150-04 | Visual Analytics Is that it? The illustrated guide to a PhD: http://matt.might.net/articles/phd-school-in-pictures/ The size of the dent How do I make an impact with my research? Remco Chang 01 - Introduction 8/36

COMP 150-04 | Visual Analytics The So What? Question Perhaps the hardest part of doing research is to answer the question of so what. Connected to the identify problem consideration. In some cases, the so what is self-evident. For example, if you built a tool to help a domain user do X more effectively, your so what depends on: How many of these domain users are there? How important is X (and doing X well)? How often do they use your tool? The Value of Visualization , Jarke van Wijk. IEEE VIS, 2005 http://win.tue.nl/~vanwijk/vov.pdf But other times, the so what is less obvious Remco Chang 01 - Introduction 9/36

COMP 150-04 | Visual Analytics The So What? Question Consider: You developed a tool to help one doctor visualize gene data Scenarios: this doctor ended up finding a cure for cancer using your tool this doctor used your tool and wrote a paper based on the result this doctor used your tool to generate an image for a paper 1. 2. 3. Wrote a library to make programming visualizations easier Scenarios: Most visualizations in the world are built using your library (e.g. d3) You used your library to start a company (e.g. plotly) You wrote a paper, but adoption never took off (e.g. infovis toolkit) 1. 2. 3. Remco Chang 01 - Introduction 10/36

COMP 150-04 | Visual Analytics The So What? Question The point is, the so what isn t always obvious. However, it is still important to articulate the so what in two ways: Writing a paper: you need to make it clear to the reviewer why your work is relevant Practical Impact: for your own purpose, before you embark on a research project, understand why you are doing what you re doing Remco Chang 01 - Introduction 11/36

COMP 150-04 | Visual Analytics More Importantly Almost everything we have discussed so far can be articulated before you start your research project Problem Statement What is the problem Who has the problem Related Work What are the current approaches Why are they deficient Your Solution What is your approach How would you evaluate So What? Who will benefit How will your solution change the world Remco Chang 01 - Introduction 12/36

COMP 150-04 | Visual Analytics Putting It Together Visual Analytics Lab s project prospectus : Template: http://www.cs.tufts.edu/comp/250VIS/pub/prospectus-template.pdf http://www.cs.tufts.edu/comp/250VIS/pub/prospectus-template.tex Examples: http://www.cs.tufts.edu/comp/250VIS/pub/prospectus- newworld.pdf http://www.cs.tufts.edu/comp/250VIS/pub/prospectus- activemodel.pdf A reflection of your own research project What is your so what ? Remco Chang 01 - Introduction 13/36

COMP 150-04 | Visual Analytics Questions? Remco Chang 01 - Introduction 14/36

COMP 150-04 | Visual Analytics Shortest Path to Publication in VIS In the VIS community, the fastest way to get published is to write application papers: Problem Statement: Working with domain experts in XYZ field, we identify a list of design requirements (desideratas) that will help improve these domain experts in doing their tasks. Solution: For InfoVis: a hand-crafted visualization that is novel-ish; For VAST: a combination of an existing ML technique with a novel-ish visualization. Evaluation: We evaluated with the domain experts and they confirm that using our tool, they can accomplish their tasks more efficiently So What: The field of XYZ is important. So helping these experts in XYZ to do their jobs better is important. Remco Chang 01 - Introduction 15/36

COMP 150-04 | Visual Analytics Shortest Path to Publications (In General) Beyond the VIS community, generally speaking, getting published is not difficult. Here s what you do: Critically read a paper in your field Identify ways that the paper can be improved Often the authors will tell you in the Limitation section Improve upon the paper Remco Chang 01 - Introduction 16/36

COMP 150-04 | Visual Analytics Shortest Path to Publications (In General) With this approach, your paper looks like: Problem Statement: We consider the work by XYZ et al. Their approach has the following limitations Solution: We found a way that addresses these limitations. Evaluation: We evaluated our approach that found that our technique is faster by XXX% in some select cases, but can be slower in other cases by up to YYY%. So What: Well, XYZ et al. s paper was published. So improving upon their solution must be important too! Remco Chang 01 - Introduction 17/36

COMP 150-04 | Visual Analytics What s Wrong with These Approaches? Nothing. Well, not really. In the case of application papers, very often these papers do not push forward in terms of science. As the name suggests, the solution is an applied one. So the application domain benefits. But if the application paper ends up curing cancer In the case of the improving on existing work paper, a similar consideration of impact needs to be examined. Optimizing an algorithm or improving upon an existing solution is always good, but these ideas are often derivatives. But if your optimization is for reducing greenhouse gas emission by 1%... Remco Chang 01 - Introduction 18/36

COMP 150-04 | Visual Analytics What s Wrong with These Approaches? Regardless, if all of your publications fall into these two categories: First, you can still get a PhD In fact, many faculty advisors in VIS only have requirements in terms of number of publications Second, you likely can graduate very quickly However, your thesis will likely not be cohesive It s hard to tell a story from papers like these and your work can be forgettable Remco Chang 01 - Introduction 19/36

COMP 150-04 | Visual Analytics So, What Project Should I Work On? This is a highly subjective question Question: would you have worked on the idea of twitter (before it existed in 2006) as a PhD project? Twitter has little technological advances It served a need at the time and reached billions of users Ultimately, its use today is a little suspect The answer touches on three things: How much math needs to be in my thesis? What is the value of impact? Am I ethically responsible for my research and software? Remco Chang 01 - Introduction 20/36

COMP 150-04 | Visual Analytics Questions? Remco Chang 01 - Introduction 21/36

COMP 150-04 | Visual Analytics Spirit of this Class We will be examining papers in the VIS (and related) communities We will read these papers We will consider them critically in terms of their contributions and flaws With each (recent) paper that is presented, we will discuss *how* the paper can be improved I.e. if you were to write a follow up paper, what would that paper be? Caveat: since many of the papers we read will be classic papers, the how to improve question can be obvious (or has already been done) The bottom line is that we want to learn from these papers Why did they succeed? What did they do right? What lessons can we take away from the paper (in terms of technical approach, writing, perspective, so what , etc.) For each of these papers, consider how you can apply the lesson or the takeaway to your own research Remco Chang 01 - Introduction 22/36

COMP 150-04 | Visual Analytics Outline of the Class http://www.cs.tufts.edu/comp/250VIS/ Four debates on the following topics: Design guidelines considered harmful The humans are dead To interact or not to interact, that is the question The future of data science is visualization Two teams (affirmative and opposing). Each team has 3 people Assuming that the class size doesn t change, this means each person will be in 2 debates Debate format: https://www.youtube.com/watch?v=yi6Im-Sb6Vw Remco Chang 01 - Introduction 23/36

COMP 150-04 | Visual Analytics Debate Structure Opening: Affirmative (A1): Pro Position (5 minutes) 2 minute break (team meeting) Opposing: Rebuttal (3 minutes) Opposing (O1): Con Position (5 minutes) 2 minute break (team meeting) Affirmative: Rebuttal (3 minutes) Statements: Opposing (O2): Statements (5 minutes) 2 minute break (team meeting) Affirmative: Rebuttal (3 minutes) Affirmative (A2): Statements (5 minutes) 2 minute break (team meeting) Opposing: Rebuttal (3 minutes) 2 minute break (team meeting) Closing: Affirmative (A3): Closing (5 minutes) Opposing (O3): Closing (5 minutes) Total time: 52 minutes Remaining class time will be used to determine a winner The class as a whole gets to vote on the winner I ll also try to invite external VIS professors to judge the debates For the breaks, I ll set up breakout rooms for the teams to discuss the rebuttals The rebuttal can be done by any person on the team (but only one person per rebuttal per team) Note that there are two rebuttals (per team), these can be done by two different people Remco Chang 01 - Introduction 24/36

COMP 150-04 | Visual Analytics Debate: How To There is a lot of material online on how to debate well. For this class, the focus is not to train you to become a better debater. The only requirement is for you to use publications as evidence for your arguments! Use publications that are not too obscure (i.e. please use reputable conferences or journals) If you re not sure if a reference is too obscure, contact me Be prepared! Think of the evidence/publications that your opposing team could use and prepare for those arguments! Work together as a team! Also, unlike typical debates, you are welcome to use slides Remco Chang 01 - Introduction 25/36

COMP 150-04 | Visual Analytics Debate Modules Each debate will take up a 2-week module (4 lectures) Lecture 1: I will give an overview of the topic (and the relevant papers) For those not in the debate, you should read the associated papers before the debate Lecture 2: Day of the debate Each team should announce the 3 key papers used in their arguments at the end of the debate Lecture 3: Losing team presents these 3 key papers Each member will present one paper. Lecture 4: Winning team presents 3 key papers Winning team gets more time! Remco Chang 01 - Introduction 26/36

COMP 150-04 | Visual Analytics Debate Preparation For the people in the debate, you should start your preparation 2 weeks ahead of time This is when I will have your reading list finalized For those who will be in Debate #1, you should get started right away! The two teams should schedule a meeting with me to go over the spirit of the debate. I will suggest possible papers or directions to pursue and answer any questions you might have. This should happen at least 1 week before the debate You should have some ideas of the 3 key papers that you will use for your arguments Remco Chang 01 - Introduction 27/36

COMP 150-04 | Visual Analytics Presentations after the Debate For each presentation: A presentation should be 10-12 minutes You should prepare slides and videos You should prepare discussion points You will lead the discussion of the paper Discussion should be about 10-15 minutes Total time (presentation + discussion) should be <25 minutes A discussion should include: Positive takeaways Writing, technical approach, perspective, results, impact, etc. Negative criticism How could the work have been improved Next paper If you were to write a follow up paper, what would the paper be about? (Feel free to list more than 1 idea) Remco Chang 01 - Introduction 28/36

COMP 150-04 | Visual Analytics Questions? Remco Chang 01 - Introduction 29/36

COMP 150-04 | Visual Analytics Debate #1: Design Guidelines Considered Harmful Visualization research has always had a strong (visual) design component. As a result, a number of design guidelines have emerged over the years. Affirmative team: Design guidelines are necessary. They help practitioners know how to design effective visualizations. They also serve as the foundation of visualization research. Opposing team: Design guidelines are rigid and have caused a great deal of harm to the community. There is no good or bad design designs are inherently contextual and task dependent. Remco Chang 01 - Introduction 30/36

COMP 150-04 | Visual Analytics Debate #2: The Humans Are Dead This is the debate about automation versus human-user control. Data exploration, data analysis, and decision-making have increasingly become automated. Is a human-user still necessary? Affirmative team: Human users are always necessary. Skynet anyone? Opposing team: Automation is inevitable. Get with the program and start praying to our robot overlords. Remco Chang 01 - Introduction 31/36

COMP 150-04 | Visual Analytics Debate #3: To Interact or Not to Interact Interaction has always been an important aspect of visual data exploration and data analysis. However, there is increasing evidence that interactions can lead to confusion (more options are not always better). In fact, if the purpose of visualization is to communicate important information, why leave it to a user to discover it? Affirmative team: Interactions are necessary. One (static) visualization cannot tell the whole story. A user needs to interact with their visualization to feel engaged and empowered. Opposing team: Interaction do not provide any cognitive benefits but only provides the fa ade of control. In many cases, taking away the control is the only way to ensure that users get the correct message. Remco Chang 01 - Introduction 32/36

COMP 150-04 | Visual Analytics Debate #4: The Future of Data Science is Visualization Visualization is intrinsically tied to data management, data analysis, and generally data science. Often seen as a tack- on, traditionally visualization is used at the end of an analysis to generate a plot. However, increasingly visualization is becoming the interface to all of data science. Affirmative team: Data science will become ubiquitous. However, not everyone will be trained to do data science. Visualization holds the key to making data science usable by the masses. Opposing team: Statistics, AI, and database will be the way of the future. Visualization will stay as the applied interface to AI, ML, Stats, and DB techniques. As such, it can only react to the advances in AI/ML/etc but should not take lead. Remco Chang 01 - Introduction 33/36

COMP 150-04 | Visual Analytics Sign Up Sheet Use this link to sign up for the debates that you will participate in https://docs.google.com/spreadsheets/d/1XvMA1__gJVV j2KrN7zF9IylAYcw3TyrDPW-yj5cLkVU First come, first serve! Feel free to email me if you have questions I will start (randomly) assigning people before class on Thursday if people haven t signed up yet. Remco Chang 01 - Introduction 34/36

COMP 150-04 | Visual Analytics Questions? Remco Chang 01 - Introduction 35/36

COMP 150-04 | Visual Analytics Others Another way to meet? https://gather.town/KLgi8z31yd7px9xM/tufts- comp250vis Remco Chang 01 - Introduction 36/36