Missing Data: Why Don’t We All Take the 2022 DVS SOTI Survey?

Yesterday the Data Visualization Society (DVS) launched its 2022 State of the Industry (SOTI) survey. As one of 7–10 individuals who helped revise the survey last year and this year, I plan to contribute directly to this annually updated dataset (now in its sixth year). I hope many of you will contribute responses, too—and invite people you know—to improve our collective understanding of dataviz as a diverse and dynamic field. This article requests your help on both counts by discussing why someone might not take this survey.

From previous years’ feedback plus direct experience, three common reasons are:

1. I forgot (or had no idea)!
2. I am unsure whether my experience fits a “Data Visualization State of the Industry” survey.
3. I am not convinced that the benefits outweigh the costs.

Reason three has multiple facets, including busy schedules, privacy concerns, and wanting examples of value. My discussion will focus primarily on fit, hesitations, and outcomes, to show who this survey is for and why. But first…

I forgot (or had no idea)!

• If you have already decided that you want to take this survey, please set yourself a reminder for a convenient time! You can add this link to your calendar (we suggest 15–20 minutes).
• The DVS will send several reminders via email, Twitter, Instagram, LinkedIn, Slack, and more. If you have already taken the survey when you receive these, please consider spreading the word. 
• Every year some people who would have liked to take this survey find out about it too late. You can help by sharing the link with anyone you know who does data visualization.

Does my experience count?

TL;DR

1. If you do data visualization in any capacity, this survey is interested in your experience.
2. Visualizing data professionally is not a requirement.
3. Membership in the DVS is not a requirement.

“State of the Industry” sounds pretty intimidating

Full disclosure: we’re still not 100 percent sure what to call this survey. It began in 2017 as a two-year project, led by Elijah Meeks and emerging from “extensive community discussions on Twitter, Slack, and elsewhere” between various individuals practicing data visualization: 

“The point … was to create a dataset so that we might together move beyond our intuitions about the shape and nature of the field (or even its existence) and base our understanding on data.”

2017 Data Visualization Survey Results

In 2019, after Elijah co-founded the Data Visualization Society with Amy Cesal and Mollie Pettit, the survey became an organization-run initiative, but DVS membership has always been optional for getting counted. Participation by practitioners beyond DVS increased notably from 2020 to 2021:

The 2021 survey committee voted to change the title from “Annual Census” (since a census has fewer questions and wider participation) to “State of the Industry,” but we still wonder: Is dataviz an industry? 

This question was productively debated at Outlier 2022 in February by the “State of the Industry Panel” (26’34”–32’24”). Panelists offered a range of thoughtful reflections, including:

• Yes, we qualify as a distinct industry. We have our own conferences, tools, courses in higher ed, and research about what we do. But we don’t stand alone. We work with many other industries, because it’s their stories we tell.
• I don’t know. Our work is less vertically defined than, e.g., architecture or graphic design. But it’s interesting to think about how to describe who we are as a collective. What if we compare our work with a skillset like writing: Is writing an industry?
• Whether we use “industry” or another label, there may be value in having a label, to impel and support activities like defining ethical standards and creating whitepapers.

We want to understand both our differences and our common ground

Whether you consider dataviz a distinct industry, a participant in multiple industries, or both, there is no question that data visualization is a skillset with diverse aspects and applications. 

If you have an answer to any of the following questions, we would love your input to help fellow practitioners explore the state of dataviz in 2022!

To share or not to share: are these outcomes worth my time and answers?

In collecting data from people who work closely with data, this survey receives thoughtful feedback every year. Two themes that have repeatedly influenced annual revisions are representation and privacy.

TL;DR

1. This survey includes sensitive questions, but it has strong privacy protections.
2. If you choose to take the survey this year, you will improve the longest-running, most extensive collection of data about dataviz practitioners for dataviz practitioners. 
3. This multi-dimensional, international dataset provides all of us with relevant material to explore and to inform discussions and decisions about our work. It is especially helpful for newcomers, career changers, learners, program planners, and resource designers.
4. Scheduling a specific time on your September calendar may help with engaging this survey as an investment rather than an interruption.

Some questions ask for information that I consider sensitive

As a respondent who definitely does not want anyone to be able to single out my income information or various other responses—as well as the person who did most of the data cleaning for 2021, and got to stress test the privacy measures developed by previous survey organizers while creating our five-year dictionary—I can attest that the following combination is effective in preserving individual respondents’ anonymity:

• The survey collects zero directly identifiable data (no emails, no names, etc.)
• Before publishing the data, the survey committee carefully omits, separates, or blurs potentially identifiable details (such as distinctive job titles, cities, etc.) that might in combination make it possible to associate a data row with a particular person.
• The DVS uses the full dataset only for analyses conducted by team members who have agreed to specific non-disclosure and data privacy controls.

The DVS survey information page includes links to precedents from 2021 and 2020 that illustrate all of the above.

In addition, everyone is free to skip any question for any reason: lack of fit, lack of time, lack of interest, not wanting to share that information, etc. This ambiguity makes it impossible to identify why any individual was silent about a particular question.

This survey is long!

15–20 minutes is not a small ask for some of us. Many data visualization practitioners find “lack of time” to be an even bigger frustration than “accessing data:”

Having learned a lot about this field by exploring our collective responses from past years and browsing how other people have engaged them, I find it helpful to think of my survey-taking session as an appointment with fellow dataviz practitioners. By engaging a shared set of questions from across the globe on an annual cadence, each of us enriches future conversations about what we do. 

Scheduling a specific time slot for this survey makes it less likely to interrupt deep work. If this approach makes sense to you, here again is the link for your September 2022 calendar. (And please share it with people you know.)

Enriching conversations with fellow data visualizers

Last year, Jill A. Brown and I announced the 2021 survey with a tour of outcomes that this yearly initiative supports. To conclude this article, I illustrate three of my favorites below, with copious links to projects and learnings. (If you have not yet taken the 2022 survey, note the call to action in my closing example!)

1. Identifying professional benchmarks for income discussions

Many of us did not start our work lives doing data visualization. How much should we charge for consulting work, or negotiate for salary? Multiple projects engage DVS survey data on this issue. Each demonstrates the value of being able to examine income ranges by variables such as years of experience, roles, industries, and demographics.

Last year, after discovering Alli Torban’s discussion “How Much Money Do Data Visualization Professionals Make?”, I catalogued and examined diverse contributors’ dashboards from 2019–2020 survey results to address a practical question that repeatedly comes up in the #connect-freelance and #topic-career-advice channels in the DVS Slack:

Examples from the 2021 survey visualization challenge include concise data stories by Verena Schrader and Matthew Osborne and an interactive tool, “How Does Your Income Compare?,” by Mia Szarvas (whose process notes and nuanced analysis in “Visualizing Your Personal Gender Pay Gap: How Does Your Salary Compare?” are well worth reading, especially the final section, “So what?”).

2. Rich opportunities for practitioners to explore the diversity of this field, while practicing skills and creating portfolio pieces on topics of mutual interest

The number one reason 2021 respondents identified for doing data visualization side projects is to build skills, followed closely by personal enjoyment, then to build my portfolio. All three goals are well served by our collective contributions to this multi-dimensional survey.

The annual results “provide a great opportunity … to dive into real, freely accessible, and up-to-date data. … I like exploring topics that give me some sense of usefulness for a certain audience …”

Martina Dossi, A Matter of Time: Linking Job Titles to Data Visualization Tasks

Here are just a few of the stories and portraits that fellow data visualizers developed from our 2021 survey responses about our roles, education, leadership, audience, preferred methods of learning, tools, and more:

To enjoy their work directly, and discover further gems, visit the 2021 SOTI challenge gallery here. (Links for each of the above examples are also listed below under Credits.)

3. Identifying ways to serve the international data visualization community better

Annual survey results help DVS leadership in planning resource offerings and community discussions.

“The data confirmed several guesses and assumptions that I’d made, and pointed out other things that surprised me.  Seeing that our audience has a strong applied focus … and significant learning on the job … changes our approach to delivery, structure, and content …”

Erica Gunn, An Insight-Informed View of DVS Education Opportunities

In the past year, the Education and Programs Directors have visualized both current trends and change over time to highlight the diversity of experience and backgrounds among the groups that the DVS seeks to serve. For example, more than one-in-three respondents in our (lengthy, all-English) 2021 survey speak languages besides English.

Collectively we represent over 40 languages! Given this variety, Jill A. Brown asks: might “data visualization of the future …. show more efforts towards multilingual visualizations, increasing accessibility for even broader audiences”?

---

My final example closes with a timely call to action from Letícia Pozza and Odd Studio. Six days before the deadline for the 2021 survey visualization challenge, this Latin American team—distributed between Brazil, Mexico, and Spain—set out to address the frequently asked question: “I’ve started studying data visualization recently, who should I follow?” To do so, they cleaned and then painstakingly researched all 400+ answers from 700+ people’s write-ins to the free-entry survey question: “Who do you find helpful for inspiration in data visualization? Feel free to list multiple influences.” Initially, they expected simply to enjoy exploring, learning, and collaborating.

One day before their project was due, however, they realized that over 90 percent of the 421 entities that respondents listed in last year’s survey are located in either North America (red) or Europe (purple):

“By experimenting and wanting to have fun with the survey data, we stumbled upon a bigger question: are we doing enough to uplift our fellow Latin American dataviz experts, or are we just reinforcing the same Global North-centered mindset? …

We cannot change this alone. So this is a call for everyone in the data visualization field to research, recommend, and comment on the next DataViz Survey at least one person from the less represented regions.

Letícia Pozza and Odd Studio

If you do so, we will make sure their heads are represented next year in VizHeads.com.”

To answer this call to action, please check out Viz Heads and take the 2022 survey!

---

Credits

Thanks to all of the following groups for making this dataset possible and useful:

• Jill A. Brown and the 2022 survey committee; everyone who offered feedback on the 2021 survey and/or 2022 draft; and our predecessors in 2017–2021, particularly Amy Cesal, Elijah Meeks, and Zander Furnas.
• DVS board members, staff, and advisors who conduct thoughtful analyses and provide logistical support.
• Each individual worldwide who has contributed to the 6,500+ rows of data accrued over the past 5 years.
• The 100 or so individuals who have inspired, questioned, delighted, and informed through your work on the 2019, 2020, and 2021 survey results.

Photos from Unsplash: 

• puzzle pieces [recolored] by Gabriel Crismariu 
• goldfish by zhengtao tang 
• duck and ducklings by Rosa Virginia
• balancing gold beads [cropped] by Elena Mozhvilo

Previously published visualizations by:

• Jill A. Brown in the 2021 SOTI Survey Report and “Your Data, Your Story”
• Sanjay Tambe and Nishant Nandardhane, “Tools and Charts”
• Sarah Emery Clark, “What’s Cookin’?”
• Kenon Vinson, “Do Your Leaders Know Your Worth?”
• Elie Francis, “Who Do We Create Data Visualizations For?”
• Federico Comitani, “The Data Visualization Society Atlas”
• Erica Gunn, “An Insight-Informed View of DVS Education Opportunities”
• Letícia Pozza and Odd Studio, “Experimenting Can Lead to Great Dataviz” / “Experimentação Para Gerar Novas Descobertas Em Dataviz”