Step 8 in the Data Exploration Journey: Build

This article is part 9 in a series on data exploration, and the common struggles that we all face when trying to learn something new. A list of previous entries can be found at the end of the article. I began this series while serving as the Director of Education for the Data Visualization Society in 2022, because so many people were asking to hear more about data exploration and the process of learning data vis. What began as an exploratory project on the “State of the Industry Survey” data grew into a 1.5 year project that produced a 30-page 2023 “Career Portraits” publication (DVS member login required). This series gives an inside view of the project, illustrates my process for approaching a big project, and demonstrates that no “expert” is immune from the challenges and setbacks of learning. Let’s see where this journey takes us!

Where we last left off, my early discovery project on the DVS State of the Industry Survey data had morphed into what would become the Career Portraits initiative for the DVS. In Step 6, we got serious about reducing scope in the focus phase for the project, and in Step 7 we talked about how the right cuts can actually inspire new growth and expose new opportunities for collaboration. Now, it’s time to come back to the core project and start the long, uphill climb of the Build stage. 

The framework for this series is my version of the double-diamond model for design, first popularized by the British Design Council in 2005. The first diamond begins with the Expand stage, where all ideas are on the table. It’s all about discovery, innovation, and sketching ideas quickly. It’s fast, sometimes sloppy, and it’s usually skin deep. 

The Focus phase is the narrowing half of the first diamond. You step back and look at all the options, choose your core project direction, and focus on the specific tasks that need to be done. The Build phase at the start of the second diamond is where we shift into low gear and do the hard work of building the real thing. This is where craftsmanship and slow, deliberate effort really start to shine. It’s where discipline, hard work, and the pursuit of perfection come into play. At the end of the Focus phase you should have a clear plan; Build is where you execute.

Of course, in reality a project oscillates back and forth between expand and build phases throughout its life cycle, but the prevailing goals of the Build phase are ensuring accuracy and optimal quality and making sure that we get our project to the finish line. 

For some people, the Build phase is pure joy. It’s a time to work your technical muscles and make clear progress toward defined objectives. This is where your achiever side shines. You get to check things off of your to-do list from the Focus phase, and you should always be making visible progress toward your goals. 

For other people, Build is a painful and boring slog. Coming back to our mountain climbing metaphor from a previous article, at the start of the second diamond you are standing at the foot of the mountain and you can’t even see the top. In that moment, you might panic and decide that it was more fun to plan the trip than it is to climb the actual mountain. You might be tempted to just turn around and go home. Or maybe you’re really excited when you first start the climb, but then your muscles start to hurt in the first 5 minutes and you wonder if maybe you’re just not cut out for this. If this is you, that’s ok! You might decide to spend your time in Expand instead, or you might experiment a bit to see how you can make Build work better for you. 

If you talk to people who actually climb mountains, they will probably tell you that planning is the least rewarding part of the experience: there is no replacement for actually being there. Yes, it’s hard work, but they’ve hooked into other rewards that make the effort part of the joy. They will tell you that you need a good plan that’s matched to your strength and endurance, along with the courage to start and a willingness to embrace the discomfort that is part of every climb. After that, you just take one step after another, lean into the effort, and keep going until it is done. It will be hard, and you will be tired. You will fight your own resistance along the way. You will need to push through all of that to get to the other side. There’s no shame in turning back – sometimes it’s the wisest choice, especially if you’ve misjudged your skill level – but this is your mountain and you will need to climb it if you want to get to the top. 

It’s important to realize that the work often gets easier as you go, because the view and sense of achievement start to pull you along. You hurt less as your muscles warm up and the endorphins kick in, and you start to find a rhythm in the work. The people who excel at Build are the ones who have enough confidence in the journey to get through that initial discomfort, knowing that there is something worthwhile on the other side. They might even enjoy the hard work and find pleasure in the challenge of pushing their own limits. Skating over lots of ideas and imagining what we could do might be really fun, but it is the effort and reward of creation that really lights Builders up. 

Neither the Expand nor the Build phase is better or worse than the other. They require different skills, and appeal to different people. Sometimes a single person is able to master and enjoy both phases, but most people lean more toward one or the other. If you’re a natural at one and struggle with the other, that’s pretty normal. 

Switching metaphors, it doesn’t really matter which hand you prefer to write with, as long as you end up with a similar result. Still, most people will choose one hand over the other for certain tasks. There are few truly ambidextrous people out there, and most of them put in significant, conscious effort to train their second hand before it can be useful for precise tasks. Expand and Build are just different strengths.

If Build isn’t your thing, have patience with yourself, and try to enjoy the challenge of learning something new. You may never get to a point where you are equally strong in both phases, but time and practice will help you to become more comfortable working in the one that comes less naturally to you. 

Things you need in the build stage

Focus, and a plan.
Coming out of the first diamond, you should have a clear plan for what you need to accomplish. Know what needs to be done, and put all of your energy into doing it.

A realistic sense of your own capabilities and strengths.
You would be crazy to summit Kilimanjaro without training, and you shouldn’t expect to be creating masterpieces out of the gate in data vis, either. Know your skillset, and choose the project that pushes at the level where you truly are, not where you wish to be. If you’re in over your head, the safest and smartest thing is to turn back, or find someone who can help.

Discipline.
This is the time to do things right. No shortcuts, no “I’ll come back to this later.” You don’t want to leave a bunch of holes in your final deliverable. This is where you clean up and resolve all of the things you left dangling in the Expand phase. For many, this is the hardest part of Build; you can’t defer the things you’d prefer not to do any longer. If you haven’t been cleaning up loose ends all along, now is the time to deal with them. Think how much better you’ll feel when they are resolved!

A commitment to excellence and craftsmanship.
Build is where you take the time to do your very best work. If you are a natural Builder, your inner perfectionist may be feeling traumatized by the fast and loose approach we took in the Expand phase: this is where that part of you can retake control and really shine. Just make sure to keep things positive, and not self-defeating – it was your job to explore during Expand. A bunch of loose sketches and semi-realistic ideas is actually what perfection means in the ideation stage! This is where you take those half-finished sketches and rework them into something real. If your Builder is feeling completely exasperated with your Expander, that might be a sign that you’ve overcommitted or that you’re not setting realistic goals. This is a good time to check in and head back into Focus if necessary.

The ability to say no.
There will be times where you’ll be tempted to go “just a little bit further” or add one more thing. This is your judgment call, but it’s important to keep focused on your goals for the project and only add what you need. If you tire yourself out on the side trails, you may not make it to the peak. If you get to the end early and still have energy, you can always go back and explore on the way back down. Compulsively adding too many things during Build is a common reason that people burn out and don’t make it to the end of the project. It may also be a sign that you’re jumping back into Expand too readily, and not sticking it out with Build. 

Enough time.
The build work should start as soon as possible; don’t leave it all to the end or push yourself up against a deadline. Nobody does their best work when they’re under the gun: that creates prime conditions for your inner perfectionist to freak out and melt down rather than help. If you forgot to account for the time it actually takes to do the work, then you should go back to Focus and revise your scope accordingly. If you over-committed, now is the time to admit it and beg forgiveness. This is where you earn your own trust. You can insist on just powering through no matter what, but know that it will be harder to let your vigilance down enough to succeed in the Expand phase next time if you do.

Self-knowledge, and compassion.
You should be pushing to your limits in Build: this is where you achieve creative growth. Doing that without injury requires that you know your strengths and what you need to get through a difficult challenge, and that you know when to stop. If you find yourself always spinning out or pathologically avoiding the Build phase, a habit of pushing yourself to injury is probably why: some part of you knows that it’s not safe, and it doesn’t trust you to go there. 

Energy breaks.
You’re at maximum risk of exhaustion in the build stage. Remember to stop and do something else from time to time. I like to have a different project in Expand while I’m working on Build so that I can switch gears and do some ideation or sketching when the Build work gets hard (without adding to my current scope!). This is a great place to ideate on Part II of your project, so that you’ll have some ideas ready when this one is done…just be sure that doesn’t turn into increasing your scope for Part I. Switching into Expand mode on a subtask also helps to break things up. Working in different phases is often more effective for me as a “rest” period than taking a complete break: I can rest the mental muscles that are tired, without having to leave flow. 

---

Remember that the Build phase should be a constant conversation between final deliverable and process. It’s really important to take your time here. Work through the problem again all the way from the beginning, taking advantage of all the things you learned in Explore. You may need to step back into Expand or Focus for a little while as you refine the detailed picture of where you’re trying to go. 

Try to get through Build without making major changes to your plan. “Easy come, easy go” is an Expand mentality. It shouldn’t be necessary to throw everything out and start over at this point. As you get deeper into Build, you’re putting in work that’s going to be painful to toss out. Instead of scrapping everything when the going gets tough, lean into it, focus on the goal, and push through. That said, you should expect your picture to shift a little as you get more information about what is (and isn’t) possible, and as you tie up all those loose ends. If you find yourself tempted to run for the exits, that’s probably a signal that you’re overdoing it. Take a rest and re-evaluate, or escape into Expand for a while and then come back.

Now that we’re acquainted with the Build phase in general, let’s take a look at what was happening in the Career portraits project in this stage. 

Content Creation

Clean up the data.
In Focus, we decided on the final variables to use and comparisons that we wanted to make. For Build, we needed to recalculate our values, adjust units, and re-aggregate several of our analyses to allow slightly different comparisons in the final document. We spent a lot of time going back and forth over whether to show counts or percentages, and when to show both. We also checked (and re-checked) our code and our results to make sure that our numbers made sense. 

Get clearer on the details.
We knew what we wanted to show in the data vis, but we still needed to choose the specific visualizations and define our tools and approach for creating them. We did a brief Expand phase to review sketches for different types of visualizations to evaluate for strengths and weaknesses and feasibility, and then focused back down quickly to the ones we knew we could build in the time that we had.

Production

Think through the format of your final deliverable.
We knew we were creating a digital pdf, but we still had to think through the page size, layout specifics and implications for font sizes, labels, chart formatting, etc. All of these affected the details of our chart and page designs, and set some hard limits on how detailed the visualizations could be.

Pick your technology(ies).
Jenn (my collaborator) and I had a mix of different skills between us, and we didn’t want wrangling with code to slow us down. In the end, we built most of the visualizations in R and then annotated them in Illustrator. Some charts were used directly from R, others were calculated in R and re-built in Illustrator from scratch, and still others were made in Figma. In some cases, D3 or another solution would have been a lot better as an end-to-end tool for production, but for a one-off print publication it was much faster for us to build from where we were and with what we knew well. This hybrid approach required additional manual work and cleanup, but it gave us more control over the final formatting than we could get easily from code. 

Build out the charts.
Once the chart types and general layout were selected, we still needed to build the visualizations and calculate the actual values for our analyses. This required another round of code edits in R, and some additional exploration about how to export and edit the charts once we were done. Jenn was able to do a lot with base chart theming in R, but there were still some visualizations that required manual work. It turns out that R can’t export editable text labels, so anything we changed had to be re-typed in Illustrator by hand. The labels alone took more than 30 hours of work to clean up. We understood that cost up front and we accepted it, because for this project it was the simplest way to get to our goal.

Generalize and clean up the code.
As we worked through the final details of the analysis, there were several opportunities to go back and restructure the code to make it more consistent. This helped to make it clearer and more robust, and it also cleaned up a couple of minor calculation errors that we might not otherwise have caught. We chose to do this step even though we weren’t planning to productionalize the analysis, because we wanted our code to be readable for use in future projects and we wanted to make sure that we caught any mistakes or bugs in the data.  

Finalize your narrative.
We had a pretty good sense of the overall discussion we were interested in and the metrics we wanted to show, but you can’t actually finalize your narrative until you’re sure that the data is solid. We re-wrote most of the supporting narrative and revised our document structure more than once as the data calculations completed, to be sure that our comments and the data details were aligned.  

Identify new analyses or content, and prune as needed.
As our picture of the report became clearer, we realized that there were some additional metrics that we wanted to include. We considered the full set of options and did another round of Focus to finish things up. Some of these were larger than we could afford, so we put those off for another day. We also removed things that no longer fit. We’d planned to include an analysis comparing answers from independent visualizers and those employed in organizations, but when we got into the details, the branching structure of the survey and distribution of responses made it hard to compare those populations in a meaningful way. It would have been possible to rework the analysis to include that comparison, but it would have meant going back to square one. Reluctantly, we added this to the list of follow up projects that we could return to later. 

Assemble the final document.
Once the analysis, text, figures and other content were complete, they needed to be assembled into a single document for publication. This process alone took a couple of months and went through several rounds of revision. We used InDesign for the layout and imported the images from files to support the many, many edits and refinements required as we worked through micro edits for the final doc.

---

There are a lot of pieces involved in Build, and it can be difficult to navigate all of the loose ends and get to something that you’re proud of. The flip side is that you get to see the work develop and grow into its final shape, and there is a lot of pleasure in creating a solution that feels well-resolved. In the next article, we’ll take a closer look at some of the individual choices going on in this stage of the project, and how practical and editorial choices came together to shape the final document.

Previous articles in this series:

Embrace the Challenge to Beat Imposter Syndrome
Step 1 in the Data Exploration Journey: Getting to Know Your Data
Step 2 in the Data Exploration Journey: Going Deeper into the Analysis
Step 3 in the Data Exploration Journey: Productive Tangents
Step 4 in the Data Exploration Journey: Knowing When to Stop
Step 5 in the Data Exploration Journey: Collaborate to Accelerate
Step 6 in the Data Exploration Journey: Cut to Realistic Scope
Step 7 in the Data Exploration Journey: Spin Off Projects

Related links:

Early Sketches for Career Portraits in Data Visualization, by Jenn Schilling
DVS Careers in Data Visualization, YouTube Playlist for interview series by Amanda Makulec and Elijah Meeks
Career Portraits project (DVS Member space login required)