Data Visualization and Discovery For Better Business Decisions PDF
Data Visualization and Discovery For Better Business Decisions PDF
tdwi.org
Research Spons or s
Research Sponsors
Adaptive Planning
ADVIZOR Solutions
Esri
Pentaho
SAS
Tableau Software
Third QUARTER 2013
TDWI best practices Report
Table of Contents
Research Methodology and Demographics . . . . . . . . . . . . . . . 3
Executive Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Data in the Eye of the Beholder . . . . . . . . . . . . . . . . . . . . . 5
Increasing Expectations for Visual Excellence . . . . . . . . . . . . . 5
Matching Visual Interfaces to User Requirements . . . . . . . . . . . 6
Who Develops and Deploys Visualizations? . . . . . . . . . . . . . . 10
Business Benefits, Barriers, and Objectives . . . . . . . . . . . . . 12
Visualization and Performance Management for Business Objectives . 13
Implementation Practices for Better Decisions . . . . . . . . . . . 16
Geospatial Analysis and Visualization . . . . . . . . . . . . . . . . . 17
Provisioning Data for Visual Access and Analysis . . . . . . . . . . 19
In-Memory Computing for Visual Analysis and Discovery . . . . . . . 19
Dashboard Strategies and Data Visualization . . . . . . . . . . . . 21
Visualization Functionality and Chart Types . . . . . . . . . . . . . 25
Most Popular Visualization Types . . . . . . . . . . . . . . . . . . . 26
Vendor Products . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
© 2013 by TDWI (The Data Warehousing InstituteTM), a division of 1105 Media, Inc. All rights reserved. Reproductions in whole
or in part are prohibited except by written permission. E-mail requests or feedback to [email protected]. Product and company names
mentioned herein may be trademarks and/or registered trademarks of their respective companies.
tdwi.org 1
Data V i s u a l i z at i o n a n d D i s c ov e r y
About TDWI
TDWI, a division of 1105 Media, Inc., is the premier provider of in-depth, high-quality education
and research in the business intelligence and data warehousing industry. TDWI is dedicated to
educating business and information technology professionals about the best practices, strategies,
techniques, and tools required to successfully design, build, maintain, and enhance business
intelligence and data warehousing solutions. TDWI also fosters the advancement of business
intelligence and data warehousing research and contributes to knowledge transfer and the
professional development of its members. TDWI offers a worldwide membership program, five
major educational conferences, topical educational seminars, role-based training, onsite courses,
certification, solution provider partnerships, an awards program for best practices, live Webinars,
resourceful publications, an in-depth research program, and a comprehensive website, tdwi.org.
Acknowledgments
TDWI would like to thank many people who contributed to this report. First, we appreciate the
many users who responded to our survey, especially those who responded to our requests for phone
interviews. Second, our report sponsors, who diligently reviewed outlines, survey questions, and
report drafts. Finally, we would like to recognize TDWI’s production team: Jennifer Agee, Michael
Boyda, and Denelle Hanlon.
Sponsors
Adaptive Planning, ADVIZOR Solutions, Esri, Pentaho, SAS, and Tableau Software sponsored the
research for this report.
Canada 8%
Survey Demographics. The largest percentage of survey
Australia/New Zealand 5%
respondents are business executives and sponsors/users (48%);
included in that group are business analysts (12%) and data Asia/ Pacific Islands 4%
analysts or scientists (15%). Forty-two percent are data and South Asia 3%
the largest segments in this second group. Respondents from Middle East 2%
1,000–10,000 33%
Other Research Methods. TDWI conducted telephone
100–1,000 19%
interviews with business and IT executives, VPs of BI/DW,
Less than 100 16%
business and data analysts, BI directors, IT application
managers, and experts in data visualization and visual data
Based on 453 survey respondents.
discovery. TDWI also received briefings from vendors that
offer related products and services.
tdwi.org 3
Data V i s u a l i z at i o n a n d D i s c ov e r y
Executive Summary
Tools and practices for Far from mere “eye candy,” data visualization is critical to fulfilling widely held goals for expanding
data visualization, data organizations’ analytics culture and driving more decisions with data. Across organizations,
discovery, and visual employees who are subject matter experts in areas such as marketing, customer service, online
analysis are enabling engagement, finance, and more need to interact with data and analyze it for significant patterns,
“nontechnical” users to trends, and anomalies. Yet, most of these professionals would hardly consider themselves “business
make effective use of intelligence users,” much less professional data scientists or data analysts. Tools and practices for data
data and reduce their visualization, data discovery, and visual analysis are enabling these “nontechnical” users to make
time to insight. effective use of data and reduce their time to insight.
Data visualization sits at the confluence of advances in technology, the study of human cognition
and perception, graphical interfaces, widespread adoption of standards for rich Internet applications,
and the continuing expansion of interest and experience in analytics and data discovery. Data
visualization can contribute significantly to the fruitful interpretation and sharing of insights
from analytics, enabling nontechnical SMEs to perform data discovery in a self-directed fashion.
Implementation of chart engines and the growth in the number and variety of visualizations
available in graphics libraries are supporting new sophistication in visual analysis, allowing users
to go beyond simple bar and pie charts to express more advanced insights about quantitative
information.
This TDWI Best Practices Report focuses on how organizations can use data visualization, visual
analytics, and data discovery to improve decision making, collaboration, and operational execution.
The report provides analysis of an in-depth research survey and user stories to reveal current
strategies and future plans for data visualization and analysis. The report offers recommendations
for successfully evaluating and deploying data visualization, data discovery, and visual analysis
technologies to achieve shorter time to insight for users across the enterprise.
Users need data visualization for a variety of BI and analytics activities, including reporting,
scorecards, operational alerting, and data discovery and analysis. Rather than just giving users “new
toys” to play with, organizations should examine how they can match visualization technologies
and practices to user requirements. Across the board, however, a key element in the success of
visualization is data interaction; users need broad capabilities for manipulating data, including to
drill down, cross cut, slice, and dice data directly from graphical interfaces.
For many organizations, dashboards take the center stage for data visualizations, especially for BI
reporting and performance management. Many users would like to consolidate views of multiple
sources and types of information into their dashboard workspaces. One new source of interest is
geographical information. Although using maps to enhance corporate data (and vice versa) is not yet
widespread, organizations in a growing number of industries are interested in geospatial analysis as
an addition to their visualization repertoire. Mobile device adoption is likely to accelerate interest
in visualizations offering location information and geospatial analysis; frontline employees in sales,
service, and support will use these technologies to enhance customer interactions.
Data visualization TDWI Research finds that organizations are pursuing a range of potential business benefits with
and discovery can their current and planned implementations of data visualization and discovery technologies.
help reduce the time Operational efficiency is the top benefit sought, according to our research; organizations seek to
users lose when implement data visualization and discovery to reduce the time users lose when they have difficulty
they have difficulty accessing, reporting, and analyzing data. With self-directed capabilities for uncovering root causes as
accessing, reporting, well as other insights from data, organizations will be able to move away from gut feel and common
and analyzing data. wisdom and use data to drive innovation in strategy and operations.
tdwi.org 5
Data V i s u a l i z at i o n a n d D i s c ov e r y
likely to want to integrate non-relational sources such as geospatial data. This report will discuss the
significance of this trend.
Although the seeming ubiquity of BI dashboards would make it appear that data visualization must
be commonplace, the reality is that most organizations are in the early stages of adoption and are still
learning what users need. TDWI Research finds that only 7% of the 453 respondents to the survey
for this report are “very satisfied” with their ability to view and interact with data visually so that
they can communicate information effectively and make decisions based on information through
graphical means. Almost a quarter (23%) said they are “not satisfied,” and about one-third each are
somewhat satisfied or somewhat dissatisfied. This report will discuss many of the factors that affect
user satisfaction.
Dashboards and scorecards should allow users to drill down to at least one if not more levels of data
to gain a detailed view behind the KPIs, metrics, and visual objects. Users should be able to spot
anomalies easily and recognize where performance is out of line with expectations. More mature and
successful dashboard implementations will allow users not only to drill down into data, but also to
perform various types of analysis to explore patterns or determine the root cause for why numbers
are out of line with expectations.
Three out of five (57%) of respondents to the TDWI Research survey are currently implementing
display or snapshot reports and/or scorecards; 31% are planning to do so, and 9% have no plans for
them (see Figure 1). Among the 7% of respondents noted earlier who are very satisfied with their
ability to view and interact with data visually, 77% are currently implementing visualization for
these types of activities.
Are users in your organization currently implementing or planning to implement data visualization
technologies for the following activities?
Currently implementing
Plan to implement
No plans
Don’t know
Display, snapshot 57%
31%
reporting, and/or 9%
scorecards 3%
26%
39%
Operational alerting 27%
8%
Visual data 33%
45%
discovery and 17%
analysis 5%
Operational Alerting
Many BI and visual discovery tools offer alerting functions. These are designed to notify personnel of
particularly important changes in the data or when situations arise that demand immediate attention.
Color is often used for alerts on dashboards. Some tools enable users to perform visual analysis
to discover why an alert condition exists, such as if sales have decreased for a certain product line.
Although alerts are often set ahead of time to be coordinated with KPIs and business rules, some
tools allow users to custom-tune alerts for particular data changes and their preferred visualizations.
Operational alerting, as the term implies, is an important ingredient in emerging operational Operational alerting is
intelligence systems. These systems focus on monitoring activities in business, distribution chains, an important ingredient
manufacturing, networks, IT systems, and more for problems, threats, and other critical in emerging operational
developments. Alerts signify that monitors have uncovered something important in (often) real-time intelligence systems.
data or event streams; some systems provide analytics for determining root causes and the best way
to address situations.
TDWI Research shows that only about one-quarter (26%) of respondents are implementing data
visualization technologies for operational alerting. More (39%) are planning to do so, while 27%
have no plans (see Figure 1). This could reflect the current immaturity of the emerging operational
intelligence field.
tdwi.org 7
Data V i s u a l i z at i o n a n d D i s c ov e r y
USER STORY
Dartmouth sharpens fundraising focus with visual analysis.
Fundraising from alumni sources has long been critical to the economics of colleges and universities. Alumni
fundraising becomes even more important as costs rise and facilities require upgrades. Colleges and universities
must be smarter about how they research and identify prospects in alumni rosters. By analyzing records that
go back decades as well as external demographic sources, institutions can discover who might have the right
combination of wealth, inclination, and affinity to be motivated to give back.
Dartmouth College, one of the premier higher education institutions in the U.S., captures a broad range of
information about its alumni—from when they were undergraduates and engaged in sports or other on-campus
activities to the progress of their careers, travels, and advanced degrees. Its large Ellucian Advance donor,
prospect, and events management system database contains information on about 120,000 constituents. “Our
responsibility is to go through all that information to pull out prospects who have the highest probability to give
back, either in general or for specific campaigns such as funding a capital project,” said Michael Foote, director of
Research and Prospect Management at Dartmouth College.
In analyzing the data, Foote said that Dartmouth wanted to avoid the usual “cycle of pain”: that is, giving
requirements to an IT reporting group, waiting for programmers to do manual lookups and build reports,
discovering that the results were not quite what was needed, and then having to resubmit the request and start
the cycle over again. Dartmouth deployed ADVIZOR Solutions to automate the linking together of separate, siloed
data tables and a visual presentation of the results. “Within seconds, I can immediately see how many in the class
of 1978, for example, are in a particular field of work, how many of those participated in football, what their giving
history has been, and so on,” said Foote. “I can initiate other types of analysis myself.”
Dartmouth uses ADVIZOR’s business analysis software to visualize relationships in the data and find outliers—
“the folks we may have missed,” Foote said, due to improper coding or other data cleansing issues. Foote’s team
has been able to build demographic profiles of individuals at different giving levels and use predictive analytics
built into the software to mine the data, analyze variables, and attach scores to statistically measure wealth,
inclination, and affinity. “We built a model based on a target group of current high donors and then applied that
model to everyone else in our database. This has helped us ensure that our prospect managers are focused on the
right alumni at the right time.”
Dartmouth’s scatterplot chart shows proposal status by age, color-coded by primary staff member name. It also
includes a bar chart filter that allows users to locate proposals in the pipeline by “ask amount” totals. Note the large
number of proposals in the “qualification” stage that are more than 6 months old. (Click image to enlarge.)
tdwi.org 9
Data V i s u a l i z at i o n a n d D i s c ov e r y
Who in your organization develops and deploys visualizations (e.g., different types of charts, graphs, or
maps) for users to implement? (Please select all that apply.)
IT developers 62%
Figure 2. Based on answers from 453 respondents; respondents could select more than one answer.
Executives do not develop visualizations, but do highly value them. Figure 2 shows that business
executives—the dominant users of BI tools in many organizations—do not commonly develop and
deploy visualizations; only 13% of respondents said that they do so. However, TDWI Research
finds that executive management highly values visualizations for display or snapshot reporting,
and/or scorecards; 81% of respondents said that visualizations for these activities are important
for executives (see Figure 3). Somewhat less critical are visualizations for executives’ operational
reporting and visual data discovery and analysis; just 27% and 37% of respondents indicated
that visualizations are important to executives for these activities, respectively. Once again, it is
possible that this is because dashboards for reports and scorecards are more mature in organizations
compared to visual operational reporting, analytics, and data discovery.
Figure 3 offers a view of the relative importance of visualizations for different types of users for the Visualizations are
three primary activities identified in this report. Nearly two-thirds of respondents (64%) said that important to marketing
visualizations are important to marketing and market analysis users for visual data discovery and and market analysis
analysis. In many organizations, the marketing function is the first to adopt new tools for analytics users for visual data
and discovery to improve insight into customer behavior and use data discoveries to fine-tune discovery and analysis.
campaigns. Half of respondents (50%) said that visualizations for these activities are also important
to users in finance, which is evidence of the increasing role of discovery analytics for CFOs and other
managers seeking a broader and deeper view for corporate financial management.
Compared with most other departments, visualizations for operational alerting are the highest
priority for IT, network, and computer security management (53%). Managers in these functions
require real-time alerts regarding threats, performance, and resource utilization problems; data
visualizations can provide faster recognition of actionable information and help administrators
respond proactively to negative trends. Customer service and support (46%) is another function
where operational reporting is critical. Rather than waiting out the lag for business or data analysts
to provide contact center managers with historical reports, visual operational intelligence can enable
managers to see and respond to customer concerns in real time.
For users in the following functions in your organization, which of these main data visualization activities
are important? (You may choose more than one answer per row.)
Display, snapshot reporting, and/or scorecards
Operational alerting
Visual data discovery and analysis
All other responses
70%
39%
Finance 50%
17%
65%
32%
Sales 47%
28%
81%
Executive 27%
management 37%
11%
59%
Marketing and 19%
market analysis 64%
24%
Operations 44%
44%
management and 44%
research 33%
IT, network, or 40%
53%
computer security 26%
management 34%
48%
Customer service 46%
and support 27%
28%
43%
Governance, risk 39%
and compliance 33%
36%
Online presence 35%
18%
management / social 42%
media marketing 46%
Figure 3. Based on answers from 432 respondents; respondents could select more than one answer per row.
Organization functions are listed in order of most total responses.
tdwi.org 11
Data V i s u a l i z at i o n a n d D i s c ov e r y
The user story below illustrates how visual reporting enabled OLX to respond more effectively to the
challenges of rapid business growth and increasing volumes of data.
USER STORY
OLX deploys visual business intelligence to guide rapid growth.
OLX, a fast-growing “Craigslist for the rest of the world” online classified advertising site, is luring investors
from around the globe as it picks up speed. Based in Buenos Aires, OLX is active in more than 105 countries
and is strong in Latin America and Asia; it supports more than 40 languages. With more than 125 million
unique visitors per month worldwide, OLX is generating up to one billion page hits per month, according to
Francisco Achaval, manager of business intelligence at OLX. In addition, since the website allows users to design
and personalize advertisements and display them in their social networking profiles, the big data analytics
opportunities are enormous.
As tempting as the diverse, non-relational data is, the first step for OLX has been to bring its already considerable
relational data velocity under control by building a data warehouse to support its main key performance indicators.
OLX has been implementing Pentaho Business Analytics and data integration software to provide users with
visual, actionable data so that “business users in different areas of the company can, in a self-service fashion,
make changes based on the insights they are getting,” said Achaval.
“The challenge has been not so much the number of metrics as the number of member dimensions that business
users want to touch,” Achaval explained. With OLX’s supply and demand business data model, the metrics are
fairly straightforward for the supply of items, demand in terms of number of customer users, and the conversion
to sale. But, with 105 countries, 700 categories, and more than 4,000 cities to analyze, aggregating the data
can be daunting. Visualization has been critical to the analysis because “while it may be easy for a BI analyst to
understand what’s happening in the numbers, to explain this to business users who are not versed in BI or OLAP,
you need visualization.”
OLX is embarking on projects to automate how the company uses KPIs and analytic insights to respond more
quickly. “We want to be able to automatically launch campaigns if we see that demand is low in a certain city or
category.” The company is also preparing to tackle big data to enable visual analysis of its variety of unstructured
data sources.
Operational efficiency is the foremost benefit sought. TDWI Research finds that improved operational
efficiency is the business benefit that most organizations surveyed (77%) hope to gain from
deploying data visualization and visual analysis technologies (see Figure 4). Poor information flow
to employees who are directly responsible for process efficiency and optimization is the Achilles’ heel
of many organizations. Employees are often delayed in operation execution when they have to look
for the right information across siloed applications and interfaces. Dashboards that can consolidate
information for easy-to-use reporting and analysis will contribute to operational efficiency.
Faster response to business change (62%) is the second most common priority among respondents.
Leaders in most organizations understand that if they can analyze data and feed insights sooner
and more frequently to decision makers, they will realize advantages over firms that are locked into
slower decision cycles. Self-service visual data discovery and analytics can relieve decision makers
of many steps in the traditional dance with IT to gain access to data and develop new reports and
visualizations. The third highest potential business benefit cited by respondents was the ability to use
visualizations to identify new business opportunities (59%).
Which of the following are the most important business benefits that your organization seeks to gain
from deploying data visualization and visual analysis technologies? (Please select all that apply.)
Improved operational efficiency 77%
Figure 4. Based on answers from 424 respondents; respondents could select more than one answer.
Skills and budget shortages are the biggest barriers. We asked respondents to name the most
significant barriers to adoption of data visualization and analysis. The top three were lack of skilled
personnel or training, not enough budget or resources, and difficulty identifying or quantifying hard
returns. The barrier indicated by the smallest percentage of respondents was insufficient computing
or networking for visualization, followed by scalability to serve increasing numbers of intended users.
These results suggest that the biggest perceived barriers are not about whether organizations have the The greatest perceived
technology infrastructure to support data visualization and analysis. Even data quality is not a barrier centers on
topmost concern; only about a third of respondents said this was a barrier. The greatest concern whether employees have
centers on whether employees will have adequate knowledge and skills to make effective use of the adequate knowledge and
tools and whether deployment can be justified from a business perspective. skills to make effective
use of the tools.
Visualization and Performance Management for Business Objectives
Organizations have a variety of information objectives guiding their implementation of data
visualization and visual analysis technologies. In Figure 5 (page 15), we can see how respondents to
the TDWI Research survey view 10 objectives that are priorities for most organizations. More than
half (55%) said that improving performance metrics and measures is very important, and another
33% said it was somewhat important. Nearly the same percentages indicate that enhancing users’
tdwi.org 13
Data V i s u a l i z at i o n a n d D i s c ov e r y
self-directed BI and data discovery is a high priority (54% and 31%, respectively). TDWI Research
finds that these two objectives are increasingly related.
Employees closer Performance management metrics and scorecards, usually presented to employees through
to their areas of dashboards, have been somewhat successful for aligning decisions and actions with strategic and
responsibility often financial objectives. The most difficult parts of performance management are often defining the KPIs
know the most about the and then ensuring that users have access to valid data to implement the measures. Managers and
data, but they lack the employees who are held accountable for certain metrics and measures need actionable information so
tools to interact with it that they can make positive changes or alert upper management when additional resources are
effectively. needed. Because they are closer to their areas of responsibility, they often know more about the data
than corporate management does, but they lack the tools to interact with it effectively. Self-directed
BI and data discovery could be helpful in giving such employees the tools they need to perform.
The user story below offers an example of how visual analytics and discovery plays a key role in
performance management.
USER STORY
Visual discovery delivers the big picture for financial analysis at AAPA.
Reducing time to insight is critical for many organizations, but sometimes not all departments and divisions
get the tools they need to make this happen for their concerns. Executives and customer-facing groups such as
marketing, sales, and service usually come first. Finance and business management users are often left to use
spreadsheets and siloed back-end accounting systems, with custom coding required to supply data for analysis.
If these users implement more sophisticated budgeting, forecasting, and planning applications, these are often
removed another step further from the data.
However, with data analysis becoming ever more essential to financial performance management, business and
finance managers are beginning to implement tools that enable them to easily drill down into the data behind key
performance indicators and scorecards in their budgeting, forecasting, and planning applications. Visual discovery
can decrease time to insight for performance management. With self-service capabilities, the tools can reduce
users’ dependency on IT to custom-build visual reports and code access to data.
Integrated tools have been a boon for the American Academy of Physician Assistants (AAPA). Based in Alexandria,
Virginia, the Academy is the national professional society for physician assistants (PAs); it advocates and
educates on behalf of the profession and the patients PAs serve. AAPA works to ensure the professional growth,
personal excellence, and recognition of PAs. Shyam Desigan, CFO and SVP of IT at AAPA, implemented the cloud-
based Adaptive Planning suite, which includes the visual discovery tool Adaptive Discovery. AAPA uses the suite to
integrate its visually interactive dashboards and scorecards with planning, budgeting, and forecasting processes.
Users can automate access from visualizations to data in the Academy’s back-end Microsoft Dynamics SL
financial management and accounting system.
“I can point my cursor at a particular number, such as the cash-flow variance, and drill down from the visual
discovery graph right in to the transactions to see the variance,” Desigan said. “I describe it like a camera. Visual
discovery gives the global view; as part of the integrated suite, we can use it to pan around or zoom in to get a
more close-up, granular view of data in the back-end system.” Desigan said that the visualization is particularly
helpful for seeing how things are moving, the variances in budgets, and other key trends. “The integrated tools cut
out a lot of the prep work to get data; developing visual graphs is automated.”
Users need visualization to create a single view of information. Figure 5 shows that organizations put
a high priority on using data visualization to create a single, graphical view of information; 50% see
this as very important and 34% as somewhat important. Organizations implementing advanced
dashboards are able to provide mashups of data from multiple sources, both internal and external,
including news and social media feeds. Such dashboards can improve employee productivity where
staff currently find it necessary to jump from report to report and across different applications to
gain a complete view.
Managers and employees who need to increase their level of data analysis require integrated access to For some users,
information as well as tools for easier implementation of advanced analytics, according to TDWI advanced analytics can
Research. For some users, advanced analytics can mean OLAP; for others, it includes activities such mean OLAP; for others,
as testing, training, scoring, and monitoring predictive models that involve many variables. Machine it includes activities
learning, neural networks, and other statistical, quantitative, or mathematical modes of deeper data such as testing, training,
analysis may also be part of an advanced analytics effort. scoring, and monitoring
predictive models that
Nearly half (47%) of respondents said that they hope to implement data visualization and analysis
involve many variables.
technologies to improve the ease of use of advanced analytics; one-third said it was somewhat
important. Visualization for big data analytics is also a priority for many; a significant percentage
(41%) find it very important (and 29% find it somewhat important) for visualization and visual
analysis to help them detect patterns and trends in big data sources. Less important currently is using
visualization for improving analysis of text and unstructured data. Just under one-quarter (23%) said
this was a very important objective, while 26% said it was less important and 22% said it was not
important at all. It is likely that this is due to the general immaturity of text and unstructured data
analysis in most organizations.
How important to users in your organization are each of the following objectives for implementing data
visualization and analysis technologies?
Enhance users’ self-directed BI and 54% 31% 9% 2% 4%
data discovery
Create a single, graphical view of 50% 34% 7% 5% 4%
information
Enable user collaboration and learning 27% 38% 25% 6% 4%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Very important Somewhat important Less important
Not important Don’t know
Figure 5. Based on answers from 416 respondents; respondents could select one answer per row. Selections are ranked
by number of responses.
tdwi.org 15
Data V i s u a l i z at i o n a n d D i s c ov e r y
Which of the following business analysis, reporting, and alerting activities are currently deployed for
users in your organization through implementation of data visualization and visual analysis technologies?
(Please select all that apply.)
KPI definition and delivery 60%
List reduction 6%
Figure 6. Based on answers from 408 respondents; respondents could select more than one answer.
Visualizations enable new forms of collaboration on data. Many tools allow users to publish charts, not
only in dashboards for viewers to share, but also through e-mail and collaboration platforms such as
Microsoft SharePoint. Dashboards can deliver context for visualizations by providing annotations
and related charts, since one chart often cannot tell the whole story. Other means of storytelling,
including animation or video and audio files, may be part of the collaboration.
Storytelling is important because visualizations are usually—and often, intentionally—left open to Some charts may
interpretation. Different viewers can draw different interpretations, which they can investigate by hide the importance
drilling down into the data. Some charts may hide the importance of certain factors, while others of certain factors,
might exaggerate them. This ambiguity makes it important for executives, managers, and users to while others might
work with visualizations as tools to engage in a productive dialogue about metrics and measures. exaggerate them.
Organizations can use visualizations to overcome the “one-way street” limitations often cited as the
bane of performance management and standard BI reporting.
Time series analysis is an important focus. A significant percentage of respondents implement
visualizations for time series analysis (39%). Users in most organizations need to analyze change
over time, and they typically use various line charts for this purpose. Some will also apply more
exotic visualizations such as scatterplots for specialized time series analysis, including examining
correlations over time between multiple data sources. Visualizations for pattern and trend analysis,
often related to time series analysis, are employed by 35% of respondents.
Time series, pattern, and trend analysis complement predictive analysis. Organizations want to use
history to forecast what will happen next and identify what factors will cause patterns to repeat
themselves. Almost a third (32%) of respondents use visualizations for forecasting, modeling,
and simulation, and 22% are doing so for predictive analysis. Again, visualizations can improve
vital collaboration on predictive analysis among different subject matter experts, who can share
perspectives and help the organization adjust strategies to be proactive. The organization will
anticipate events and be prepared with the most intelligent way to respond.
tdwi.org 17
Data V i s u a l i z at i o n a n d D i s c ov e r y
USER STORY
Planet Fitness uses visual GIS analysis to find the sweet spots for retail.
Where should we locate a new store? What kinds of customers will come, and how far are they willing to travel?
How will new zoning laws affect our strategy? Answering these questions can be critical to the success of retail
organizations, and questions of a similar nature often confront decision makers in real estate, business and
industrial development, healthcare services, city planning, and more.
Rather than having to look at data silo by silo, GIS analysis can help decision makers by giving them a single
mashup view of relevant data within the context of maps. “All of this comes together to paint you a picture of a
story that is in fact already there in the data,” said Matt Felton, president of Datastory Consulting. “But if you
don’t have the right lens to see it, you can’t see it.”
Datastory Consulting, an offshoot of the Maryland-based commercial real estate firm MacKenzie, is helping Planet
Fitness use the Esri ArcGIS platform to sharpen its strategy for locating new facilities. Planet Fitness has been
growing fast; it has more than 600 low-cost health club locations in the U.S. With GIS analysis playing a key role,
the company has been active in Maryland, opening 20 locations in just the first quarter of 2013. Working closely
with John Schultz, senior vice president and principal at MacKenzie, which has a brokerage relationship with
Planet Fitness, Datastory enables Planet Fitness to bring multiple variables to bear on finding location “sweet
spots.” Through visual data interaction with “cloud maps”—what Datastory calls its rich, online data mashups
of different sources shown on GIS maps—Planet Fitness decision makers can consider options even as new and
ongoing developments take place in the location scenario.
“The cloud maps work as a collaboration tool to track current locations, the potential sites Planet Fitness is
considering, and changes that affect the status of all sites,” said Felton. Decision makers can apply variables and
attributes in the cloud maps and update the status of potential sites as they become more or less interesting, as
construction begins on chosen locations, or as other milestones in the process of opening a site occur. “Deals get
done smarter and faster, with a clearer perspective, after looking at data and sometimes thousands of variables
to narrow down to sites with the highest potential,” Felton said.
Geospatial analysis of the potential for Planet Fitness club membership cannibalization among facilities in the
same region. (Click image to enlarge.)
tdwi.org 19
Data V i s u a l i z at i o n a n d D i s c ov e r y
In your organization, where physically does data that is accessed for users’ visual analysis and
discovery reside? (Please select all that apply.)
Database or file on disk 76%
On mobile device 7%
Figure 7. Based on answers from 393 respondents; respondents could select more than one answer.
Adoption of 64-bit operating systems has made it easier for developers and users of BI and analytics
systems to exploit very large memory and bring powerful functions closer to the data. With
in-memory computing, the traditional I/O bottleneck constraint—where queries have to read
information from tables stored only on disk—becomes less of a factor. Users can perform, on their
own, types of analysis that would be too slow with disk-dependent systems and limited in scope
because not enough data is available. In-memory computing could therefore be an advantage for
complex, highly interactive analytics or in circumstances where it would hurt the performance of
operational data sources to go against live data.
Several leading visual discovery and analysis tool providers include an in-memory data mart as part
of their applications. These integrated offerings load data from a variety of disk and spreadsheet
sources into memory for display and analysis. By integrating this data automatically, the applications
avoid the need for additional IT setup and management.
However, in-memory computing is not a silver bullet, nor is it the right approach in all cases. As data
volumes rise, management of the memory space can become an issue. Organizations also have to
consider how frequently they need to update the data in memory. Data management issues can arise
if users are essentially creating in-memory, siloed data marts that then become difficult to update and
maintain from a data quality perspective.
Most respondents Overall, TDWI Research finds that most respondents give the speed of data access and selection for
give the speed of data data visualizations a middling grade, with just over a third somewhat satisfied and just under a third
access and selection somewhat unsatisfied. Only 6% are very satisfied; however, this segment is implementing in-memory
for data visualizations computing at a higher rate. Nearly two-thirds of these respondents are working with data in memory
a middling grade. but on a different server or system from where visualization types are rendered; 42% of these
respondents have data in memory on the same machine.
The user story below describes the potential XL Group sees for implementing visual analytics on big
data sources.
USER STORY
XL Group taps visual analysis to improve insights and enhance collaboration.
In a data-driven world, business innovation depends on a strong bond between subject matter experts who know
the business and data analysts and scientists who have expertise in finding patterns, trends, and correlations
in the data. In many organizations, however, the two cultures live separately; without a common language,
communication and collaboration are difficult. As a result, organizations can be slow to realize potential
competitive advantages and may miss shifts in customer demand.
New big data sources are creating “powerful and potentially transformative analytical opportunities” at XL Group,
according to Kimberly Holmes, head of strategic analytics. XL, through its subsidiaries, is a global insurance and
reinsurance company that offers property, casualty, and specialty products to firms throughout the world. The
internal and external big data sources are giving business and data analysis teams much larger data sets than
they have ever had, in order to support decision makers who analyze risk and develop strategic and potentially
profitable new products. “Our mandate is to develop decision-making tools using a broader set of data and
advanced analytics to enable business leaders to make better decisions.”
By implementing advanced analytics, XL is “testing conventional wisdom and pushing beyond hindsight and
one-way analysis,” said Holmes. A critical objective of the implementation is “minimizing the challenges of
conveying complex relationships in the data to decision makers.” XL is in the early stages of implementing SAS
Visual Analytics to “bridge the gap between the business decision maker’s expertise and that of the analyst.” SAS
Visual Analytics, which XL is deploying as a hosted system, offers advanced visualization backed by an in-memory
analytics engine.
“Visual analytics is a way of enabling business leaders to explore data and look for patterns without having
an extensive education in data analysis,” Holmes said. “If the business leader is working alongside the analyst
and leading the exploration of the data, it will lead to deeper understanding of the data and the business. This
collaboration is easier with visual analytics because the business leader can see the story in the data.”
tdwi.org 21
Data V i s u a l i z at i o n a n d D i s c ov e r y
Who are the primary users of your organization’s dashboards or similar visual data analysis, access, and
reporting workspaces for users? (Please select all that apply.)
Business executives and management 77%
Customers 14%
External partners/suppliers 8%
Figure 8. Based on answers from 388 respondents; respondents could select more than one answer.
22 TDWI rese a rch 1 See the 2012 TDWI Best Practices Report Mobile Business Intelligence and Analytics: Extending Insight to a Mobile Workforce,
available at tdwi.org/bpreports.
Dashboard Strategies and Data Visualization
More than half of respondents (53%), however, are somewhat dissatisfied or worse with self-service More than half of
customization of look, feel, and scope. This indicates that many dashboards are not giving users the respondents are
flexibility that they need. As further evidence of this weakness, about half of respondents (51%) are somewhat dissatisfied or
somewhat dissatisfied or worse with their ability to update or add new information from their worse with self-service
dashboards. A similar percentage (54%) are dissatisfied with their dashboard’s adaptability to new customization of look,
business conditions. Organizations are clearly in need of tools and practices that will allow their feel, and scope.
dashboards to be more adaptable to dynamic business requirements and more capable of giving users
scope for self-service customization.
How satisfied are users in your organization with the following capabilities or qualities in their
dashboards or similar graphical user workspaces?
Clarity of dashboard’s business 13% 43% 19% 10% 15%
purpose
Dashboard fitness for business 7% 30% 26% 17% 20%
processes
Ability to update or add new 7% 24% 23% 28% 18%
information
Data drill-down for KPI/metrics 12% 34% 23% 16% 15%
Figure 9. Based on answers from 374 respondents; respondents could select one answer per row.
The user story below illustrates the role dashboards and self-directed visual analysis can play in
helping a small business to grow by using data effectively.
USER STORY
Visual data discovery enables Lucky Voice to “punch above its weight.”
“Spreading happiness through unforgettable singing experiences” is the mission statement of Lucky Voice, founded
in 2005 as a provider of private-room karaoke venues and now expanding to offer online experiences and software
technology for use by other companies. Lucky Voice is enabling closet singers in the UK—and beyond, via the
Internet—to overcome shyness, enjoy a few drinks with friends, and let out the inner crooner. “It’s serious in
a business sense but not serious in that we’re just selling fun,” said Nick Thistleton, co-founder and managing
director of Lucky Voice.
Thistleton has been driving Lucky Voice’s expanding use of Tableau Software for visual, interactive, and actionable
data analysis and reporting. It started when the company had built its own booking system and needed reporting.
“I asked our teams to come up with their reporting requirements and we ended up with a massive list for detailed
reports,” Thistleton recalled. “Our in-house software developers said, ‘Okay, we can do this, but it will take weeks
and when we finish, you’ll want us to change things and you’ll have new reports that you hadn’t thought of.’”
tdwi.org 23
Data V i s u a l i z at i o n a n d D i s c ov e r y
The company’s developers suggested that Thistleton look at third-party software, which seemed like a big leap for
a small company. “I downloaded a couple of trials and didn’t know where to start,” he said. “You needed to be a
data analyst to even understand how to work the programs.”
Once introduced to Tableau Software by a friend, Thistleton was able on his own to start developing meaningful
and highly visual reports from the booking system. “Then, I started to plug into other systems that were
generating data, including systems that could tell us about customer behavior, such as what songs people
like to sing, what they are drinking, and their satisfaction. We kind of taught ourselves how to get at least basic
visualizations. It was also a huge leap forward to be able to access live data rather than having to download data
into a spreadsheet and create pivot tables just to see old data. We are continuing to plug into sources we had
never thought about to find out things we didn’t know.”
Thistleton said company personnel are using dashboard reports at all levels. Venue managers, for example, can
develop dashboards to see the budget for the night and how much revenue has come in so far. “They can click
on what drinks have sold and whether we need to create incentives to sell more of something to meet budget.
Everyone from the top to the bottom of the business has some kind of relationship with the Tableau system. We
have only been using it for about a year, but already, if you took it away from us, we wouldn’t know what to do
with ourselves.”
Lucky Voice’s on-shift dashboard enables employees to see the budget for the night and how much of selected items
has been sold so far, as well as who is working that night and what they have sold. (Click image to enlarge.)
Current use of coloring and focus is significant. In Figure 10, we can see that about 4 in 10 survey
respondents indicated that users in their organizations are implementing tools for color-coding data
and creating linkages to charts (45%), and coloring, brushing, and highlighting in charts (40%).
Color is vital to conveying information; humans are sensitive to the culturally conditioned meaning
of colors, which means that users have to choose them carefully. They must also be aware that many
humans are color blind and cannot differentiate easily between certain colors.
Functionality requirements vary somewhat by activity. Looking at the results for Figure 10 filtered for For those using data
the three main data visualization activities described earlier in this report (see Figure 1), we can gain visualization for visual
insights into functional priorities. For respondents who are currently implementing data visualization discovery and analysis,
technologies for display or snapshot reporting, and/or scorecards, the highest currently used filtering, drill-down, and
functions are filtering, data drill-down, and coloring of data and linkage to charts. For those who are coloring are the most
currently implementing data visualization for operational alerting, not surprisingly, alerts rise to the used functionalities.
third highest in current use; at 64%, the percentage for this segment is twice the overall current-use
percentage shown for this functionality in Figure 10 (32%). Finally, for those using data visualization
for visual discovery and analysis, filtering, drill-down, and coloring are the most used functionalities.
tdwi.org 25
Data V i s u a l i z at i o n a n d D i s c ov e r y
Which of the following types of functionality are currently deployed or are planned to be deployed in your
organization’s dashboards or visual analysis applications?
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Figure 10. Based on answers from 356 respondents; respondents could select one answer per row.
how parts compare to the whole, these standard types can be shared with an expectation that other
users will easily grasp their meaning.
Significant shares of users implement more advanced visualization types. Users who are performing
more advanced visual analytics are implementing a range of somewhat more specialized types to
understand data and communicate insights with others. These include histograms (44%), which are
usually displayed as a type of bar graph that expresses the frequency distribution of data; scatterplots
(43%), which help users to see correlations between variables; and heat maps (36%), which use color
to show relative values and “what’s hot” compared to other values in a matrix. Tree maps, used by
17% of respondents, are a variation on heat maps.
Which of the following visualization types are currently being implemented by users in your organization?
(Please select all that apply.)
Bar charts 91%
Tables 70%
Spreadsheets 64%
Counts 57%
Histograms 44%
Scatterplots 43%
Statistics 41%
Sparklines 29%
3D visualizations 13%
Logical maps 9%
Word clouds 9%
Data constellations 9%
Multiscape 5%
Para boxes 3%
Figure 11. Based on answers from 352 respondents; respondents could select more than one answer.
tdwi.org 27
Data V i s u a l i z at i o n a n d D i s c ov e r y
Because technology As with the previous chart, Figure 11 gives readers of this report a sense of the breadth of
progress is enabling visualization types available. It allows them to compare the extent of their implementations with
ever broader and what our research indicates other organizations are doing. The value of visualization types is in the
deeper data analysis, eye of the beholder; although some are not implemented by a large percentage of respondents, this
we expect no slowdown does not mean that they lack value. In fact, they could very likely deliver unique insight that more
in innovation with data common visualization types will not show. For this reason and because technology progress is
visualization. enabling ever broader and deeper data analysis, we expect no slowdown in innovation with data
visualization.
Vendor Products
The firms that sponsored this report are among the leaders and innovators in providing technologies
for data visualization, data discovery, and visual analytics. To get a sense of where the industry
as a whole is headed, the next section takes a brief look at the portfolios of these vendors. (Note:
the vendors and products mentioned here are representative, and the list is not intended to be
comprehensive.)
tdwi.org 29
Data V i s u a l i z at i o n a n d D i s c ov e r y
Esri Pentaho
Founded as Environmental Systems Research Institute in Pentaho, founded in 2004, has an open source heritage and
1969 by Jack and Laura Dangermond, Esri began producing provides commercial professional and enterprise editions
software for mapping and GIS in 1982. The Redlands, of its Pentaho Business Analytics technologies through a
California–based company’s customer base stretches across subscription model as well as open source versions. Along
business, governmental, and NGO organizations. Esri’s with reporting, interactive data discovery, and predictive
ArcGIS platform for mapping and geographical analysis analytics capabilities, Pentaho also provides data access and
includes a range of tools and solutions for decision makers integration. Pentaho’s most recent release includes Pentaho
who need to bring together data sources with geographic Mobile, the full Business Analytics platform on Apple
information and mapping tools. Esri Location Analytics iPads, and Instaview. Built to meet demand for ad hoc
augments business systems with mapping visualizations and discovery, visualization, and exploration of large and diverse
special analytics that complement existing functionality big data sources, Instaview brings analytics, data access,
without disrupting existing business workflows. Location and preparation together in a simple, easy-to-use package.
analytics also provides access to external spatial data, Instaview enables users to specify a desired data set from the
including demographics, lifestyle, business, and weather. larger pool, and the tool will automatically aggregate the data
Esri’s easy-to-adopt solutions make them impactful and load it into an in-memory columnar database. Because
immediately to a business and can provide organizations with the system is pluggable, users can work with either Instaview’s
a platform for growth to fully exploit geospatial visualization supplied visualizations or plug their own visualizations into
and analysis. Esri also offers developer tools, free mapping the system. Organizations with standardized chart libraries
software, specialized geospatial applications, and industry- and engines or that need to use industry-standard charts can
specific ArcGIS solutions. implement them as part of Instaview.
Average Delay
40.0
10.9 min
20.0
10.0
0.0
the business’s profit performance by year, see the prediction for future
profit performance, and then examine different scenarios by adjusting Tableau Software: Airline flights and delays. This interactive
variables that influence the profit. (Click image to enlarge.) dashboard tracks and displays flight delays across several dimensions.
(Click image to enlarge.)
tdwi.org 31
Data V i s u a l i z at i o n a n d D i s c ov e r y
Recommendations
Improve data visualization and visual analysis for nontechnical users. “Nontechnical” users are by
definition not expert in the tools and practices of accessing data and creating visualizations, but
they often know their data well. Their struggle is in trying to interact with data. With visualization,
organizations can give nontechnical users easier and more powerful means of data interaction.
Match visualization capabilities to users’ types of activities. Some users, including executives, need
visualization primarily for display or snapshot reporting and scorecards. Others need operational
alerting. Still others need visual data discovery and analysis, which can demand different
visualization capabilities. Ensure that the technology deployment fits each purpose.
Increase data interactivity with broader visualization functionality. Just as many users need to go
beyond static and tabular data, they need the flexibility to explore visualization options other than
standard bar and line charts. Evaluate tools that offer more visualization options, including the
ability to plug in visualizations that may not be part of the BI tool’s library.
Use dashboards to establish a single view of information. TDWI Research finds that users would like
to see dashboards provide a complete and consolidated interface to view all the information they
need. However, at large organizations in particular, users must deal with many application interfaces.
Aim at consolidating interfaces into a single or small number of dashboards.
Make self-service data visualization and discovery capabilities a priority. The biggest trend in BI and
analytics today is self-service. Users want tools and platforms that allow them to interact with data
on their own without hands-on IT development and supervision. Self-directed visual reporting and
data discovery can free users throughout organizations to be more creative in their analysis and
selection of visualizations to express insights.
Address time series analysis requirements with improved visualization. TDWI Research shows that
evaluating how business performance and other conditions change over time is a common need for
nearly all users. Yet, some users have only primitive means of analysis for this requirement. Provide
users with greater interactivity and breadth of visualization options for time series analysis.
Evaluate geospatial analysis and GIS sources for visual analysis needs. Location is fast becoming a
vital new dimension for data analysis. Location’s relevance is growing for users in fields that may not
have considered geospatial analysis previously. Organizations should consider whether mapping and
geospatial analysis functionality could give users new insights for strategic and operational decisions.
Develop a strategy for integrating desktop and mobile dashboards. TDWI Research finds that in most
organizations, existing dashboards and visualizations are not accessible from mobile devices. Yet,
with mobile device use growing, more users will demand BI and analytics capabilities on devices. To
avoid chaotic dashboard proliferation, organizations should develop a comprehensive strategy.
Evaluate in-memory computing to support visual analysis. In-memory computing is an alternative to
the performance constraints and design complexity that exist in standard disk-based environments.
It is not always the right solution if users need access to near-real-time, live data. However, for highly
interactive and iterative visual analytics, in-memory computing should be considered.
Make improved operational efficiency a goal of data visualization and analysis. TDWI Research finds
that improved operational efficiency is the top business benefit organizations seek from deploying
data visualization and analysis technologies. Organizations should therefore ensure that the right
functionality and visualization options are available for users’ dashboard reporting and analytics as
they address operational challenges.
tdwi.org