Big Data Analytics: Profiling the Use of Analytical Platforms in User Organizations
This report examines the rise of “big data” and the use of analytics to mine that data. Companies have been storing and analyzing large volumes of data since the advent of the data warehousing movement in the early 1990s. While terabytes used to be synonymous with big data warehouses, now it’s petabytes, and the rate of growth in data volumes continues to escalate as organizations seek to store and analyze greater levels of transaction details, as well as Web- and machine-generated data, to gain a better understanding of customer behavior and drivers.
Analytical platforms. To keep pace with the desire to store and analyze ever larger volumes of structured data, relational database vendors have delivered specialized analytical platforms that provide dramatically higher levels of price-performance compared with general-purpose relational database management systems (RDBMSs). These analytical platforms come in many shapes and sizes, from software-only databases and analytical appliances to analytical services that run in a third-party hosted environment. Almost three-quarters (72%) of our survey respondents said they have implemented an analytical platform that fits this description.
In addition, new technologies have emerged to address exploding volumes of complex structured data, including Web traffic, social media content and machine-generated data, such as sensor and Global Positioning System (GPS) data. New nonrelational database vendors combine text indexing and natural language processing techniques with traditional database technology to optimize ad hoc queries against semi-structured data. And many Internet and media companies use new open source frameworks such as Hadoop and MapReduce to store and process large volumes of structured and unstructured data in batch jobs that run on clusters of commodity servers.
Business users. In the midst of these platform innovations, business users await tools geared to their information requirements. Casual users—executives, managers, front-line workers—primarily use reports and dashboards that deliver answers to predefined questions. Power users—business analysts, analytical modelers and data scientists—perform ad hoc queries against a variety of sources. Most business intelligence (BI) environments have done a poor job meeting these diverse needs within a single, unified architecture. But this is changing.
Unified architecture. This report portrays a unified reporting and analysis environment that finally turns power users into first-class corporate citizens and makes unstructured data a legitimate target for ad hoc and batch queries. The new architecture leverages new analytical technology to stage, store and process large volumes of structured and unstructured data, turbo-charge sluggish data warehouses and offload complex analytical queries to dedicated data marts. Besides supporting standard reports and dashboards, it creates a series of analytical sandboxes that enable power users to mix personal and corporate data and run complex analytical queries that fuel the modern-day corporation.
Read the entire study.