There’s no doubt that the amount of data is growing, hence the interest of many in the Big Data topic. ICT firms are firmly positioning themselves in the Big Data ecosystem, striving for a piece of the action. Some wonder whether this Big Data topic is truly valid, and not just a Big Hype. In that light we want to share some of the results of the Big Data study we did last year. This study indicates that 2/3 of Dutch organisations see the need to implement Big Data before 2017. In fact, over 45% of the respondents deem Big Data to be necessary before the end of 2015.
To validate the hype one needs to know how relevant data trends are. Below you will find a snapshot from our study providing the relevance. Responses are divided to those organisations that have actually implemented a Big Data project and those that have a sizeable data initiative running and are eligible for a data project (the Big Data potentials). The reason to distinguish between the two is that there are substantial differences in the way they look at data and the perception of what a Big Data project beholds and can achieve.
(Q.: How relevant were the following data trends when you initiated your Big Data project / plan to implement a Big Data project?).
Organisations that have implemented a Big Data project value the following data trends:
- Being able to process unstructured data
- Being able to integrate external data
- Being able to process real-time or near-time data
Organisations planning to implement a Big Data project value the following data trends:
- Being able to process the data faster
- Being able to analyze the data faster
- Being able to analyze and process larger datasets
Volume
Within Big Data organisations, 53% currently work with data volumes up to 10 terabyte. Within three years the number of organisations working with this relatively low amount of data will diminish to 26%.
On the other hand about 20% currently works with data volumes larger than 100 terabyte. This will grow to 44% within the next three years. Of these, 19% expects to work with volumes larger than a petabyte within three years.
This steep growth curve is also visible for the organisations with Big Data potential. Of these 11% currently work with datasets larger than 100 terabyte, where 39% expects to work with these volumes within three years.
It is mainly the large enterprises (more than 500 employees) that will fuel this growth: 56% indicate they plan to work with datasets larger than 100 terabyte within 3 years.
Variety
Though variety is one of the core elements of Big Data, most of the respondents indicate that they primarily attach value to text and transactional data. For the near future they do not expect that this will change drastically. Social data and web data are increasing in importance and are potentially the largest growth areas, however they remain less important compared to text and transactional data. Video, voice and sensor data are not (yet) a priority for most of the respondents.
Velocity
Velocity has become an issue since SAP introduced Hana, Oracle introduced Exalytics and IBM introduced PureData Systems. In this perspective distributed data collection and processing systems like Hadoop and no-SQL might find their way into the end-user organisation. However our study findings indicate that the majority still uses traditional relational database systems (over 60%). Still, 45% of the organisations with a Big Data project is contemplating or testing Hadoop, while at the same time 55% of the organisations with Big Data potential indicate that they are not looking at distributed data processing systems. Apparently velocity becomes more important once one is further down the Big Data road.
Impact on IT systems
In summary, the results indicate that the first priority of organisations is to focus on the amount of data, data growth and on keeping grip on internal text and transactional data. This does not mean that there isn’t a case for Big Data. On the contrary: initial investments associated to Big Data are becoming visible as organisations indicate that because of the expected data growth they will need to improve performance and increase the capacity of their IT systems (40%). It is the storage and data processing vendors like NetApp, IBM and SAP that are most likely to benefit from these investments.
The study also reveals that there are more V’s to Big Data than the three mentioned before, most importantly the Value of data. There is no question that the value of data is increasing and that business benefits are expected – and can be achieved – by implementing a Big Data strategy. This perception applies to midmarket as well as large enterprise, IT user as well as business user. The sheer volume and growth of data make it very clear that end-user organisations are ready to get serious with Big Data.
