The evolution of Big Data

The data format Flat File

Data is stored in the text file usually contains no structure (flat file). Not capable of effective data retrieval and data integration can not make it without any pattern or surrounding structures. The data stored in the flat file will encounter problems of exploitation in the times today. It’s like a nightmare when there is any processing of data in the application. Although the application was developed at that time is consistent with the development of technical and data.

Edgar Frank Codd was a British computer scientist who, while working for IBM, invented the relational model for database management, the theoretical basis for relational databases. He has presented 12 rules for the relational database and suddenly the world’s data types become available, technical rules of law. The relational database is a land full of promise for all database users have no structure. Relational databases bring to the relationship between the data as well as improve the performance of data retrieval.

Relational database

Since Edgar F Codd’s 12 rules proposed for RDBMS, there are many different providers to start building applications and tools to support the relationship between the database. Entity relationship modeling (ER) was also developed to describe a database in an abstract way.

Huge data growth

Everything seems fine with RDBMS database in the world. Applications and tools for RDBMS developers to make your life a lot easier. Due to the popularity and ease of installation of RDBMS, almost all data are stored in the RDBMS. New era applications built social media grows fast. Each organization has the pressure to deliver a better experience to users based on the data we have.


Data Warehousing

Massive data growth is currently creating great challenges for organizations who want to build intelligent systems based on data and provide the user experience in real time to the customer. Various organizations immediately began construction of data warehouse solutions, where data is stored and processed. Trends business intelligence (business intelligence) has become the daily needs. The data received from the overnight trading system will be used to build intelligence reports. Although this is a great solution, it offers many challenges. Model of relation database and the data warehouse task all are built based on the database model of traditional relations, which are already in the mind and still more challenging when there was the appearance of the data without structure.

Fun challenge

Each organization has the expertise to manage structured data, but the world has changed more unstructured data. Smart information in the video, photo, SMS, text messaging, social networking, … All these things need to be brought to a single platform and build a unified system for what the business needs. The way we do business has also changed. There is a time when users use only those features that tech support, however, users now require these features and the technology must be built to support. Smart application needs in real time from the data flow speed is fast becoming necessary.

A large number (volume) data at high speed (Velocity) different (Variety) is the properties of the data. The system of traditional databases has limited when addressing the new challenges. This is big science needs data. We need to innovate the way of processing and data management. We need creative ways to collect data and presented to the user.

Share this post