Learn about Big Data concept and analysis




There is an enormous amount of data being collected from around the world. However, not all this information may be relevant to your decision-making process. To better use that large volume of raw data and turn it into something useful, you must first analyze that big data into a coherent summary or report. The process of summarizing a large dataset using statistical techniques is called Big Data analysis.

Although the term “Big Data” sounds impressive and powerful, in reality, you will find out that most algorithms used for everyday tasks such as sorting or simple numerical summaries are already well known since the last century. Those well-established procedures suddenly become “Big Data Analysis” because nowadays we have much more detailed information available than before according to RemoteDBA.com analysts.

To give you an example, let’s suppose that we want to count how many times a word is used by all the 45 million Gmail users. Traditionally this task would probably take ages, and your computer will need several days at least to perform it. Nowadays, however, simply creating a Hadoop cluster that consists of hundreds of machines and then storing all emails into HDFS storage could complete the same job in just one hour!

Analysis of big data

The data size is now big enough, so we can’t use those simple algorithms we have been using since our school days (such as sorting or finding the most frequent words). Of course, there are also other factors, such as the fact that most modern datasets are usually unstructured (e.g., tweets, Facebook posts, etc.), making them even more challenging to handle.

Nowadays, due to the advances in computer hardware (such as cluster architectures) and the amount of information available on the internet; Big Data analysis is no anymore something you can see only at NASA or particle-physics laboratories. Instead, it is now blending into our daily lives, where SaaS applications are constantly collecting data about their users’ behavior to adapt themselves for future requests better.

It is also worth noting that besides providing a better decision-making process, another benefit coming from Big Data analysis is the possibility to make automatic predictions which could be otherwise very hard if not almost impossible using traditional techniques. The ever expanding private information stored online has already made it possible for companies or other organizations to automatically determine your gender, age group, geographical location, etc., based only on the information you provide.

What are the drawbacks?

The main problem is that there are also some drawbacks coming along with extensive data analysis, mainly because humans will always be needed for collecting and preparing raw data before accurate analysis can be performed. Therefore, all those explicitly designed techniques to make automated decisions (like self-driving cars) still need a man behind the wheel who could eventually override any decision taken by algorithms.

Big Data Analysis cannot provide perfect results overnight. Still, if used correctly, it can certainly give us a better insight into our surroundings, thus making our life easier and more enjoyable.

The data size is now big enough, so we can’t use those simple algorithms we have been using since our school days (such as sorting or finding the most frequent words). Of course, there are also other factors, such as the fact that most modern datasets are usually unstructured (e.g., tweets, Facebook posts, etc.), making them even more challenging to handle.

Nowadays, due to the advances in computer hardware (such as cluster architectures) and the amount of information available on the internet; Big Data analysis is no anymore something you can see only at NASA or particle-physics laboratories. Instead, it is now blending into our daily lives, where SaaS applications are constantly collecting data about their users’ behavior to adapt themselves for future requests better.

Benefit of Big Data

It is also worth noting that besides providing a better decision making process, another benefit coming from Big Data analysis is the possibility to make automatic predictions which could be otherwise very hard if not almost impossible using traditional techniques. The ever expanding private information stored online has already made it possible for companies or other organizations to automatically determine your gender, age group, geographical location, etc., based only on the information you provide.

The main problem is that there are also some drawbacks coming along with extensive data analysis, mainly because humans will always be needed for collecting and preparing raw data before accurate analysis can be performed. Therefore all those techniques which were explicitly designed to make automated decisions (like self-driving cars) still need a man behind the wheel who could eventually override any decision taken by algorithms.

Big Data Analysis cannot provide perfect results overnight, but if used correctly, it can certainly give us a better insight into our surroundings, thus making our life easier and more interesting. The data size is now big enough, so we can’t use those simple algorithms we have been using since our school days (such as sorting or finding the most frequent words). Of course, there are also other factors, such as the fact that most modern datasets are usually unstructured (e.g., tweets, Facebook posts, etc.), making them even more challenging to handle.

Nowadays, due to the advances in computer hardware (such as cluster architectures) and the amount of information available on the internet; Big Data analysis is no longer something you can see only at NASA or particle-physics laboratories. Instead, it is now blending into our daily lives, where SaaS applications are constantly collecting data about their users’ behavior to adapt themselves for future requests better.

It is also worth noting that besides providing a better decision making process, another benefit coming from Big Data analysis is the possibility to make automatic predictions which could be otherwise very hard if not almost impossible using traditional techniques. The ever expanding private information stored online has already made it possible for companies or other organizations to automatically determine your gender, age group, geographical location, etc., based only on the information you provide.

The main problem is that there are also some drawbacks coming along with extensive data analysis, mainly because humans will always be needed for collecting and preparing raw data before accurate estimation can be performed. Therefore all those techniques which were explicitly designed to make automated decisions (like self-driving cars) still need a man behind the wheel who could eventually override any decision taken by algorithms.

(Visited 23 times, 1 visits today)

Advert:




Be the first to comment

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.