I think that during the past three years you have heard about big data a lot, you turn on the TV and you find the announcer talking about big data, you hold your phone and you browse Facebook and you read a lot of news about big data, you want to relax and you search for something fun on YouTube to find that there are dozens of clips about big data !
I also found a lot of searching for:
- big data architect, distributed data processing engineer, and tech lead.
- big data engineer vs data engineer salary.
- lead data engineer roles and responsibilities.
- data warehouse engineer roles and responsibilities.
- big data definition.
Well the time has come when you will come face to face with big data, but this time on Seo7u.
Big Data is everywhere
If you ever try to understand what Big Data is, you'll probably think it's a science fiction trail, or at least a far-fetched dream your great-grandchildren might see.
But dear reader, this is not true, as you are now living in the midst of the fourth industrial revolution that will change everything, which is the information revolution.
Data is today the new oil, contributes daily to changing our lives, whether for the worse or for the better, but this data is precisely who?
It's your data, mine, and everyone's data you know or don't know, your medical history, your Facebook likes, the time you're on YouTube, your bank account purchase history.
Big Data is all of this and more. Imagine that everything you do is recorded somewhere, every situation you've been through, every medical report and every medication you took, your college grades, even your geographical history and the restaurants you frequent.
And even your diet and exercise data with which you are trying to remove the fat you gained from those restaurants, all this and more, dear reader, is the so-called big data.
But if you want a more academic definition of Big Data, it can be defined as: Large sets of data that include a lot of information, images, numbers, and records that are complex, and difficult to analyze using a specific method or tool.
It's classed as massive, it's impossible to have on our regular devices, but rather is stored in the cloud (better by definition isn't it?).
Read also: What is Data Analysis in a simple explanation?
The term itself Big Data dates back to the nineties of the twentieth century, when computer scientist John Macy launched it on complex big data that ordinary tools cannot contain or deal with, and imagine that this was in the nineties, so how are the conditions today?
Read also: What is IT, what are its specialties and how to get started with it (comprehensive guide)
How big is big data?
I know you have this question on your mind and you want to ask it, but you're just afraid it's a stupid question. Do not worry, dear reader, this question is not stupid at all, on the contrary, it is a very intelligent question, but the answer to it will definitely surprise you, because the numbers we are talking about exceed any human ability to imagine it, not even deal with it.
Imagine with me that in the year 2020, the data reached about 44 zettabytes, and if you do not imagine what a zetabyte is, then each of them is equal to 1024 exabytes, and the exabytes themselves are equal to 1024 petabytes, and a petabyte - I know you are tired, but complete with me - is equal to 1024 terabytes - and this is a term You know it well - a petabyte, you know, is 1024 gigabytes.
That is, if you have 1 terabyte of storage in your computer, the amount of data in the whole world in the year 2020 is about 47 billion storage units of yours, you might say that this number is very large, but what is the problem with it? Well there are actually a lot of problems with it.
As you remember from my other article on Seo7u What is Data Science, the data in 2019 was only 29 zettabytes (30,000 exabytes), meaning that in one year the data doubled by 150%, which is really a staggering amount of data.
And if this is not enough for you, then this number in 2015 was about 7.7 zettabytes, that is, in 5 years the data increased by 571%, and what is even more terrifying is that in 2005 the data was about 130 exabytes (0.12 zettabytes), that is, in 15 Only a year, the data increased by about 35.00%. Yes, the number is correct, thirty-five thousand percent.
According to the report of the International Data Corporation IDC; The amount of data in the world in the year 2025, that is, after only four years, will reach 175 zettabytes, and thus the data for humans will continue to multiply and multiply at an even faster rate.
Read also: Privacy in the Internet world: Can Google, Facebook and others be trusted?
Where does big data come from?
In all possible brevity; This data comes from us humans, but how? Everything you do on the Internet stores and feeds this massive wave of big data, literally everything from your use of social media apps, to your bank records, medical records, movements, everything you can imagine.
And do not underestimate this amount of big data, it is very huge, for example in one day:
- 320 billion emails are sent.
- 500 million tweets are posted on Twitter.
- 350 million photos are uploaded to Facebook.
- Videos are watched on Facebook for the equivalent of 100 million hours.
- 65 billion messages are sent on WhatsApp.
- 95 billion videos and photos are shared on Instagram.
- There will be 5 billion searches, of which 3.5 billion are on the Google search engine.
And I won't discuss with you how many Seo7u visitors benefit from our articles daily.
These huge numbers related to big data, which can not be imagined, are increasing every day and every second with the increase in the number of people who use the Internet, in addition to the connection of other things in our lives to the Internet, for example the Internet of Things or IOT will contribute to the doubling of these numbers.
In the future, a large part of the big data will come from health systems associated with it, the self-driving cars and automation and robots that will occupy every aspect of our lives, which will depend absolutely on Big Data.
Because of the fluidity of the term Big Data, researchers have agreed on five basic characteristics or properties of Big data, which they called the Five V principles, which are:
Volume:
It is related to the large volume of Big Data that we produce at every moment.
Velocity:
This is the speed at which this data is generated and shared. Whether it is related to social media or the results of scientific research.
Variety:
The forms in which the beige data is not related to its textual or visual, but rather to the nature of the data itself in terms of being Structured or Unstructured.
Veracity:
This is the accuracy of this data, the degree to which we can trust this data, and this characteristic is most evident in medical data and records.
Value:
This is the point that we will discuss now, which is related to the impact of this Big Data on our lives and all its aspects.
Also Read: Top Data Science Applications (Endless Incredible Uses)
Big data is changing our lives
Perhaps now, dear reader, you would like to know how big data affects your life and reality, but the word “affect” is not accurate in this context, but rather the most accurate change. Big data constitutes a new, different world in terms of; Medicine, banking, jobs, transportation, transportation, education, and everything else.
Perhaps power was the most important in some eras, and perhaps in other eras wealth was the most important, but in our time information is the most important, it is the raw material for power and wealth, so dear reader while reading this article you are getting richer and stronger.
Our daily life and big data
I will start, dear reader, with examples that you touch directly on a daily basis, for example, the ads you see on Facebook daily are not random, but are directed at you…
Not just because of your age or geography, but the Pages you follow, your purchasing habits, the hours you're active on Facebook, and other subtleties of you, your friends, and the celebrities you follow.
I think you say I'm the biggest topic, but this information Facebook has about you and me contributes to changing the world - for better or worse -....
In the previous US elections between Trump and Hillary Clinton; This data contributed greatly to Trump's victory, who paid millions of data analytics companies to show him reports and advice, and he was what he wanted, and this data turned the scales in his favor.
Netflix also uses data about your preferences, the times you watch them, and the length of your viewing sessions in order to get you addicted to your series and movies.
For example, Netflix has many business posters that it shows to viewers according to their data. It chooses the poster that will attract each follower according to his preferences and watch history.
As for Amazon, YouTube and Spotify; They all use your different data to understand you and your tastes to make recommendations and recommendations for you: products in the case of Amazon, videos in the case of YouTube, songs in the case of Spotify.
These are just the tip of the iceberg. The examples are almost innumerable, and thinking about them might make us—or actually make us—skeptical whether our choices really stem from our free will or from being controlled by various algorithms.
This point in particular has caused great controversy in the past years, and many legal disputes between human rights organizations and major technology companies.
Read also: What are the fields of computer science?
Medicine and Big Data
Big data has greatly contributed to the development of the medical system so far, and this is because it made it easier for us to make decisions in difficult matters such as diagnosis and treatment protocols.
It also facilitated the recognition of dangerous situations; It also happens a lot today with the wearers of smart watches that measure heart rate and blood pressure.
One of the most prominent contributions of big data to medicine is its improvement in the diagnosis process, especially for cancer:
As the presence of giant databases containing x-rays of cancer patients or suspected of having cancer has contributed to providing the necessary data to train artificial intelligence in diagnosing cancer, and now artificial intelligence is much better in diagnosing cancer than the human expert and his skill is increasing every moment.
Read also: Artificial Intelligence: The doors to the future are open
Also, big data helped discover treatments and treatment protocols in more than one way. On the one hand, it may happen that the artificial intelligence based on it will show us that the medicine we use in treating disease (X) is effective in treating disease (Y), by analyzing the data of patients who They have disease (X) and disease (Y) and are taking this medication.
This is in addition to the fact that Big Data and artificial intelligence have improved the ways in which we can discover new drug compounds, and this has been evident in the Corona pandemic, as well as they are also helping us in the field of personalized medicine that relies on genetic data in order to determine the best treatment that suits patients.
And that is not all. There are many other benefits that we would not have achieved without big data, including but not limited to: Control of risks and diseases, immediate alerting in emergencies, optimization of drug supply chains, providing more accurate predictive analysis of patients' condition, reducing medical and surgical errors.
Security and Big Data
Perhaps this is also one of the important aspects in which big data has helped us a lot, but every day that goes by, with every progress big data makes we are relatively safer, and I don't just mean the accident aspect - where big data reveals the potential for accidents or big problems and tells us ahead of them - And I don't mean that it helps us organize the roads, no, not only that.
Big data has contributed to the emergence of an amazing invention in our world, which is the self-driving cars, those cars that do not have a bad mood or suicidal tendencies, and do not even send e-mails while driving, which contributed to the decrease in accident rates in the countries in which they are used, which by It is expected to eliminate road accidents permanently in the future.
And not only that, big data also helps to solve crimes even that happened many years ago, and you may hear about solving a crime that happened decades ago by comparing the DNA that the police had with the national DNA base, and vice versa as big data has contributed to the exoneration of a person after He spent 20 years in prison for a crime he did not commit.
It is expected that in the coming years, big data will be able to predict the occurrence of crime before it occurs if it has the amount of data required to do so, and it can also help predict the commission of crimes by ex-prisoners after their release, among dozens of other things that are science fiction today.
Also Read: Learning Artificial Intelligence (All the Resources and Information You Need)
Be careful; Big data can steal your job
I have shown you the positive aspects of the data revolution, but nothing is 100% good, the fourth industrial revolution like any other revolution causes many harmful changes to humans, for example in the third industrial revolution hundreds of thousands of people lost their jobs, and this unfortunately will happen Also in the Fourth Industrial Revolution.
A lot of jobs will be done by machines and software code, all the jobs that need organized steps and methods - which are almost most jobs - will be done by artificial intelligence, and there are jobs that will be completely destroyed, regardless that 50% of today's jobs will be occupied by machines and artificial intelligence based on Big data in just 10 years.
This is precisely the matter that concerns governments at the moment, because despite the fact that big data and artificial intelligence will create jobs, but this number is small in front of the jobs that you will eliminate, and if you are afraid, dear reader, there is a site called WILL ROBOTS TAKE MY JOB that tells you statistics about whether you will be replaced.
There are many jobs that have been affected in our time by the signs of the Fourth Industrial Revolution, such as vehicle drivers, language proofreaders, accountants and data entry people, so make sure, dear reader, to follow our articles on the Winners website, because it will be your refuge from this disaster that will eliminate many jobs in the years next few.
Read also: The most important areas in demand in the online job market
The dangers that big data brings us
And if the risk of losing your job is not enough, there are many other problems that we have to worry about, for example in terms of privacy; Big data could potentially wipe out every privacy possible, and that's because everything we do from the food we order, to our prescriptions and our bank accounts is registered to it, which will leave us very restricted.
There is also the danger of the impact of big data on the freedom of will, this momentum of data will make service providers and large companies control you with what they know about you, and direct you either to buy their products or addiction by sitting for long periods in front of them, or worse, affect your important decisions such as your vote Or your impression about yourself and your self-confidence.
In the short term, there is the problem of Bias, either in terms of ethnic and religious minorities or gender bias towards females. If algorithms are given biased data from big data, their decisions, judgment, and nominations will be biased, which will negatively affect society.
This can cause serious damage, especially if it is in the areas of crime and crime prediction, as crime prediction algorithms are biased against black or poor citizens.
For the reasons mentioned above; There are many countries that pass laws to protect their citizens and their data, led by the European Union and Britain, which issued the GDPR, or what is known as the European General Data Protection Regulation that deters companies from misusing big data and regulates this process.
And in the end I had to stress that the future is something mysterious, that everything we know shifts and sometimes turns to the opposite in moments, and there may be a lot of talk about big data, artificial intelligence and robots if they are all good or all bad.
But like everything else it has its good sides and its bad sides, and we have to work hard to nurture the good sides and reduce the bad sides.
