In this post I will summarise applications of big data in business, science and society as well as possible future applications of big data.

I will also summarise the technological requirements of big data and how it is gathered, stored, processed and visualised.

Applications of big data in business

Big data helps businesses and organisations to use data to their advantage and turn it to identification of new opportunities.

This improves business decision making, making operations more efficient, turns higher profits and making customers happier with their product / service.

Big data and systems like cloud based analytics or Hadoop makes it possible to reduce costs fore organisations. This mostly applies to the costs of storing large amounts of data.

With the efficiency ans speed of systems like Hadoop better and faster decision making is now possible. This is due to ability to analyse new sources of data, in-memory analytics. Organisations are now able to analyse information much faster or even immediately which in turn allows to make business decisions based of what they have learned much faster.

Organisations with the ability to measure customer needs and their satisfaction with analytics gives organisations the ability to give the customers what they want.

More and more businesses are able to create new products that come from big data analytics.

In media and entertainment industry that revolves around user-generated data through various social media platforms, businesses were able to leveraged that data in order to predict the interests of their audience, getting insights into customers reviews and behavioural patterns.

Spotify, an on-demand music streaming service uses big data to to serve targeted music recommendations to individual users.

Similar use od big data is employed at Amazon Prime that offers e-books, video streaming as well as music streaming.

Applications of big data in science

There are wast benefits of application of big data in healthcare.

Healthcare analytics have great potential in areas like improving quality of life, prediction of potential outbreaks of epidemics but also significantly reducing costs of treatments ant time of diagnosis.

Big data application in healthcare means that doctors now can make more informed decisions as they have access to wider range of data.

There are many healthcare data types created in hospitals or clinics those include:

• Clinical data: doctors motes,  prescriptions, reports from medical imaging, laboratory data, pharmacy data
• Machine generated data: generated from monitoring vital signs, emergency care data, medical journals
• Patient data that includes electronic patient records

Another example for the use of big data in healthcare is predictive analytics that use data like pre-existing conditions, habit patterns, which enable to foresee how vulnerable an individual is to cancer and in turn allows for early treatment.

There are many more applications of big data in healthcare and those include:

• Tracking Patient Vitals

It is much easier to monitor patients in emergency rooms plugged to monitoring devices because any change in patterns can be quickly alerted to doctors in hospitals that often do not have enough staffing.

Other uses of big data usage in healthcare include:

• Improved hospital administration
• Fraud prevention and detection

Applications of big data in society

With big data already having an impact on business world and healthcare does it have any application in society?

The short answer is yes but lets fins some examples.

The main use of gig data that has impact on society are transformation of cities into smart cities. Using variety of sensors, tracking movement of public transport and that data that is generated enable cities to be more efficient and provide citizens with better and more personalised services while cutting down unnecessary costs and greatly reducing waste of resources.

Cities transportation systems that utilises big data analytics can be greatly beneficial.

Enabling data streaming gathered from variety of sensors, on-vehicle devices and smart traffic lights, processing in real time and communicating traffic information with drivers directly to their smart phones.

Recognition of traffic patterns by analysing real time data, reduction of roads congestion by traffic prediction and adjusting traffic controls accordingly.

Future applications of big data

• Smart Cities

There are many possible uses of big data in the future. Previously mentioned evolving smart-cities could turn into fully automates and even semi-autonomous cities providing public with with benefits of automated and personalised transportation systems.

There are many speculations on topics of ethics and security but from purely technological point of view the potential of transformations of cities into smart cities driven by big data analytics are huge.

Smart transportation is one of the areas that can transform how we commute.

Recognition of traffic patterns in real time, solving road congestion problems, utilising smart traffic lights systems and even driver-less cars that utilise real-time big data analytics.

• Smart Environment

Environment data gathered from large amount of sensors and processed can provide weather information that will lead to improving agriculture, better energy management and informing the public about hazardous conditions.

• Healthcare

Healthcare is another area where future use of big data analytics can have life changing effects. We all know the current technology we use on daily basis, our smartphones, smart watches and sport / fitness trackers. Ability to gather data in real time from all kinds of smart wearable devices can save your live one day.

I had a pleasure to attend Glasgow TEDx Talks where Dr Jack Kreindler talked about the future of healthcare, data and technology.

In the above picture Dr Jack Kreindler holds a bio-sensor that when attached to patients body is able to measure and transmit hart rate, ECG, body temperature, posture (standing, sitting, falling), stress, breathing rate.

To do this 10 years ago (for the drivers of formula 1) it costed about 10000 dollars.

Today a cost os such device it about $1 a day.

With devices like this and with many people using them, for a $1 a day (including all of the machine intelligence to compute the data) it is believed that the crisis that healthcare is facing with chronic diseases can be greatly reduced.

Among many different tests the results have shown that such device with big data analytics and cloud computing can alert patients of medical problems even before symptoms occur.

Technological requirements of Big Data

I have already mentioned some of the technological requirements that are very much linked to big data characteristics and especially the volume of it.

• Storage

As mentioned previously traditional storage solution are not well suited for big data due to its volume and large variety of types of data.

Number of organisation who already have data storing capabilities in-house. Those organisations that are exploring the option of using big data analytics may need to reevaluate and opt for storage types that are more efficient for big data analytics such as cloud computing or the use of flash storage that may be more suited due to its performance advantages.

The largest companies such as Facebook for best efficiency and performance use. Those may consist of clusters of severs with direst-attached storage and using tools like Hadoop while often using PCIe flash based storage for improved latency.

• Processing

Processing bid data and performing analytics operations require powerful computing resources.

While companies like Google can afford to build custom mage infrastructure of server clusters with powerful processing power smaller companies may opt for cost effective cloud computing (renting) that can be used on demand saving them large investments that would come from the purchase, set-up and maintenance of of their oven big data analytics server infrastructure. Smaller companies are also able tu use technologies like previously mentioned Hadoop.

• Gathering data

There are many sources of big data. Most companies collect data about their customers, business operations, cities and councils collect data from sensors, big stores collect in-store customer traffic data.

Depending on the type of organisation the type of data may vary but all of those who collect big data have one thing in common, storage of gathered data. This is closely tied to the above mentioned storage requirements and will depend on type and size of the organisations.

There are meny technological choices including own big data analytics sever infrastructure, cloud computing or distributed storage systems that store data that is gathered from company customers that are interacting with their websites, shops, services.

Data is also gathered from loyalty cards, social media websites and apps, satellites and so on.

• Big data Visualisation

Visualisation can be thought of as another characteristic of big data and currently it is a most challenging aspect that data scientist face.

Enormous amounts of data, event in a processed and analysed state is often not very useful insight for a human being. It may not be easily comprehended and auctioned on.

While traditional graphs did a good job for traditional data analysis in the past with big data there are simple too many data points (billions) to plot which usually fails due to in-memory limitations and poor scalability which is one of the important requirements in big data. When it comes to big date scalability is a must.

Solutions to those problems are constantly being developed. Some include the use of data clustering, tree maps, parallel coordinates or circular network diagrams.