Big Data – Techniques

Types of problem suited to big data analysis

There are many types of problems where big data analytics nay be suitable. This includes problems that often require large amount of data for it to work.

• Netflix

A good example of that could be be a video streaming service like Netflix that uses user data to keep users engaged. With data analytics Netflix can recognise users watching trends as well as regional “viewing testes” which in true enables them to display highly probable recommendations to their users.

• Online shopping – Amazon

In similar way any type of online shopping business problem can benefit from big data analytics. By leveraging customer data and by finding trends and learning about behaviours such companies can increase their sales by recommending the most likely needed products to their customers. An example of such company would be well known Amazon.

• Healthcare – Hospitals

Another type of problem that big data analytics is well suited for is already briefly mentioned healthcare. Big data analytics can be applied to reduce diagnosis time, recognise patient appointment keeping patterns, greatly reduce operational costs as well as improve appointment waiting times.

Big data is also helping to solve problems that shift managers are facing in hospitals. This helps to balance number of staff present at any given time and reduce unnecessary costs. Hospitals in Paris are trialling big data and machine learning technologies in order to forecast admission rates which leads to efficient deployment of resources.

The number of problems that big data analytics is suited for can be infinite but we have to remember that not every problem can be solved by big data analytics. We have to remember about previously mentioned requirements for big data, the 5Vs as well as technological requirements. An example would be a problem scenario where we may not have enough data to be analysed (Volume). In that case no matter how powerful our software or hardware is it may prove difficult to come up with a viable machine learning model.

Data mining methods

Data mining is a process or practice that aims to extract information from pre-existing datasets.

There are many types of data mining methods. Those include:

• Statistical Techniques
• Clustering
  
• Visualization
  
• Decision Tree
  
• Association Rules
  
• Neural Networks
  
• Classification

Statistical techniques help to discover patterns and build predictive models. Statistics are also useful in summarizing data.

Clustering analysis is a way of identification of similarities in data. Clustering is often called a segmentation. An example of this could be an insurance company who can group and sort their customers based on their, age, income, policy type….

There are many types of clustering including:

• Partitioning
• Hierarchical Agglomerative methods
• Density-Based Methods
• Grid-Based Methods
• Model-Based Methods

A Visualisation is also a type of data mining method.

Visualisation helps to recognise patterns in a more meaningful / visual way.

Induction Decision Tree

Decision Tree is

Types of visualisation

Application of big data techniques to a problem

…