Big Data

What other issues emerge with the Volume of data?

1) Determining the relevance within large data volumes 2) How to use analytics to create value from relevant data

Implications of Big Data

1) How do we store this much data 2) How do we manipulate this much data? 3) How do we get this much data to information? 4) How do we use tools to turn this information into knowledge? Into Widsom? 5) How do we make decisions from this much data

What devices contribute to the velocity of data?

1) RFID tags (radio frequency ID tags) on products 2) Product sensors 3) Smart devices

Three Stages of the Evolution of Business Analytics

1) Statistical Analytics 2) Relational Analytics 3) Big Data Analytics

What are the main types of formats of data

1) Structured, numeric data in traditional databases 2) Information created from line-of-business applications 3) Unstructured text documents, emails, video, audio, social media, etc.

What are the main ways that Big Data is being used?

1) To better understand our target customers 2) To understand and optimize the business process 3) To improve health 4) To improve security/law enforcement 5) To improve and optimize cities and countries 6) To improve sports performance

What factors contribute to the increase in the volume of data?

1) Transaction-based data 2) Unstructured data 3) Sensor and machine to machine data

What are the three elements of Big Data?

1) Volume 2) Velocity 3) Variety

How does Big Data improve and optimize cities and countries?

Examples: - Optimize traffic flow (real-time traffic info, social media info, weather info) - Create smart cities (infrastructure and utility joined)

How does Big Data improve sports performance?

Examples: - Video analytics of player performance - Track what happens before, during, and after a successful play - Sensor technology built into sports equipment - Smart tech to track sleep, nutrition - Track social media conversations

What are the ingredient of Big Data?

Social Data + Enterprise Data = Big Data

Excessive data volume is no longer a ______ issue.

storage - now have the capacity/technology to store this data (Cloud)

What is Big Data?

= "a situation where the volume, velocity, and variety of data exceed an organization's standard storage or computing capacity for accurate and timely decisions"

What is relational analytics?

= Analytics that allow us to do some relational analysis = "rubix cube" approach: relationships between the metrics and the question being asked - Takes metrics and manipulates them until we get an answer

What is Big Data analytics?

= Analyzing not just the data brought in through sales/operations, but data that just dwarfs anything we normally track - Too big for an organization's standard relational analytics to handle

How does Big Data help to improve health?

= Understand and predict disease patterns - Examples: - Smartwatches/Fitbits: understand links between lifestyle and disease - monitor and predict flu epidemics/disease outbreaks/illness patterns

How does Big Data help to understand and optimize the business process?

= Use predictive models to: Ex: retailers - optimize product stock - optimize supply chain or delivery route using data from geographic positioning and RFIDs

How does Big Data improve security and Law Enforcement?

= Various security servies - Examples: - foil terrorist plots - detect cyber attacks - predict criminal activity - detect fraudulent transactions - Police where crime is predicted to occur

What is the objective behind better understanding target customers?

= create predictive models - Telecom companies: predict customer churn - Retailers: preduct when products will sell - Insurance companies: predict how customers will behave (drive cars, health habits, etc.)

What is Sensor/Machine to Machine data?

= data collected by traffic cameras, products (radio frequency ID tags), fitbits, etc. - In the automated manufacturing process, when machine parts talk to other parts

What is the velocity element of Big Data?

= data is streaming in at unprecendented speeds and must be dealt with in a timely manner - velocity at which data is coming into the organization - data is coming in faster and faster and faster

Who deals with Big Data analytics?

= data scientists

What is Unstructred Data?

= data that is not limited/defined by any sort of parameter - No clear definition to this data - Often streams in from social media platforms - Ex: texts, videos, audio, tweets, etc.

What is the variety element of Big Data?

= data today comes in all types of formats

What is the problem with transaction-based data?

= eventually run out of space - As a business becomes more sophisticated, able to capture more and more data and need a place to store all of this information

How does Big Data help to better understand target customers?

= get a more complete picture of customers - allow companies to expand their tranditional data sets to those with social media, browser, text analytics, or sensor data

What is the biggest challenge when dealing with the variety of data?

= managing, merging and governing different varieties of data

What is transaction-based data?

= operational data stored throughout the year - stored over time in data warehouses/organizational processes

How "big" is Big Data?

= produce 5 exabytes of data every 2 days (produced this amount of info from the dawn of civilization until 2003)

What is the biggest challenge when dealing with the velocity of data?

= reacting quickly enough

What is Enterprise Data?

= structured data ** internal to firm - data within the enterprise that helps run the business - Ex: client, product, trade, or transaction data

So, what is Big Data?

= tying these two together - "tying together your enterprise data models with the value that lies in the unstructured data around your firm" * All businesses need to combine both internal and external data in order to grasp big data

What is Social Data?

= unstructured data **external to firm - content including text, audio, video, images, social media, and other public channels

What is statistical analytics?

= using basic tools (excel, etc) - doesn't require specific coding knowledge - CANNOT come close to handling the amount of data we have today

See reading #8

- What is Data Exhaust? - Policy Issues with Big Data

Through Big Data Analytics, retailers could...

- increase profit margins by more than 60% through full optimization of big data analytics

Big Data Facts

- pace of data production is rapidly accelerating - significant exponential growth of data in a very short period of time - Big Data industry is exploding = need to have skills in this area

Through Big Data Analytics, the healthcare industry could...

- save $300 billion a year = equivalent of reducing the healthcare costs of every man, woman, and child by $1,000/year

What is Hadoop?

** Removes constraints on the storage and processing of Big Data = open source software project (accessible/free to everyone) that enables distributed processing of large data sets across clusters of servers - Breaks down data into manageable chunks via Map and Reduce process