ISDS 2001 FINAL - Joni C.
Hadoop
*Hadoop is an open source framework for storing and analyzing massive amounts of distributed, unstructured data *Originally created by Doug Cutting at Yahoo!
RESULTS of Dow Chemical
As Dow transitions from a traditional manufacturer to a solutions provider, JMP has become an essential tool for analyzing and presenting data, sharing it in a collaborative process with colleagues and customers, and using it to project new initiatives
Variety
Data today comes in all types of formats-ranging from traditional databases to hierarchical data stores created by the end users and OLAP systems, to text documents, e-mail, XML, meter-collected, sensor-captured data, to video, audio, and stock ticker data. By some estimates, 80 to 85 percent of all organizations' data is in some sort of unstructured or semi-structured format.
Data scientist
LSU Masters of Science in Analytics.
Where does most business data come from?
Machines
A LIFE COACH IN YOUR POCKET STUDY
This case study was NOT about consumer entertainment when using apps.
SOLUTION with Dow chemical
Thousands of Dow employees rely on JMP statistical discovery software to gain a competitive edge. JMP is used in many facets of Dow's operations
EBAY Why did Ebay need a Big solution?
eBay is the world's largest online marketplace, and its success requires the ability to turn the enormous volumes of data it generates into useful insights for customers. Big Data is essential for this effort.
CabSense
for consumers to use in finding a taxi in New York city -rating of street corners using interactive street maps -Cabsense does not give the exact location of all the cabs in real time -Cabsense does not give cab drivers the exact location of all New Yorkers
location information
from mobile phones can be used to create profiles of user behavior and movement
Business process engineering
which is defined as a major restructuring of organizational business processes with respect to changes in organizational culture and new information technology initiatives being undertaken by an organization.
*What is the role of analytics and Big data in modern day politics?
* Big Data and analytics have a lot to offer to modern-day politics. This case study makes it clear that even though elections are unpredictable, politics and elections are suitable arenas for Big Data. The main characteristics of Big Data, namely volume, variety, and velocity (the three Vs), readily apply to the kind of data that is used for political campaigns. Big Data Analytics can help predict election outcomes as well as targeting potential voters and donors, and have become a critical part of political campaigns.
ANALYTICS AS A SERVICE (Aaas)
*"agile analytics" *Aaas in the cloud has economies of scale, better scalability, and higher cost savings *Data/text mining +Big data >Cloud computing
INFORMATION AS A SERVICE (IaaS)
*"information on demand" *Goal is AGILITY-make info available quickly to people, processes, and applications *Tries to REDUCE not increase redundant data and the time it takes to build and deploy new information
Bid data FACTS
***Big Data by itself, regardless of the size, type, or speed, is worthless unless business users do something with it that delivers value to the organization. ***Big Data plus "big" analytics yields value. ***The traditional means for capturing, storing, and analyzing data are not capable of dealing with Big Data effectively and efficiently and so a new breed of technologies are needed to take on the Big Data (developed or purchased or hired or outsourced). ***Traditional data warehouses have not been able to keep up with the variety and complexity of data
How can smartphone data be used to predict medical conditions?
*. Predictive analytics can calculate life expectancy of the user, who can begin discovering health opportunities *. Also, most smartphones are equipped with accelerometers and GYROSCOPES to measure jerk, orientation, and sense motion *The app did NOT create the behavior profile and compare it with census data from the Bureau of Labor.
Critical success factors for big data analytics
*A clear business need (alignment with the vision and the strategy) *Strong, committed sponsorship (executive champion) *A fact based decision making culture
Data scientist FACTS
*Data scientists use a combination of their business, communication, and technical skills to investigate Big Data looking for ways to improve current business analytics practices (from descriptive to predictive and prescriptive) and hence to improve decisions for new business opportunities. *A data scientist is considered a Big Data guru. *Data scientist positions are in high demand and offered with very high salaries and very high expectations.
Do you think Big Data analytics could change the outcome of an election?
*It may well have changed the outcome of the 2008 and 2012 elections. Many agree that the Democrats clearly had the competitive advantage in utilizing Big Data and Analytics over the Republicans in the 2008 and 2012 presidential elections
NoSQL
*NoSQL (not only SQL) *A new style of database *to store and process large volumes of unstructured, semi- structured and multi structured data *Can handle Big Data better than traditional relational database technology
POSITIVE impact on managers' activities and their performance
*Research into managerial use of DSS and expert systems found managers 1. spent more of their time planning saw their decision making quality enhanced 2. were able to devote less of their time "fighting fires" (a crisis that pops up) 3. spent less time in the office and more in the field 4. gained more power as they gained more information and analysis capabilities
Big data and STREAM ANALYTICS
*Stream analytics is also called Data-in-motion analytics and real time data analytics *One of the V's in Big data=velocity *Analytic process of extracting actionable information from continuously flowing/streaming data
Use of location-based analytics GLOBAL INTELLIGENCE
*US transportation command (USTRANSCOM) *Yes, the US military does benefit from location based analytics
DATA AS A SERVICE (DaaS)
*one of the major service models that underlie the service oriented DSS/BI *Accessing data "where it lives" *Enriching data quality with CENTRALIZATION
Cloud computing example
*web based email>>Cloud computing application *CENTRALIZED hardware/software/infrastructure *CENTRALIZED updates/upgrades ** ex) Google's Gmail
Velocity
- refers to both how fast data is being produced and how fast the data must be processed (i.e., captured, stored, and analyzed) to meet the need or demand. RFID tags, automated sensors, GPS devices, and smart meters are driving an increasing need to deal with torrents of data in near—real time.
Big data
-Using data to understand customers/ clients and business operations to sustain and foster growth and profitability is a) an increasingly challenging task for today's enterprise b) is not a new technological fad, rather, its a business priority
Reality mining
-is a specialized data mining of location based data -real time location information=real time insight -useful in marketing promotional campaigns -footpath
What were the CHALLENGES for investment bank?
. As data volumes and variability increased, the legacy system was not fast enough to respond to growing business needs and requirements. It was unable to deliver real-time alerts to manage market and counterparty credit positions in the desired timeframe
GREAT CLIPS CASE How is geospatial analytics employed at Great Clips?
. They use geospatial analysis to help analyze the locations based on the requirements for a potential customer base, demographic trends, and sales impact on existing franchises in the target location. *Great Clips depends on a growth strategy that is driven by rapidly opening new stores in the right locations and markets. Great Clips is NOT using dynamic segmentation.
KNOW INPUTS to the ANALYTICAL system
1. Market research 2. social media 3.census data 4. election databases
Know the analytic system OUTPUTS or goals
1. voter mobilization 2. organize movements 3. increase number of volunteers 4. raise money contributions
How to SUCCEED with Bid Data
1.Simplify 2.Coexist 3.Visualize 4.Empower 5.Integrate 6.Evangelize 7. Govern
Business investments ought to be made for the good of the business, not for the sake of mere technology advancements. Therefore the main driver for Big Data analytics should be an alignment with the vision and the strategy and at any level-strategic, tactical, and operations. Which of the critical success factors for Big Data analytics is being described?
A clear business need
Geospatial analytics
A critical emerging trend in analytics is the incorporation of location data
In ____?______, the numbers rather than intuition, gut feeling, or supposition drive decision making. There is also a culture of experimentation to see what works and doesn't. To create ____?____, senior management needs to do the following: recognize that some people can't or won't adjust; be a vocal supporter; stress that outdated methods must be discontinued; ask to see what analytics went into decisions; link incentives and compensation to desired behaviors
A fact based decision making culture
It is a well-known fact that if you don't have committed executive backing, it is difficult (if not impossible) to succeed. If the scope is a single or a few analytical applications, the support can be at the departmental level. However, if the target is enterprise-wide organizational transformation, which is often the case for Big Data initiatives, _____________________ needs to be at the highest levels and organization-wide. Which one best Critical Success Factor for Big Data Analytics best fills the blank in the previous sentence?
A strong, committed sponsorship
INVESTMENT BANK ACHEIVES TRUTH: How can Big data benefit large scale trading banks?
Big Data can potentially handle the high volume, high variability, continuously streaming data that trading banks need to deal with. Traditional relational databases are often unable to keep up with the data demand.
What were the SOLUTIONS for INVEST MENT BANK?
Big data offered the scalability to address this problem. The benefits included a new alert feature, less downtime for maintenance, much faster capacity to process complex changes, and reduced operations costs. *it provided real time access to trading data
impacts of analytics in organizations
Business process reengineering
Dublin : How can big data analytics help ease the traffic problem in large cities?
By integrating geospatial data from buses and data on bus timetables into a central geographic information system, you can create a digital map of the city. Then, using the dashboard screen, operators can drill down to see the number of buses that are on time or delayed on each route. With big data analytics, users can produce detailed reports on areas of the network where buses are frequently delayed, and take prompt action to ease congestion
CABSENSE 1. What are the various options that CabSense provides to users?
Cabsense does NOT give the exact location of cabs in real time for New Yorkers and visitors wanting to hire a cab.
Oklahoma: Why perform consumer analytics?
Consumer analytics helps a company's customers make better purchasing and usage decisions. OG&E is using it to help conserve on energy usage and ultimately to delay their need to build new fossil fuel generation plants
Oklahoma: What is meant by dynamic segmentation?
DYNAMIC SEGMENTATION- refers to real-time or near-real-time customer segmentation analytics that will enhance their understanding about individuals' responses to the price signals and identify the best customers to be targeted with specific marketing campaigns.
Variability
Data flows can be highly inconsistent, with periodic peaks, making data loads hard to manage.
Dow Chemical**
Dow enhances reliability with advanced analytics in JMP and is adding billions and billions in value with six sigma.
LUXOTTICA: What does 'big data' mean to Luxottica?
For Luxottica, Big Data includes everything they can find about their customer interactions
DUBLIN City council is leveraging Q: Is there a strong case to make for large cities to use Big Data Analytics and related information technologies? Identify and discuss examples of what can be done with analytics beyond what is portrayed in this application case.
For the Dublin case, Big Data Analytics were used primarily to ease traffic problems and better understand the traffic network
Location based analytics
Geospatial analytics Geospatial data Location information
Stream analytics applications
Health Services-PATIENT MONITORING is big thing
What was the proposed SOLUTION for Dublin?
IBM gave operators the ability to see the system as a whole instead of just individual corridors.
LIFE COACH STUDY: How can location-based analytics help individual consumers?
If a user on a smart phone enters data, the location sensors of the phone can help find others in that location who are facing similar circumstances, as well as local companies providing services and products that the consumer desires
OPENING VIGNETTE: OKLAHOMA GAS AND ELECTRIC EMPLOYS ANALYTICS TO PROMOTE SMART ENERGY USE
In the energy industry, SMART GRIDS are one of the most impactful applications of stream analytics.
QUIZNOS How can location based analytics help retailers in targeting customers?
Location-based behavioral targeting can help to narrow the characteristics of users who are most likely to utilize a retailer's services or products. This sort of analytics would typically target the tech-savvy and busy consumers of the company in question. Quizno's was NOT targeting consumers who cut coupons from the local newspaper and redeemed them at Subway. It used location-based analytics.
Big data TECHNOLOGIES
Map reduce Hadoop NoSQL
MapReduce
MapReduce is a technique popularized by Google that distributes the processing of very large multi-structured data files across a large cluster of ordinary machines/computer processors
What was the proposed SOLUTION for EBAY
Now that the solution is in place, eBay can more cost effectively process massive amounts of data at very high speeds. The new architecture serves a wide variety of new use cases, and its reliability and fault tolerance has been greatly enhanced. The load balancing helped the company meet its Big Data needs with the extremely fast data handling and application availability requirement, enabling the buying and selling of practically anything in an online marketplace.
1. What types of incentives might the consumers respond to in changing their energy use?
OG&E has started working on consumer-oriented efficiency programs to shift the residential customer's usage out of peak demand cycles. OG&E is targeting customers with its smart hours plan. * they have NOT started working on corporate oriented inefficiency programs
How is ParkPGH different from a "parking space-reporting" app?
ParkPGH does more than just report current parking-space availability. It is also capable of predicting future parking availability. *ParkPGH did NOT find the best corners for hailing a taxi cab based on the person's location, the day of the week, and the time of day.
KNOW THIS
Predictive analytics is beginning to enable development of software that is directly used by a consumer. One key concern in employing these technologies is the loss of PRIVACY
component based service orientation fosters :
Reusability, substitutability, extensibility, NOT ORIGINALITY
CHALLENGES of big data analytics
Skill availability (data scientists are in short supply)
TURNING MACHINE GENERATED STREAMING DATA INTO VALUABLE BUSINESS INSIGHTS
The company selected to work with Splunk, one of the leading analytics service providers in the area of turning machine-generated streaming data into valuable insights and provided beneficial results in the areas of application troubleshooting, operations, compliance, and security. *ALSO helps threat assessments and security monitoring
Dublin: What were the CHALLENGES Dublin City was facing?
The major problem was the difficulty in getting a good picture of traffic in the city from a high-level perspective.
The traditional location-based analytic techniques using geocoding of organizational locations and consumers hampers the organizations in understanding "true location-based" impacts. This is because of the following reasons EXCEPT for
The rich, detailed granularity of using postal codes has vast potential to pinpoint the growth opportunities of individual customers.
Value proposition
This characteristic of Big Data is its potential to contain more useful patterns and interesting anomalies than "small" data.
Volume
This is obviously the most common trait of Big Data. Many factors contributed to the exponential increase in data volume, such as transaction-based data stored through the years, text data constantly streaming in from social media, increasing amounts of sensor data being collected, automatically generated RFID and GPS data, and so forth.
Stream analytics VS PERPETUAL ANALYTICS Perpetual analytics is:
This type of analytics evaluates every incoming observation against all prior observations when analyzing Big Data in the context of intelligent systems and recognizing how the new observation relates to all prior observations enables the discovery of real-time insights
CHALLENGE W Dow Chemical
To turn data into knowledge that ensures the reliability of products, fosters innovation and informs decisions
Geospatial analytics examples SABRE AIRLINE SOLUTIONS APPLICATION
Traveler Security Geospatial-enabled dashboard Assess risks across global hotspots Interactive maps --Find current travelers --Respond quickly in the event of any travel disruption
Oklahoma 1. How does geospatial mapping help OG&E?
Using geospatial mapping and visual analytics, OG&E views a near-real-time version of data about its energy-efficient prospects spread over geographic areas and comes up with marketing initiatives that are most suitable for these customers. Geospatial mapping gives OG&E an easy way to narrow down to the specific customers in a geographic region based on their meter usage.
The 6 V's in BIG DATA
Volume Velocity Variability Veracity Value proposition Variety
The Web 2.0 Revolution and Online Social Networking
Web 2.0 is related to analytics *changing the web from passive to active a) consumer is the one that creates the content b) user created content is growing
Big Data Meets big science at CERN
World's largest particle PSYCHICS laboratory -Located near Geneva, Switzerland
What is CERN and why is it important
Worlds largest psychics lab and located in Geneva Switzerland
footpath
a system that ascertains how people move within a city or within a store
Park PGH
app for finding parking spot -real time parking for downtown pittsburgh
recommendation engines
are a huge success for retailer line Amazon.com
Empower
because Big Data and self-service business intelligence go hand in hand. Organizations with Big Data are over 70 percent more likely than other organizations to have BI/BA projects that are driven primarily by the business community, not by the IT group. Across a range of uses - from tackling new business problems, developing entirely new products and services, finding actionable intelligence in less than an hour, and blending data from disparate sources - Big Data has fired the imagination of what is possible through the application of analytics.
Visualize
because according to leading analytics research companies like Forrester and Gartner, enterprises find advanced data visualization platforms to be essential tools that enable them to monitor business, find patterns, and take action to avoid threats and snatch opportunities
Integrate
because blending data from disparate sources for your organization is an essential part of Big Data Analytics. Organizations that can blend different relational, semi-structured, and raw data sources in real time, without expensive up-front costs, will the ones that get the best value from Data.
Govern
because data govern has always been a challenging issue in IT, and it is getting even more puzzling with the advent of Big Data. More than 80 countries have data privacy laws. The European Union (EU) defines seven "safe harbor privacy principles" for the protection of their citizens' private data. In the US, Sarbanes-Oxley affects all publicly listed companies.
Simplfy
because it is hard to keep track of all of the new database vendors, open source projects, and Big Data service providers. It will be even more crowded and complicated in the years ahead.
Coexist
because using the strengths of each database platform and enabling them to coexist in your organization's data architecture are essential. There is ample literature that talks about the necessity of maintaining and nurturing synchronicity of traditional data warehouses with the capabilities of new platforms
Evangelize
because with the backing of one or more executive sponsors, future business graduates from LSU E.J. Ourso College of Business like yourself can get the ball rolling and instill a virtuous cycle: The more departments in your organization realize actionable benefits, the more pervasive analytics becomes across your organization. Fast, easy-to-use visual analytics is the key that opens the door to organization-wide analytics adoption and collaboration.
What were the CHALLENGES of EBAY?
eBay was experiencing explosive data growth and needed a solution that did not have the typical bottlenecks, scalability issues, and transactional constraints associated with common relational database approaches. eBay also needed a solution to perform rapid analysis on a broad assortment of the structured and unstructured data it captured. The solution did NOT integrate into a single Big Data Center infrastructure.
A use case in the energy industry for stream analytics
is a classic SMART GRID application for the electric power supply chain.
Critical event processing
is a method of capturing, tracking, and analyzing streams of data to detect events (out of normal happenings) of certain types that are worthy of the effort.
privacy
is the right to be left alone and the right to be free from unreasonable personal intrusions
Geospatial data
is the static location data used by these location based analytic applications
Cloud computing
originates from a reference to the internet as a "cloud" and is a combination of several information technology components as services
Veracity
refers to the conformity to facts: accuracy, quality, truthfulness, or trustworthiness of Big Data
real time location intelligence
targeting the right customer based on their behavior over geographic locations
Service oriented
thinking is one of the fastest growing paradigms today *Service orientation +DSS/BI which includes optimization, data mining, text mining, simulation, automated decision systems