Ch11

For data mining to work, two critical conditions need to be present:

1) the organization must have clean, consistent data, and (2) the events in that data should reflect current and future trends. The recent financial crisis provides lessons on what can happen when either of these conditions isn't met.

column or field

A column in a database table. Columns represent each category of data contained in a record (e.g., first name, last name, ID number, date of birth).

data warehouse

A data warehouse is a set of databases designed to support decision making in an organization. It is structured for fast online queries and exploration. Data warehouses may aggregate enormous amounts of data from many different operational systems.

relational database

A database where multiple tables are related based on common keys relational databases are far and away the most popular. And all SQL databases are relational databases.

A table of file refers to

A list of data

Police use

Ad hoc query to access internal data such as 911 logs and police reports, and combines this with outside data including neighborhood demographics, payday schedules, weather reports, traffic patterns, sports events, and more. Experienced officers dive into this data, exploring when and where crimes occur

Customer loyalty cards

At casinos are used as switching costs

Walmart's supply chain

Back-office scanners keep track of inventory as supplier shipments come in. Suppliers are rated based on timeliness of deliveries, and you've got to be quick to work with Wal-Mart. In order to avoid a tractor-trailer traffic jam in store parking lots, deliveries are choreographed to arrive at intervals less than ten minutes apart.

CRM data

CRM or customer relationship management systems are often used to empower employees to track and record data at nearly every point of customer contact. Someone calls for a quote? Brings a return back to a store? Writes a complaint e-mail? A well-designed CRM system can capture all these events for subsequent analysis or for triggering follow-up events

Data quality

Can our data be trusted as accurate? Is it clean, complete, and reasonably free of errors? How can the data be made more accurate and valuable for analysis? Will we need to 'scrub,' calculate, and consolidate data so that it can be used?

Data sourcing

Can we even get the data we'll need? Where can this data be obtained from? Is it available via our internal systems? Via third-party data aggregators? Via suppliers or sales partners? Do we need to set up new systems, surveys, and other collection efforts to acquire the data we need?

DBA (Database Administrator)

Creates and maintains databases, queries, and reports. Needs to know query languages and how to set user permissions and care for security. Also needs good communication skills and above average critical thinking.

Data aggregators sell digital dossiers to companies that want to know their customers better. This practice is called ________.

Customer profiling

Some of the key areas where businesses are leveraging data mining include the following:

Customer segmentation—figuring out which customers are likely to be the most valuable to a firm. Marketing and promotion targeting—identifying which customers will respond to which offers at which price at what time. Market basket analysis—determining which products customers buy together, and how an organization can use this information to cross-sell more products or services. Collaborative filtering—personalizing an individual customer's experience based on the trends and preferences identified across similar customers. Customer churn—determining which customers are likely to leave, and what tactics can help the firm avoid unwanted defections. Fraud detection—uncovering patterns consistent with criminal activity. Financial modeling—building trading systems to capitalize on historical trends. Hiring and promotion—identifying characteristics consistent with employee success in the firm's various roles.

When does data become information?

Data becomes information when it's presented in a context so that it can answer a question or support decision making. And it's when this information can be combined with a manager's knowledge—their insight from experience and expertise—that stronger decisions can be made.

Data from external sources

Data bought from sources available to all might not yield competitive advantage on its own, but it can provide key operational insight for increased efficiency and cost savings. And when combined with a firm's unique data assets, it may give firms a high-impact edge.

Walmart and data mining

Data mining also helps the firm tighten operational forecasts, helping to predict things like how many cashiers are needed at a given store at various times of day throughout the year. Data drives the organization, with mined reports forming the basis of weekly sales meetings, as well as executive strategy sessions.

data mining

Data mining is the process of using computers to identify hidden patterns and to build models from large data sets.

Online Analytical Processing (OLAP)

Data used in OLAP reporting is usually sourced from standard relational databases, but it's calculated and summarized in advance, across multiple dimensions, with the data stored in a special database called a data cube. This extra setup step makes OLAP fast (sometimes one thousand times faster than performing comparable queries against conventional relational databases). Given this kind of speed boost, it's not surprising that data cubes for OLAP access are often part of a firm's data mart and data warehouse efforts.

Walmart challenges abroad

Despite its success, Wal-Mart is a mature business that needs to find huge markets or dramatic cost savings in order to boost profits and continue to move its stock price higher. The firm's success also makes it a high impact target for criticism and activism. And the firm's data assets could not predict impactful industry trends such as the rise of Target and other upscale discounters.

Retail link

Each time an item is scanned by a Wal-Mart cash register, Retail Link not only records the sale, it also automatically triggers inventory reordering, scheduling, and delivery. This process keeps shelves stocked, while keeping inventories at a minimum.

TPS (Transaction Processing System)

For organizations that sell directly to their customers, transaction processing systems (TPS) represent a source of potentially useful data. Example when customer pays with card retailer learns info about that customer

Generic algorithms (GAs)

Genetic algorithms are model building techniques where computers examine many potential solutions to a problem, iteratively modifying (mutating) various mathematical models, and comparing the mutated models to search for a best alternative. Genetic algorithms have been used to build everything from financial trading models to handling complex airport scheduling, to designing parts for the international space station

Data quantity

How much data do we need?

reasons why many organizations have data that can't be converted to actionable information.

Incompatible systems- Legacy systems often limit data utilization because they were not designed to share data, aren't compatible with newer technologies, and aren't aligned with the firm's current business needs. Operational Data Can't Always Be Queried -Another problem when turning data into information is that most transactional databases aren't set up to be simultaneously accessed for reporting and analysis

Database

Is a single table or a collection of related tables.

Data mining disadvantages

It's possible to over-engineer a model, building it with so many variables that the solution arrived at might only work on the subset of data you've used to create it. You might also be looking at a random but meaningless statistical fluke. Sometimes durable and useful patterns just aren't in your data. Finally, sometimes a pattern is uncovered but determining the best choice for a response is less clear

How to determine if you are looking at a random occurrence with data?

One way to test to see if you're looking at a random occurrence in the numbers is to divide your data, building your model with one portion of the data, and using another portion to verify your results

canned reports

Reports that provide regular summaries of information in a predetermined format. often developed by information systems staff and formats can be difficult to alter.

Data can also be collected by external sources

Sometimes it makes sense to combine a firm's data with bits brought in from the outside. Many firms, for example, don't sell directly to consumers (this includes most drug companies and packaged goods firms). If your firm has partners that sell products for you, then you'll likely rely heavily on data collected by others.

e-discovery

Sometimes the law requires organizations to dive into their electronic records. E- discovery refers to identifying and retrieving relevant electronic information to support litigation efforts. E-discovery is something a firm should account for in its archiving and data storage plans. Unlike analytics that promise a boost to the bottom line, there's no profit in complying with a judge's order—it's just a sunk cost. But organizations can be compelled by court order to scavenge their bits, and the cost to uncover difficult to access data can be significant, if not planned for in advance.

Firms can retrieve data from

Supply chain management (SCM) and enterprise resource planning (ERP) systems and from surveys Many CRM products also have survey capabilities that allow for additional data gathering at all points of customer contact.

Data is hard to match with a specific customer

That pays with cash but they can be linked if they use a loyalty card

Query-and-reporting tools

The idea behind query and reporting tools is to present users with a subset of requested data, selected, sorted, ordered, calculated, and compared, as needed. Managers use these tools to see and explore what's happening inside their organizations.

artificial intelligence (AI)

The science of designing and programming computer systems to do intelligent things and to simulate human thought processes, such as intuitive reasoning, learning, and understanding language.

With large scale data analytics projects

This work should start with a clear vision with business-focused objectives. When senior executives can see objectives illustrated in potential payoff, they'll be able to champion the effort, and experts agree, having an executive champion is a key success factor. Focusing on business issues will also drive technology choice, with the firm better able to focus on products that best fit its needs. Once a firm has business goals and hoped-for payoffs clearly defined, it can address the broader issues needed to design, develop, deploy, and maintain its system

Walmart sharing data keeping secrets

To help suppliers become more efficient, and as a result lower prices, Wal-Mart shares data with them. Walmart does not share data with information brokers This sharing allows smaller firms to pool their data to provide more comprehensive insight on market behavior. But Wal-Mart stopped sharing data with these agencies years ago. The firm's scale is so big, the additional data provided by brokers wasn't adding much value, and it no longer made sense to allow competitors access to what was happening in its own huge chunk of retail sales.

What is the goal of data?

To turn into information.

ad hoc reporting tools

Tools that put users in control so that they can create custom reports on an as-needed basis by selecting fields, ranges, summary conditions, and other parameters.

SQL (Structured Query Language)

Used to create and manipulate databases most popular language for database is SQL

Walmart's data mining powers

Wal-Mart also mines its mother lode of data to get its product mix right under all sorts of varying environmental conditions, protecting the firm from "a retailer's twin nightmares: too much inventory, or not enough" (Hays, 2004). For example, the firm's data mining efforts informed buyers that customers stock up on certain products in the days leading up to predicted hurricanes. Bumping up prestorm supplies of batteries and bottled water was a no brainer, but the firm also learned that Pop-Tarts sales spike seven fold before storms hit, and that beer is the top prestorm seller. This insight has lead to truckloads full of six packs and toaster pastries streaming into gulf states whenever word of a big storm surfaces

Data relevance

What data is needed to compete on analytics and to meet our current and future goals?

data governance

What rules and processes are needed to manage data from its creation through its retirement? Are there operational issues (backup, disaster recovery)? Legal issues? Privacy issues? How should the firm handle security and access?

Data hosting

Where will the systems be housed? What are the hardware and networking requirements for the effort?

Business Intelligence (BI)

a broad category of applications, technologies, and processes for gathering, storing, accessing, and analyzing data to help business users make better decisions

data mart

a database focused on addressing the concerns of a specific problem (e.g., increasing customer retention, improving product quality) or business unit (e.g., marketing, engineering). Example Marts and warehouses may contain huge volumes of data. For example, a firm may not need to keep large amounts of historical point-of-sale or transaction data in its operational systems, but it might want past data in its data mart so that managers can hunt for patterns and trends that occur over time.

Getting data in

a format that can be used for analytics is hard, complex, and challenging work. Large data warehouses can cost millions and take years to build. Every dollar spent on technology may lead to five to seven more dollars on consulting and other services

DBMS (Database Management System)

a program used to create, process, and administer a database. This is where Databases are created, maintained, and manipulated

Dashboards

a sort of heads-up display of critical indicators, letting managers get a graphical glance at key performance metrics.

consumer-level targeting

allows the firm to tailor its marketing messages to specific subgroups, promoting the right offer through the right channel at the right time and the right price.

Various database might be focused on

combination of functional areas (sales, product returns, inventory, payroll), geographical regions, or business units Firms often create specialized databases for recording transactions, as well as databases that aggregate data from multiple sources in order to support reporting and analysis.

When is data considered a competitive advantage ?

data a firm can leverage is a true strategic asset when it's rare, valuable, imperfectly imitable, and lacking in substitutes. however, advantages based on capabilities and data that others can acquire will be short-lived.

analytics (business analytics)

describing the extensive use of data, statistical and quantitative analysis, explanatory and predictive models, and fact-based management to drive decisions and actions

CLV (customer lifetime value)

epresents the present value of the likely future income stream generated by an individual purchaser1. Once you know this, you can get a sense of how much you should spend to keep that customer coming back. You can size them up next to their peer group and if they fall below expectations you can develop strategies to improve their spending.

manager using an OLAP tool can quickly

explore and compare data across multiple factors such as time, geography, product lines, and so on. In fact, OLAP users often talk about how they can "slice and dice" their data, "drilling down" inside the data to uncover new insights. And while conventional reports are usually presented as a summarized list of information, OLAP results look more like a spreadsheet, with the various dimensions of analysis in rows and columns, with summary values at the intersection.

With data mining the

first findings don't always reveal an optimal course of action. Example An analysis of product sales data showed several money-losing products, including a type of bread known as "milk loaf." Drop those products, right? Not so fast. Further analysis showed milk loaf was a "destination product" for a loyal group of high-value customers, and that these customers would shop elsewhere if milk loaf disappeared from Tesco shelves. The firm kept the bread as a loss-leader and retained those valuable milk loaf fans

the importance of recruiting a data mining and business analytics team that possesses three critical skills:

information technology (for understanding how to pull together data, and for selecting analysis tools), statistics (for building models and interpreting the strength and validity of results), and business knowledge (for helping set system goals, requirements, and offering deeper insight into what the data really says about the firm's operating environment). Miss one of these key functions and your team could make some major mistakes.

Data that can be purchased from aggregators

may not in and of itself yield sustainable competitive advantage since others may have access to this data, too. However, when combined with a firm's proprietary data or integrated with a firm's proprietary procedures or other assets, third-party data can be a key tool for enhancing organizational performance.

Data

raw numbers or facts

row or record

represents a single instance of whatever the table keeps track of.

Database

simply a list (or more likely, several related lists) of data.

Walmart

source of competitive advantage is scale. But firms don't turn into giants overnight. Wal-Mart grew in large part by leveraging information systems to an extent never before seen in the retail industry.

key

the field used to relate tables in a database.