Module 11: Introduction to Big Data Techniques (FinTech)

Machine learning can produce models that overfit or underfit the data. Underfitting occurs when...

(Model is not complex enough) Occurs when the machine fails to identify actual patterns and relationships, treating true parameters as noise

Machine learning can produce models that overfit or underfit the data. Overfitting occurs when...

(Too complex a model) Occurs when the machine learns the input and output data too exactly, treats noise as true parameters, and identifies spurious patterns and relationships

Challenges of Big Data

1. High-Quality data is required 2. Accounting for outliers, bad or missing data, sampling biases 3. Processing and organizing volume data

High-frequency Trading (example of algorithmic)

That identifies and takes advantage of intraday securities mispricings.

Storage (Data Processing Method)

This is archiving and accessing data.

Curation (Data Processing Method)

This is assuring data quality by adjusting for bad or missing data.

Capture (Data Processing Method)

This is collecting data and transforming it into usable forms.

Search (Data Processing Method)

This is examining stored data to find needed information.

Transfer (Data Processing Method)

This is moving data from their source or a storage medium to where they are needed.

Deep Learning

a technique that uses layers of neural networks to identify patterns, beginning with simple patterns and advancing to more complex ones. Deep learning may employ supervised or unsupervised learning. Some of the applications of deep learning include image and speech recognition.

Data Processing Methods Include the Following:

Capture. This is collecting data and transforming it into usable forms. Curation. This is assuring data quality by adjusting for bad or missing data. Storage. This is archiving and accessing data. Search. This is examining stored data to find needed information. Transfer. This is moving data from their source or a storage medium to where they are needed.

Fintech Definition

Developments in technology that can be applied to the financial services industry. Companies that are in the business of developing technologies for the finance industry are often referred to as fintech companies.

Machine Learning

A computer algorithm designed to learn, detect, and recognize patterns in a the input data: 1. Training Dataset in which the algorithm looks for relationships 2. Validation Dataset to test and analyze the predictive ability 3. Test Dataset to analyze their predictive ability 4. Required vast amounts of data, any distribution, but a "black box"

Internet of Things (IoT)

All devices with connections to the internet

Artificial Intelligence

Computer systems that can be programmed to simulate human cognition

Algorithmic Trading

Computerized securities trading based on a predetermined set of rules. For example, algorithms may be designed to enter the optimal execution instructions for any given trade based on real-time price and volume data. Algorithmic trading can also be useful for executing large orders by determining the best way to divide the orders across exchanges.

Data Science

Concerns how we extract information from Big Data. Data science describes methods for processing and visualizing data.

Visualization Techniques

Include the familiar charts and graphs that display structured data. To visualize less structured data requires other methods. Some examples of these are word clouds that illustrate the frequency with which words appear in a sample of text, or mind maps that display logical relations among concepts.

Corporate Exhaust

Includes bank records and retail scanner data.

Alternative Data (Nontraditional)

Individuals generate usable data such as social media posts, online reviews, email, and website visits. Businesses generate potentially useful information such as bank records and retail scanner data. These kinds of data are referred to as corporate exhaust. Sensors, such as radio frequency identification chips, are embedded in numerous devices such as smartphones and smart buildings. The broad network of such devices is referred to as the Internet of Things.

Neural Networks (Example of AI)

Programmed to process information in a similar way to the human brain.

Big Data

Refers to all the potentially useful information that is generated in the economy. This includes not only data from traditional sources, such as financial markets, company financial reports, and government economic statistics, but also alternative data from nontraditional sources.

Characteristics of Big Data

Volume, Velocity, and Variety Volume: grows by order of magnitude (megabytes to petabytes) Velocity: speed of communication (real-time data has low latency) Variety: the structure in which the data exists

Text Analytics

the analysis of unstructured data in text or voice forms. An example of text analytics is analyzing the frequency of words and phrases. In the finance industry, text analytics have the potential to partially automate specific tasks such as evaluating company regulatory filings.

Supervised Learning

the input and output data are labeled, the machine learns to model the outputs from the inputs, and then the machine is given new data on which to use the model

Unsupervised Learning

the input data are not labeled, and the machine learns to describe the structure of the data

Natural Language Processing (NLP)

the use of computers and artificial intelligence to interpret human language. Speech recognition and language translation are among the uses of natural language processing. Possible applications in finance could be to check for regulatory compliance in an examination of employee communications, or to evaluate large volumes of research reports to detect more subtle changes in sentiment than can be discerned from analysts' recommendations alone.