What is Machine Learning

supervised learning

•Supervised Learning is where both the predictors, Xi, and the response, Yi, are observed.

irreducible vs. reducible error

•The accuracy: The prediction depends on two quantities. The reducible and irreducible error. •Irreducible error is var(ε_i): Inherent noise or variability in data that cannot be reduced. Reducible error: Error that can be reduced by improving the model or learning process We can increase the accuracy of the model by choosing the most appropriate machine learning model.

prediction (estimation)

If we can produce a good estimate for f(X_i) and var(ε) is not too large then we can make accurate predictions for the response, Y, based on a new value of X.

ML learns a function

The aim is to learn a function or model that can accurately map inputs X to outputs Y (the response) by capturing patterns and relationships in the data.

disadvantages of non parametric methods

•A very large number of observations is required to obtain an accurate estimate of f

white box models

•Highly interpretable and transparent. •Internal workings and decision-making processes are visible and easily understood. •Structure, parameters, and rules can be inspected and analyzed. Examples: linear regression, decision trees, rule-based models.

unsupervised learning

•In this situation only the Xi's are observed. •We need to use the Xi's to guess what Y would have been and build a model from there. •A common example is market segmentation where we try to divide potential customers into groups based on their characteristics.

nonparametric methods

•It learns directly from the data without making explicit assumptions regarding functional form or distribution of data. •Non-parametric models adapt to complex patterns and capture nonlinear relationships but tend to need more training data. •They do not make explicit assumptions about the functional form of f.

black box models

•Less interpretable and operate with limited transparency. •Internal mechanisms are complex and not easily interpretable. •Internal workings and parameters are often hidden or not directly accessible. •Focus on high predictive performance without explicit rationale. •Examples: deep neural networks, support vector machines, ensemble methods.

parametric methods

•Parametric learning assumes a specific functional form or distribution for the data. •It estimates a fixed number of parameters that define the model. •The model is fully defined once these parameters are estimated. •Simpler representation but may make strong assumptions about the data distribution. •These assumptions can limit their flexibility and applicability in complex scenarios.

classical programming

•Programmer provides specific instructions. •Deterministic results based on explicit rules. •Cannot generalize to new situations.

regression vs. classification

•Regression Problems: covers situations where Y is continuous, numerical, quantitative (supervised) •Classification Problems: covers situations where Y is categorical, qualitative (supervised)

machine learning

Computer learns patterns from data. •Estimates relationships between inputs and outputs. •Probabilistic results based on learned patterns. Can generalize to new situations and discover hidden patterns.

examples of non parametric methods

Examples of non-parametric learning algorithms include decision trees, random forests, support vector machines, and KNN. •Non-parametric models tend to be more flexible and expressive.

examples of parametric methods

Examples of parametric learning algorithms include linear regression, logistic regression, and Gaussian Naive Bayes.

inference (interpretability)

The nature of the relationship between Y and the X's Important predictors: Only a few variables significantly affect Y, and identifying them is valuable. Positive or negative: The relationship can be positive (Y increases with X) or negative (Y decreases with X). Linearity vs. complexity: It can be a simple linear relationship or more complex.

advantages of non parametric methods

They accurately fit a wider range of possible shapes of f.