ETC3250/5250: Introduction to Machine Learning

.info-box.w-50.bg-white[
These slides are viewed best by Chrome or Firefox and occasionally need to be refreshed if elements did not load properly. See <a href=lecture-11b.pdf>here for the PDF <i class="fas fa-file-pdf"></i></a>. 
]

<br>

---

# .monash-blue[ETC3250/5250: Introduction to Machine Learning]

<br>

.bottom_abs.width100[

Lecturer: *Professor Di Cook*

Department of Econometrics and Business Statistics

<i class="fas fa-envelope"></i>  ETC3250.Clayton-x@monash.edu

<i class="fas fa-calendar-alt"></i> Week 11b

<br>

]

---
# Learning objectives for this class

- .monash-orange2[Select and develop] appropriate models for regression, classification or clustering 
- .monash-orange2[Estimate and simulate] from a variety of statistical models, and measure the uncertainty of a prediction using resampling methods
-  Manage large data sets in a modern software environment, and .monash-orange2[explain and interpret] the analyses undertaken clearly and effectively
- .monash-orange2[Apply] business analytic tools to produce innovative solutions in finance, marketing, economics and related areas

---

- .monash-orange2[What have we covered]
    - Key concepts

]]
.column[.top50px[

- What .monash-orange2[type of problem] do you have? 
    - supervised (regression, classification), 
    - unsupervised (PCA, cluster analysis)
- Should you use a .monash-orange2[flexible or less flexible] model? 
    - Parametric (more assumptions, easier estimation, strong inference)
    - Non-parametric (more flexible, fewer assumptions, more observations needed, less interpretable)
        - e.g. kNN, smoothers
- .monash-orange2[Measuring fit]
    - MSE, `\(R^2\)`, BIC
    - accuracy, misclassification
- .monash-orange2[Bias vs variance] trade-off
    - bias: error that is introduced by modeling a complicated problem by a simpler model
    - variance: how much your estimate would change if you had different training data

]]

---

- .monash-orange2[What have we covered]
    - Key concepts
    - Re-sampling

]]
.column[.top50px[

- Training/test/validation sets
- LOOCV, k-fold cross-validation
    - shortcut for computing `\(MSE_i\)`
- Bootstrap
    - out-of-bag error

]]
---

- .monash-orange2[What have we covered]
    - Key concepts
    - Re-sampling
    - Models

]]
.column[.top50px[

- Linear (and polynomial) regression
- .monash-orange2[Logistic regression]
    - Logit function
    - Parameter interpretation
    - relationship to neural networks
- .monash-orange2[Linear discriminant analysis]
    - assumptions
    - relationship to normal model
    - dimension reduction
- Quadratic discriminant analysis
    - heteroskedastic group variance-covariance

]]

---

- .monash-orange2[What have we covered]
    - Key concepts
    - Re-sampling
    - Models

]]
.column[.top50px[

- .monash-orange2[Decision trees]
    - Regression: SST - (SSL+SSR)
    - Classification: Gini, entropy
- .monash-orange2[Random forests]
    - Bagging
    - Sampling variables
    - Permutation 
    - Diagnostics: Variable importance, Vote matrix, Proximity
- .monash-orange2[Support vector machines]
    - maximal margin
    - relationship to LDA
    - kernels
- .monash-orange2[Neural networks]
    - deconstructing the model fit
    - instability

]]

---

- .monash-orange2[What have we covered]
    - Key concepts
    - Re-sampling
    - Models
    - Visualisation

]]
.column[.top50px[

- .monash-orange2[Importance]
    - Understanding structure in data, inform the model choice
    - Diagnose 
    - Check assumptions
- .monash-orange2[Model in the data space]
- .monash-orange2[Inference] using randomization, bootstrap and permutation
- .monash-orange2[Tours]
    - relationship to a biplot
    - Matching structure to variable contribution
    - Types: grand, guided, 
- .monash-orange2[Parallel coordinate plots]
    - Ordering variables
    - Scaling of axes
- .monash-orange2[Pairs plots]
]]

---

- .monash-orange2[What have we covered]
    - Key concepts
    - Re-sampling
    - Models
    - Visualisation
    - Dimension reduction

]]
.column[.top50px[

- .monash-orange2[Principal component analysis]
    - Eigendecomposition of variance-covariance
    - Scaling of variables
    - Choosing k
    - Total variance, and proportion
    - biplot
    - relationship to projection pursuit
- .monash-orange2[Regularization]
    - Ridge regression
    - Lasso
    
]]

---

- .monash-orange2[What have we covered]
    - Key concepts
    - Re-sampling
    - Models
    - Visualisation
    - Dimension reduction
    - Cluster analysis

]]
.column[.top50px[
- .monash-orange2[Interpoint distance], similarity and dissimilarity
    - Rules for distance
- `\(k\)`-means
    - algorithm
    - random starts
- Hierarchical
    - Linkage
    - Dendrogram
- Choosing `\(k\)`, comparing solutions and summarising
    - Cluster fit stats: WBRatio
    - Summary statistics by cluster
    - Dimension reduction to show clusters
    - Multivariate plots of clusters
- Model-based clustering

]]
---
layout: true
class: shuriken-full white

.blade1.bg-green[.content.center.vmiddle[
.white.font_large[How do you do well in this class?]
]]
.blade2.bg-purple[.content.center.vmiddle[
Turn up to class, summarise your notes after each, note what you understand, and what you don't 🌊
]]
.blade3.bg-deep-orange[.content.center.vmiddle[
Do exercises from the textbook related to material each week, check your answers with online solutions 🧗
]]
.blade4.bg-pink[.content.center.vmiddle[
Participate actively in computer labs, work with team mates to solve problems, get best answers 🍒
]]
---

---

---

---

---

---
layout: false
class: split-50
.flex[
.w-45[
# After this course

.font_small[This unit covers the methods and practice of statistical machine learning for modern data analysis problems. Topics covered will include recommender systems, social networks, text mining, matrix decomposition and completion, and sparse multivariate methods. All computing will be conducted using the R programming language.

Prerequisites: ETC3250 or FIT3154]
]
.w-45[
.monash-blue2[**ETC5550 - Advanced Statistical Modelling**]

.font_small[This unit introduces extensions of linear regression models for handling a wide variety of data analysis problems. Three extensions will be considered: generalised linear models for handling counts and binary data; mixed-effect models for handling data with a grouped or hierarchical structure; and non-parametric regression for handling non-linear relationships. All computing will be conducted using R.

Prerequisites: ETC2410, ETC2420, ETC3440 or equivalent.]
]]
---

For those of you wrapping up your studies, good luck in your future, and I hope that what you have learned in this class can be useful in your journey.

My data analyses, slides, papers, web site uses the [R](https://www.r-project.org) .monash-orange2[workflow]. Very powerful framework for a business analyst/data scientist. There is .monash-orange2[no competition] with [python](https://www.python.org). Both extraordinary resources for our world. The people who develop and maintain these resources need our applause. We are extremely .monash-orange2[lucky] to be living in a world where these resources are available free to us.

---

background-size: cover
class: title-slide
background-image: url("images/bg-02.png")

<a rel="license" href="http://creativecommons.org/licenses/by-sa/4.0/"><img alt="Creative Commons License" style="border-width:0" src="https://i.creativecommons.org/l/by-sa/4.0/88x31.png" /></a><br />This work is licensed under a <a rel="license" href="http://creativecommons.org/licenses/by-sa/4.0/">Creative Commons Attribution-ShareAlike 4.0 International License</a>.

.bottom_abs.width100[

Lecturer: *Professor Di Cook*

Department of Econometrics and Business Statistics

<i class="fas fa-envelope"></i>  ETC3250.Clayton-x@monash.edu

<i class="fas fa-calendar-alt"></i> Week 11b

<br>

]