_index.md (8851B)
1 +++ 2 title = 'Machine Learning' 3 +++ 4 # Machine learning 5 Exam - [cheat sheet](formula-cheat-sheet.pdf) available for formulas! 6 7 * [Introduction](introduction) 8 * [What is ML?](introduction#what-is-ml) 9 * [Supervised ML](introduction#supervised-ml) 10 * [Classification](introduction#classification) 11 * [Regression](introduction#regression) 12 * [Unsupervised ML](introduction#unsupervised-ml) 13 * [What isn't ML?](introduction#what-isn-t-ml) 14 * [Methodology](methodology) 15 * [Performing an experiment](methodology#performing-an-experiment) 16 * [What if you need to test many models?](methodology#what-if-you-need-to-test-many-models) 17 * [The modern recipe](methodology#the-modern-recipe) 18 * [Cross-validation](methodology#cross-validation) 19 * [What to report](methodology#what-to-report) 20 * [Classification](methodology#classification) 21 * [What's a good error (5%)?](methodology#what-s-a-good-error-5) 22 * [Performance metrics](methodology#performance-metrics) 23 * [Confusion matrix (contingency table)](methodology#confusion-matrix-contingency-table) 24 * [Precision and recall](methodology#precision-and-recall) 25 * [Regression](methodology#regression) 26 * [Errors & confidence intervals](methodology#errors-confidence-intervals) 27 * [The no-free-lunch theorem and principle](methodology#the-no-free-lunch-theorem-and-principle) 28 * [Cleaning your data](methodology#cleaning-your-data) 29 * [Missing data](methodology#missing-data) 30 * [Outliers](methodology#outliers) 31 * [Class imbalance](methodology#class-imbalance) 32 * [Choosing features](methodology#choosing-features) 33 * [Normalisation & standardisation](methodology#normalisation-standardisation) 34 * [Normalisation](methodology#normalisation) 35 * [Standardisation](methodology#standardisation) 36 * [Whitening](methodology#whitening) 37 * [Dimensionality reduction](methodology#dimensionality-reduction) 38 * [Linear models](linear-models) 39 * [Defining a model](linear-models#defining-a-model) 40 * [But which model fits best?](linear-models#but-which-model-fits-best) 41 * [Mean squared error loss](linear-models#mean-squared-error-loss) 42 * [Optimization & searching](linear-models#optimization-searching) 43 * [Black box optimisation](linear-models#black-box-optimisation) 44 * [Random search](linear-models#random-search) 45 * [Simulated annealing](linear-models#simulated-annealing) 46 * [Parallel search](linear-models#parallel-search) 47 * [Branching search](linear-models#branching-search) 48 * [Gradient descent](linear-models#gradient-descent) 49 * [Classification losses](linear-models#classification-losses) 50 * [Least-squares loss](linear-models#least-squares-loss) 51 * [Neural networks (feedforward)](linear-models#neural-networks-feedforward) 52 * [Overview](linear-models#overview) 53 * [Classification](linear-models#classification) 54 * [Dealing with loss - gradient descent & backpropagation](linear-models#dealing-with-loss-gradient-descent-backpropagation) 55 * [Support vector machines (SVMs)](linear-models#support-vector-machines-svms) 56 * [Summary of classification loss functions](linear-models#summary-of-classification-loss-functions) 57 * [Probability](probability) 58 * [Probability basics](probability#probability-basics) 59 * [Probability theory](probability#probability-theory) 60 * [(Naive) Bayesian classifiers](probability#naive-bayesian-classifiers) 61 * [Logistic "regression" (classifier)](probability#logistic-regression-classifier) 62 * [Information theory](probability#information-theory) 63 * [Maximum likelihood](probability#maximum-likelihood) 64 * [Normal distributions (Gaussians)](probability#normal-distributions-gaussians) 65 * [1D normal distribution (Gaussian)](probability#1d-normal-distribution-gaussian) 66 * [Regression with Gaussian errors](probability#regression-with-gaussian-errors) 67 * [n-D normal distribution (multivariate Gaussian)](probability#n-d-normal-distribution-multivariate-gaussian) 68 * [Gaussian mixture model](probability#gaussian-mixture-model) 69 * [Expectation-maximisation](probability#expectation-maximisation) 70 * [Deep learning](deep-learning) 71 * [Deep learning systems (autodiff engines)](deep-learning#deep-learning-systems-autodiff-engines) 72 * [Tensors](deep-learning#tensors) 73 * [Functions on tensors](deep-learning#functions-on-tensors) 74 * [Backpropagation revisited](deep-learning#backpropagation-revisited) 75 * [Multivariate chain rule](deep-learning#multivariate-chain-rule) 76 * [Backpropagation with tensors - matrix calculus](deep-learning#backpropagation-with-tensors-matrix-calculus) 77 * [Making deep neural nets work](deep-learning#making-deep-neural-nets-work) 78 * [Overcoming vanishing gradients](deep-learning#overcoming-vanishing-gradients) 79 * [Minibatch gradient descent](deep-learning#minibatch-gradient-descent) 80 * [Optimizers](deep-learning#optimizers) 81 * [Momentum](deep-learning#momentum) 82 * [Nesterov momentum](deep-learning#nesterov-momentum) 83 * [Adam](deep-learning#adam) 84 * [Regularizers](deep-learning#regularizers) 85 * [L2 regularizer](deep-learning#l2-regularizer) 86 * [L1 regulariser](deep-learning#l1-regulariser) 87 * [Dropout regularisation](deep-learning#dropout-regularisation) 88 * [Convolutional neural networks](deep-learning#convolutional-neural-networks) 89 * [Deep learning vs machine learning](deep-learning#deep-learning-vs-machine-learning) 90 * [Generators](deep-learning#generators) 91 * [Generative adversarial networks](deep-learning#generative-adversarial-networks) 92 * [Vanilla GANs](deep-learning#vanilla-gans) 93 * [Conditional GANs](deep-learning#conditional-gans) 94 * [CycleGAN](deep-learning#cyclegan) 95 * [StyleGAN](deep-learning#stylegan) 96 * [What can we do with a generator?](deep-learning#what-can-we-do-with-a-generator) 97 * [Autoencoders](deep-learning#autoencoders) 98 * [Turning an autoencoder into a generator](deep-learning#turning-an-autoencoder-into-a-generator) 99 * [Variational autoencoders](deep-learning#variational-autoencoders) 100 * [Tree models and ensembles](tree-models-and-ensembles) 101 * [Tree models](tree-models-and-ensembles#tree-models) 102 * [Decision trees (categorical)](tree-models-and-ensembles#decision-trees-categorical) 103 * [Regression trees (numeric)](tree-models-and-ensembles#regression-trees-numeric) 104 * [Generalization hierarchy](tree-models-and-ensembles#generalization-hierarchy) 105 * [Ensembling methods](tree-models-and-ensembles#ensembling-methods) 106 * [Bagging](tree-models-and-ensembles#bagging) 107 * [Boosting](tree-models-and-ensembles#boosting) 108 * [AdaBoost (binary classifiers)](tree-models-and-ensembles#adaboost-binary-classifiers) 109 * [Gradient boosting](tree-models-and-ensembles#gradient-boosting) 110 * [Stacking](tree-models-and-ensembles#stacking) 111 * [Sequences, models for sequential data](models-for-sequential-data) 112 * [Sequences](models-for-sequential-data#sequences) 113 * [Markov models](models-for-sequential-data#markov-models) 114 * [Embedding models](models-for-sequential-data#embedding-models) 115 * [Recurrent neural networks](models-for-sequential-data#recurrent-neural-networks) 116 * [LSTMs](models-for-sequential-data#lstms) 117 * [Matrix models](matrix-models) 118 * [Recommender systems](matrix-models#recommender-systems) 119 * [Matrix factorization](matrix-models#matrix-factorization) 120 * [Bias control](matrix-models#bias-control) 121 * [The 'cold start' problem](matrix-models#the-cold-start-problem) 122 * [Graph models](matrix-models#graph-models) 123 * [Validating embedding models](matrix-models#validating-embedding-models) 124 * [Reinforcement learning](reinforcement-learning) 125 * [What is reinforcement learning?](reinforcement-learning#what-is-reinforcement-learning) 126 * [Approaches](reinforcement-learning#approaches) 127 * [Random search](reinforcement-learning#random-search) 128 * [Policy gradient](reinforcement-learning#policy-gradient) 129 * [Q-learning](reinforcement-learning#q-learning) 130 * [Alpha-stuff](reinforcement-learning#alpha-stuff) 131 * [AlphaGo](reinforcement-learning#alphago) 132 * [AlphaZero](reinforcement-learning#alphazero) 133 * [AlphaStar](reinforcement-learning#alphastar) 134 135 [Programming reference](programming-reference)