Background
At the end of the last millennium, I graduated with an MSc in Computer Science at ETH Zurich. Now, about 20 years later, I have decided to leave my comfort zone and return to ETH Zurich. My aim was to strengthen my knowledge in those subjects that did not yet exist in my Master’s degree. Nowadays this means Machine Learning, Data Mining, Deep Learning and Big Data. In these fields I have attended during 3 semesters subjects of the Master’s program.
Misconceptions Regarding Machine Learning
Before I started Machine Learning, it was clear to me: this is about neural networks. But I was completely wrong. Like a lot of people who discuss my studies with me. Everyone immediately asks me if I am an expert on neural networks, especially deep neural networks. Interestingly, this is not the case. During one semester each of Machine Learning and Advanced Machine Learning, neural networks were only a marginal topic. Of course, the Perceptron was discussed as a basis. And there were also some lessons about Backpropagation, Autoencoders, Convolutional Neural Networks and about Generative Adversarial Networks. But that was only a small part of the lecture. Machine Learning cannot be reduced to neural networks.
What Machine Learning Really Is
On the whole, it’s pure statistics.
It must be said, however, that statistics have changed massively in recent decades. In interaction, statistics and Machine Learning have pushed each other and both developed rapidly. The fact that more and more resources are available (data, computational power, …) has supported this development.
Machine Learning is basically either regression or classification. Other typical learning problems include density estimation, dimension reduction and data compression.
Generally speaking, Machine Learning always follows the same pattern:
- You choose a suitable model (either on the basis of available data or on the basis of existing knowledge).
- You extract So-called features from the data sets.
- You train the model based on the features of the training data.
- You validate the model.
- Finally, you test the model with features from data not yet used.
Many people automatically think of a neural network for the first step above, the model. In fact, this is of course a possible option. But if you want to solve everything with a neural network, you will probably miss better approaches. And by “better” I mean both, more efficient calculation as well as a higher quality of the result. Neural networks have great advantages, I might report about that in a later blog post. But they also have disadvantages. Building and training a neural network is an art that requires a lot of experience and even more test data.
If you are interested in Machine Learning, I recommend that you delve into the following topics:
- Feature Extraction, there is no way around Principal Component Analysis (PCA).
- Model Validation and Selection techniques like Cross Validation (CV, you can’t do Machine Learning without it) and Bootstrap.
- Regression in general, particularly Linear Regression, Ridge Regression and LASSO.
- Classification algorithms like Linear Discriminant Analysis (LDA), K Nearest Neighbors (KNN), Support Vector Machines (SVM, the ultimate workhorse among the classification algorithms), Random Forests and last but not least AdaBoost.
I hope that in this short blog post I could show that Machine Learning is more than neural networks. And if you are interested in this topic, the article provides some key ideas and algorithms. Machine Learning already determines our life today – and it is becoming more and more important. So I advise all of you to leave your comfort zone, as I did, and try to understand what’s behind it as well as the incredible potential of the different algorithms. One or the other topic I will probably take up and deepen in a following blog post.