LaTeX
General
Contributed by MJ Booysen
Overleaf is a great platform to use when writing reports in LaTeX. It is very user friendly, there are no package dependency issues, and greatly helps with collaboration (especially between student and supervisor). Here is a crash course on using Overleaf - Video Playlist.
For Stellenbosch University, here is a template that can be used for Skripsies, Theses and Dissertations: Overleaf Template.
Machine Learning
Intro to Machine Learning
Contributed by James G. de Villiers
Machine learning modules
Two well known Python modules for machine learning are Tensorflow and PyTorch. Both have unique advantages and disadvantages with numerous learning resources, but I feel that Tensorflow's is more complete and the implementation of the module more intuitive. Tensorflow also has the added advantage of being linked to Keras with detailed descriptions of all functions within the module.
Tensorflow setup
Setting up Tensorflow correctly is vital or the module will not function correctly. A very good step by step tutorial on installing Tensorflow, installing PyCharm and setting up virtual environments is presented by a YouTube channel called Tech With Tim. Specifically episode 1 of his Python Machine Learning Tutorials playlist discusses the installation and setup of Tensorflow.
Installation steps can also be found on Tensorflow's website, but the Tech With Tim video is crucial as it also shows how to configure your IDE (I suggest using PyCharm or VSCode) for Tensorflow.
Note: You can also use Jupyter Notebooks which integrate very well with Google Colab to run them in the cloud.
Tensorflow tutorials
Tensorflow's website contains very good introductory tutorials on the use of the module for machine learning (here). They do however expect a certain degree of prior knowledge surrounding Python programming from the reader, which is why I preferred the Tech with Tim videos as an introductory course. The Tensorflow tutorials are very good for quick revision. Keras' website also has tutorials, some of which differ from the Tensorflow tutorials. The best is to use both and decide which website's methods work best for yourself.
PyTorch setup
The installation and setup of PyTorch is less involved than Tensorflow's process. The PyTorch installation steps can be found here.
PyTorch tutorials
Learning to use PyTorch is made every easy by another YouTube channel named Sentdex. I however preferred to use his website (https://pythonprogramming.net/), specifically his Deep Learning and Neural Networks with Python and Pytorch course. This course is provided with detailed discussions surrounding the various functions used to create neural networks. I would suggest completing this course even if you are going to use Tensorflow, purely because his course explains the fundamentals very well.
Computer vision tutorials
The Tensorflow, Keras and PyTorch tutorials all contain exercises on classifying MNIST images with neural networks. The first tutorials usually involve the use of fully connected layers, but in later tutorials the same objective is achieved with convolutional layers. I suggest looking into the functions of the modules to better understand what happens within them. With PyCharm and Tensor- flow you can ctrl+click any function to view it. Also with Tensorflow play with both the model.fit and model.train on batch functions. If you are ever unsure on how to implement a function simply Google tensorflow *function*. The Tensorflow and Keras websites provide thorough explanations and implementation examples for all functions.
Computervision using convolutional layers can be learnt from the Tensorflow website. These tutorials teach you how to use convolutional layers for tasks such as image classification, segmentation and object detection.
Generative neural networks
Ensure that you have a thorough understanding of the use of deep neural net- works for image classification before moving on to image generation. Topics you should ensure you understand thoroughly are loss functions, normalisation layers, activation functions and back propagation. This will allow you to easily bridge the gap between image classification and generation. Furthermore, if you understand how the various functions are implemented behind the scenes in Tensorflow you'll be able to bend them to your will and in many instances simplify your network's setup procedures.
Two common generative neural networks are variational auto encoders and generative adversarial networks. For tutorials on auto encoders visit this as well as this. I found the Keras Blog's tutorials on auto encoders and specifically variational auto encoders (VAEs) more understandable. But I would study both methods of implemen- tation. After completing both Tensorflow and Keras' tutorials on VAEs you can attempt to create one with Tensorflow probabilistic layers. This greatly simplifies the setup process, but does various steps for you, mean- ing you won't develop an understanding on the function of VAEs from this tutorial. if you are still unsure about the inner workings of the VAE visit this for a detailed discussion on the mathematics and reasoning behind a VAE's setup.
Generative adversarial networks (GANs) outperform auto encoders on image generation tasks, but are notoriously difficult to train. Before attempting tutorials on GANs I would suggest first reading the original GAN paper, Generative Adversarial Nets. After this you can begin with the Hands On Generative Adversarial Networks with Keras textbook from Packt Publishing. They have a similar textbook for PyTorch. This textbook contains a step-by-step discussion and example of the implementation of the various types of GAN's developed over the past few years. Examples on how to set up various types of GANs can also be found on Tensorflow's website under Generative. I would however suggest starting with the Packt textbook.
Further reading on improved training techniques for GANs is strongly advised. Papers including NIPS 2016 Tutorial: Generative Adversarial Nets, and Improved techniques for training GANs, are excellent sources of information surrounding GAN training.
Note that deconvolutional layers lead to checkerboard artifacts in the output images to solve this problem use a combination of upsampling and convolutional layers as explained here.
Machine learning help
I found that working through multiple tutorials on the same topic greatly improved my understanding on the network's working behind the scenes. Machine Learning Mastery by Jason Brownlee is the website I visited most often when faced with a problem. He pro- vides excellent explanations on a wide range of topics. I preferred Googling the topic I was struggling with followed by his website's name (eg. DCGANs machine learning mastery). You will then be presented with numerous blog articles on the topic. It is best to start with his “A Gentle Introduction to ...” blog posts and from there progress to the “How to Develop ...” blog posts. Sentdex presents a course on setting up neural networks without the assistance of modules like Tensorflow and PyTorch. If you are still unsure about the details surrounding the training of neural networks, especially how the weights and biases are updated, I would suggest working through his Neural Networks from Scratch in Python course.