Resources

Courses

	Attention, Transformers, LLMs [Topics] [Practical] Topics Attention: Applications (2014-2017) [Notebook] Transformers (2017-2022) [Notebook] LLMs (2020-2023) [Notebook] Practical Tutorial on Attention is all you need (Vaswani et al. 2017).
	Advanced Convolutional Neural Network Architectures (2012 - 2018) [Topics] [Practical] Topics [ Slides/Notebook ] History of vision architectures (1998-2012) Network in Network (2014) InceptionNet (2014) ResNet (2015) Pre-activations: Improved ResNet (2016) DenseNet (2016) WaveNet (2016) Depthwise Separable Convolutions (2017) Squeeze-and-Excite: ResNet (2017) Neural Architecture Search (2018 - Present) Practical [ Notebook (DIY) ] [ Notebook (Soln.) ] Task: Build an image classifier Dataset: CIFAR-10 Libraries: PyTorch, torchvision Learning objectives: Build and train InceptionNet on CIFAR-10 Build and train DenseNet on CIFAR-10
	Network Pruning [Topics] [Practical] Topics [ Slides/Notebook ] Motivation Network Pruning: Pipeline Network Pruning: Unstructured vs Structured Network Pruning: Criterion Network Pruning: Prune Rate Network Pruning: Lottery Ticket Hypothesis Knowledge Distillation Quantization Practical [ Notebook (DIY) ] [ Notebook (Soln.) ] Task: Build a land use classifier Dataset: EuroSat (RGB) Libraries: PyTorch, torchvision Learning objectives: Build and train a simple classifier Use different techniques of pruning available in PyTorch to reduce the size of the network Observe the effect of pruning on model accuracy
	Deep Autoencoders and Variational Autoencoders [Topics] [Practical] Topics [ Slides/Notebook ] Autoencoders: Motivation & History Autoencoders: Loss function Undercomplete Autoencoders Overcomplete Autoencoders Stacked / Deep Autoencoders Variational Autoencoders Applications of Autoencoders Practical [ Notebook (DIY) ] [ Notebook (Soln.) ] Task: Explore the use of autoencoders for Data compression Data generation Data interpolation Dataset: MNIST Libraries: PyTorch, torchvision Learning objectives: Build and train the following types of autoencoders Sparse autoencoder with L1 penalty Sparse autoencoder with KL penalty Contractive Autoenoder Variational Autoenoder Explore the quality of their learned latent space
	Noise Reduction in Machine Learning [Topics] [Practical] Topics [ Slides/Notebook ] Sources of noise Impact of noise Examples of noise Noise reduction techniques Rolling Window Convolution Digital Filters Machine Learning Fitlers Denoising Autoencoders Practical [ Notebook (DIY) ] [ Notebook (Soln.) ] Task: Denoise corrupted images Dataset: Fashion MNIST Libraries: PyTorch, torchvision Learning objectives: Build a simple Denoising Autoencoder (DAE) for images Train such a DAE on images with a given corrupted noise Visualize the quality of reconstruction