1. Understand the underlying theoretical concepts of modern deep learning methods.

• 1.1 Describe the fundamental principles of deep learning.

• 1.2 Explain the differences between supervised and unsupervised learning. 

• 1.3 Critically analyse the role of gradient-based optimization in deep learning.

• 1.4 Evaluate the impact of network depth on model performance.

• 1.5 Discuss the theory of Variational Autoencoders (VAEs), Generative Adversarial Networks (GANs), and introduce Diffusion Models.

2. Be able to compare, characterise and quantitatively evaluate various deep learning approaches. 

• 2.1 Describe and compare different architectures used in deep learning.

• 2.2 Critique the performance of CNNs in image analysis tasks.

• 2.3 Analyse the effectiveness of RNNs for sequential data processing.

• 2.4 Critically evaluate the performance of generative models in various tasks, including GANs and Diffusion Models.

• 2.5 Discuss transformer-based approaches to Large Language Models (LLMs).

3. Understand the limitations of deep learning.

• 3.1 Identify common challenges faced in deep learning.

• 3.2 Explain the concept of overfitting and strategies to mitigate it.

• 3.3 Analyse the ethical considerations associated with deep learning.

• 3.4 Critically evaluate the robustness and generalisation ability of deep learning models.

4. Be able to apply deep learning techniques to real-world problems in computer vision, speech, text analysis, and graph processing.

• 4.1 Implement CNNs for image classification tasks.

• 4.2 Apply RNNs to speech recognition and machine translation tasks.

• 4.3 Develop generative models for synthetic data generation.

• 4.4 Implement graph neural networks (GNNs) as a generalisation of CNN theory for graph-based data analysis.

Introduction to Deep Learning

Course Coverage:

• Overview of Deep Learning
  • Definition and Scope of Deep Learning
  • Historical Context and Evolution of Neural Networks
  • Biological Basis of Neural Networks: Non-linear activations and spiking neurons
  • Importance of Deep Learning in Modern AI.

Learning Paradigms in AI

Course Coverage:

• Supervised Learning

  • Basic Concepts and Examples

  • Applications in Classification and Regression

• Unsupervised Learning

  • Fundamental Techniques (Clustering, Dimensionality Reduction)

  • Use Cases and Challenges

• Generalization and Overfitting

  • Definitions and Differences

  • Implications for Model Training

Perceptrons and Neural Networks

Course Coverage:

• Perceptrons
  • Single-Layer Perceptrons: Structure and Function
  • Limitations and the XOR Problem
• Multi-Layer Perceptrons (MLP)
  • Deep vs. Shallow Networks
  • Advantages of Deep Architectures

Gradient-Based Optimization in Deep Learning

Course Coverage:

• Stochastic Gradient Descent (SGD)

  • Concept and Mathematical Foundation

  • Variants of SGD (Momentum, Nesterov)

• Backpropagation

  • Derivation and Role in Neural Network Training

  • Practical Implementation of Backpropagation

Introduction to Generative Models

Course Coverage:

• Theory of Variational Autoencoders (VAE)

  • Mathematical Principles of VAEs

  • Applications of VAEs in Data Compression and Generation

• Diffusion Models

  • Overview of Diffusion Models and Their Role in Generative Modeling

Comparison of Deep Learning Architectures

Course Coverage:

• Perceptrons and Neural Networks:

  • Single-Layer vs. Multi-Layer Perceptrons

  • Non-Linear Activation Functions

• Convolutional Neural Networks (CNNs)

  • Mathematical Foundations

  • Convolution Operations

  • Pooling Layers: Max and Average Pooling

• CNN Architectures

  • Classic Architectures: LeNet, AlexNet, VGG

  • Advanced Architectures: ResNet, Inception

• Recurrent Neural Networks (RNNs)

  • Understanding RNNs

    • Sequential Data Processing

    • Backpropagation Through Time (BPTT)

  • RNN Variants

    • Long Short-Term Memory (LSTM)

    • Gated Recurrent Units (GRU)

  • Applications of RNNs

    • Speech Recognition

    • Machine Translation.

• Generative Models

  •  Introduction to Generative Data Augmentation Techniques

    • Types: VAEs, GANs

    • Mathematical Principles of Generative Models

  • Generative Adversarial Networks (GANs)

    • Structure and Functioning of GANs

    • Challenges in Training GANs (Mode Collapse, Convergence Issues)

  • Applications of Generative Models

    • Image Generation and Enhancement

    • Data Augmentation Techniques

  • Incorporate Diffusion Models into Generative Models

Performance Evaluation of Deep Learning Models

• Performance Metrics

  • Accuracy and Loss Metrics

    • Cross-Entropy, Mean Squared

    • Error

  • Evaluation Techniques

    • Confusion Matrix, Precision, Recall, F1-Score

    • ROC Curves and AUC

• Case Studies in Deep Learning

  • Image Analysis

    • Performance of CNNs in Image Classification

    • Object Detection and Semantic Segmentation

  • Sequential Data Processing

    • RNNs in Time Series Forecasting

    • LSTMs in Language Modeling

  • Generative Networks

    • VAEs vs. GANs in Image Generation

    • Applications in Art and Content Creation

• Introduction to Transformer Models
  • Image Transformers in LLMs
  • Attention Mechanisms and Their Role in Natural Language Processing

Challenges in Deep Learning

• Data Requirements
  • Large Datasets
    • The Need for Big Data in Training Deep Models
    • Challenges in Data Collection and Labeling
  • Imbalanced Data
    • Techniques to Handle Imbalanced Datasets
    • Synthetic Data Generation (SMOTE, ADASYN)
• Computational Costs
  • Hardware Requirements
    • Role of GPUs and TPUs in Deep Learning
    • Impact of Computational Power on Model Training
  • Optimization Techniques
    • Techniques to Reduce Computational Load
    • Distributed Training and Model Parallelism
• Interpretability and Transparency
  • The Black Box Problem
    • Lack of Transparency in Deep Models
    • Importance of Model Interpretability
  • Techniques for Interpretability
    • Saliency Maps, Grad-CAM, LIME
    • Explainable AI (XAI) Approaches

Overfitting and Generalization in Deep Learning

• Overfitting in Deep Models
  • Causes of Overfitting
    • High Model Complexity
    • Insufficient Training Data
  • Strategies to Mitigate Overfitting
    • Regularization Techniques: L1, L2, Dropout
    • Early Stopping and Cross-Validation
• Generalization Challenges
  • Transfer Learning
    • Concepts and Benefits
    • Pre-Trained Models and Fine-Tuning
  • Adversarial Examples
    • Impact on Model Robustness
    • Techniques to Improve Robustness
• Ethical Considerations in Deep Learning
  • Bias in AI Models
    • Sources and Implications of Bias
    • Strategies to Mitigate Bias
  • Privacy Concerns
    • Data Privacy in AI Applications
    • Techniques to Ensure Data Security
  • Fairness and Accountability
    • Ensuring Ethical AI Practices
    • Case Studies in Ethical AI Implementation

Computer Vision with Deep Learning

• Introduction to Attention Mechanisms
• Convolutional Neural Networks (CNNs) Transfer Learning in CNNs

Speech and Language Processing with RNNs

• Applications in Real-Time Translation Systems

Generative Models for Synthetic Data Creation

• Diffusion Models for Generative Data Augmentation

Graph Neural Networks (GNNs)

• Introduce GNNs as a Generalization of CNN Theory

• Applications in Medicine and Biology