Course Outline
DAY 1 - ARTIFICIAL NEURAL NETWORKS
Introduction and ANN Structure.
- Biological neurons versus artificial neurons.
- Modeling an Artificial Neural Network (ANN).
- Activation functions utilized in ANNs.
- Common classes of network architectures.
Mathematical Foundations and Learning Mechanisms.
- Reviewing vector and matrix algebra.
- State-space concepts.
- Optimization concepts.
- Error-correction learning.
- Memory-based learning.
- Hebbian learning.
- Competitive learning.
Single Layer Perceptrons.
- Perceptron structure and learning processes.
- Introduction to pattern classifiers and Bayes' classifiers.
- Utilizing the perceptron as a pattern classifier.
- Perceptron convergence.
- Limitations of perceptrons.
Feedforward Artificial Neural Networks.
- Structures of multi-layer feedforward networks.
- The backpropagation algorithm.
- Training and convergence in backpropagation.
- Functional approximation using backpropagation.
- Practical and design considerations for backpropagation learning.
Radial Basis Function (RBF) Networks.
- Pattern separability and interpolation.
- Regularization Theory.
- The relationship between regularization and RBF networks.
- Design and training of RBF networks.
- Approximation capabilities of RBF networks.
Competitive Learning and Self-Organizing Artificial Neural Networks.
- General clustering procedures.
- Learning Vector Quantization (LVQ).
- Competitive learning algorithms and architectures.
- Self-organizing feature maps.
- Characteristics of feature maps.
Fuzzy Neural Networks.
- Neuro-fuzzy systems.
- Background on fuzzy sets and logic.
- Design of fuzzy systems.
- Design of fuzzy Artificial Neural Networks.
Applications
- Discussion of several Neural Network application examples, including their advantages and associated challenges.
DAY 2 - MACHINE LEARNING
- The PAC Learning Framework
- Guarantees for finite hypothesis sets – consistent case
- Guarantees for finite hypothesis sets – inconsistent case
- General considerations
- Deterministic vs. Stochastic scenarios
- Bayes error noise
- Estimation and approximation errors
- Model selection
- Rademacher Complexity and VC Dimension
- Bias-Variance tradeoff
- Regularization
- Overfitting
- Validation techniques
- Support Vector Machines
- Kriging (Gaussian Process regression)
- PCA and Kernel PCA
- Self-Organizing Maps (SOM)
- Kernel-induced vector spaces
- Mercer Kernels and Kernel-induced similarity metrics
- Reinforcement Learning
DAY 3 - DEEP LEARNING
This section will be taught in relation to the topics covered on Day 1 and Day 2
- Logistic and Softmax Regression
- Sparse Autoencoders
- Vectorization, PCA, and Whitening
- Self-Taught Learning
- Deep Networks
- Linear Decoders
- Convolution and Pooling
- Sparse Coding
- Independent Component Analysis
- Canonical Correlation Analysis
- Demos and Applications
Requirements
A solid grasp of mathematics.
A strong understanding of fundamental statistics.
Basic programming skills are not mandatory but are recommended.
Testimonials (2)
Working from first principles in a focused way, and moving to applying case studies within the same day
Maggie Webb - Department of Jobs, Regions, and Precincts
Course - Artificial Neural Networks, Machine Learning, Deep Thinking
It was very interactive and more relaxed and informal than expected. We covered lots of topics in the time and the trainer was always receptive to talking more in detail or more generally about the topics and how they were related. I feel the training has given me the tools to continue learning as opposed to it being a one off session where learning stops once you've finished which is very important given the scale and complexity of the topic.
