The Joy of Finally Learning - Artificial Neural Networks Overview

The Joy of Finally Learning - Artificial Neural Networks Overview
Slide Note
Embed
Share

An informative talk highlighting the core concepts of Artificial Neural Networks (ANNs), including Perceptrons, learning functions, and biological inspirations. Discover the practical applications and workings of ANNs in recognizing patterns like hand-written characters and spoken words. Dive into the world of function approximations and the interconnection of neurons, exploring the birth of error functions and the forward and backward passes in binary classifiers.

  • Neural Networks
  • Artificial Intelligence
  • Machine Learning
  • Perceptron
  • Function Approximation
jess_2
jess_2 Follow

Uploaded on Apr 28, 2025 | 0 Views

Download Presentation

Please find below an Image/Link to download the presentation.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.

You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.

Download Presentation

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.

E N D

Presentation Transcript

  1. The Joy of Finally Learning www.mldawn.com

  2. OUTLINE OF THE TALK What is an Artificial Neural Network (ANN)? What is a Perceptron? Bayes Rule and Birth of Error Functions (with Derivations) The Birth of Binary-Cross Entropy Function (with Derivations) Forward Pass and Backward Pass in a Binary Classifier A Demo

  3. WHAT IS A NEURAL NETWORK? An ANN is a function approximator! What is a function? Examples of functions: real-valued, discrete-valued, and vector-valued Applications of ANNs: Recognizing hand-written characters Recognizing spoken words Recognizing faces

  4. WHAT IS A NEURAL NETWORK? Unknown Function !!! ?( ) ? ( ) ??? ? ?(?) Image Cat / Dog (Prob. Vector) ? ( ) ?( ) Stock Information Price ($) (real-valued) Car Information Number (discrete-valued) ANN approximates ?( ) for us

  5. ANN BIOLOGICAL MOTIVATION ANNs are inspired by Biological Learning Systems (BLS) BLS: A massive interconnection of neurons ANNs: Densely interconnected set of simple unites

  6. ANN BIOLOGICAL MOTIVATION For each unit in ANN: Possibly several inputs A Single real-valued output BLS processes are highly parallelized Over nearly 1011 neurons ANNs do NOT fully reflect a BLS Output of neurons are different Researchers try to: 1. Model BLS processes 2. Develop effective ANNs (no regards to BLS compatibility)

  7. SIMPLEST ANN A PERCEPTRON

  8. A PERCEPTRON A type of ANN that is based on a unit called perceptron There is a rule about input/output of a perceptron: Input: A vector of real-valued data Linearly combine the components of the input ?? ?0+ ?1?1+ ?2?2+ + ????> 0 ?? ?????? ? ?1, ,?? = +1 1

  9. A PERCEPTRON What are the weights? What is that ?0? Let s imagine an additional input ?0= 1 This simplifies the inequality: ?=0 ????> 0 Or in vector form: ?. ? > 0 ? Can we show the perceptron smarter? Learning a perceptron = Choosing the weights ? ? = sgn(?. ?) Where: ??? ? = +1 If z > 0 The hypothesis space: ? = ? ? ?(?+1)} 1 otherwise

  10. A PERCEPTRON: A DEEPER LOOK ?0= 1 ?0 ? sgn(?) ?1 ? ?1 sgn(?) ? ? ?1 ?1 ?1

  11. A PERCEPTRON: A DEEPER LOOK ?0= 1 ?0 ?1 ?1 ? sgn(?) ?2 ? ?2 ? ? sgn(?) ?1 ?1 ?1 ?2 ?2 ?2

  12. THE BIRTH OF ERROR FUNCTIONS DO I TRUST MY ANN?

  13. BAYES RULE Let s assume the following notation: M : Our Model D: Our Evidence Then the Bayes Rule: ? ?|? =? ?|? ? ? ? ?

  14. TAKING THE LOGS() : A SIMPLIFICATION ? ? ??? = ??? ? ???(?) ? ?|? =? ?|? ? ? ln? ? ? = ln ? ?|? ? ? ln?(?) ? ? ??? ? ? = ??? ? + ???(?) ??? ? ? = ?? ? ?|? + ?? ?(?) ???(?)

  15. THE BAYESIAN MACHINERY (MAPE) ??? ? ? = ?? ? ?|? + ?? ?(?) ???(?) ???????? ? ? ? ???????? ?(?|?) ??? ? ? = ?? ? ?|? ?? ?(?)+???(?) ???? ?? ??? ????? ???? ???????? ?? ????? ????????? ? = ??? ? ?

  16. MAPE AND ANNS : SUPERVISED LEARNING How can we apply the MAPE to an ANN with weights w? Let s assume that we have a set of examples (di,ti) where di is the i-th input and ti the corresponding target. Then we can rewrite the likelihood as: ? ? ? ??? ? (?? ,??) ? ? ? = ? (??,??) ? ? ? & t ANN ? (??,??) ? = P(ti|di,?) ?(??|?) ?(??)

  17. MAPE AND ANNS : SUPERVISED LEARNING ??? ? ? = ?? ? ?|? ?? ?(?)+???(?) ????? ?????? ????? ??? ??????? ? ? ? ??? ? ?? ??? ???? ??????????? ?? ??? ??????? ?.?????? ???? ?????? ???????????? ??? ? ? = ?? ? ?|? ?? ?(?)+???(?) P(ti|di,W) ?(??) P tidi,W ? ?? ln? ? + ln?(?) ? ??? ? ? = ln ?=1 ln? ? + ln?(?) ? ??? ? ? = ?=1 ln P tidi,W ? ?? ? ? ??? ? ? = ln P tidi,W ln? ?? ln? ? + ln?(?) ?=1 ?=1

  18. MAPE AND ANNS : SUPERVISED LEARNING ? ??? ?? ???? ??? ???????????? ? ?? ????? ??????????? ??? ? ? = ln P tidi,W ln? ? ?=1 ln( ) (? ?)2 2?2 1 ? ? = 2??? 2+ ?1 2+ + ?? 2) (?1 ? ? = ??? ? ? = ln P tidi,W ?=1

  19. BINARY CROSS-ENTROPY ERROR FUNCTION THE DERIVATION

  20. BINARY CLASSIFICATION We have 2 classes: Positive and Negative Binary Classification: ??= 1 means that ?? belongs to the positive class ??= 0 means that ?? belongs to the negative class Artificial Neural Networks (ANNs): Probabilistic approach ANN computes the probability of the Positive Class: ? ??= 1 ??) Could a linear model work? Say, ? = ??? + ? Not Bounded! We need a Squasher!

  21. BINARY CLASSIFICATION ?0= 1 ?0 Input Output [0, 1] ??= 1 ??= 0 ? 1 ?1 ? = ? ? = ?1 1 + ? ? ??????? ... ?? ?? Compute Error Back- Propagation

  22. BINARY CLASSIFICATION The output of our ANN is probability. Let s denote: ??: Probability ?? belongs to the Positive class (i.e., ??= 1) (1 ??): Probability ?? belongs to the Negative class (i.e., ??= 0) ? ????,? = ?? ?? ??= 1 ?? ??= 0 1 ?? Bernoulli Distribution ??(1 ??)1 ?? ? ????,? = ??

  23. BINARY CLASSIFICATION ??(1 ??)1 ?? ? ????,? = ?? ? ? ??(1 ??)1 ??] ? = ln P tidi,W = ln[?? ?=1 ?=1 ? ??+ ln 1 ??] = [ln?? ?=1 ? ? = [??????+ ? ????(? ??)] ?=?

  24. FORWARD PASS ?0= 1 ?0 ? 1 ?1 ? = ? ln? 1 ? ln(1 ?) ? = ? ? = ?1 1 + ? ? ??????? ????? = ? ?(?) = ?(?(?1?1+ ?0)) ? ? Binary Cross-Entropy Error Binary Cross-Entropy Error Function Function

  25. BACKWARD PASS ?? ??1 =?? ?? ?? ?? ??1 ? ?+(1 ?) (1 ?) ?? = ? 1 ? ?1= (? ?) ?1 ?? ??1 =?? ?? ?? ?? ??0 ? ?+(1 ?) (1 ?) ?? = ? 1 ? ?0= (? ?) So the learning rule for both our learnable parameters (i.e., weights) is as follows: ?1= ?1 ? ?? = ?1 ? [(? ?) ?1] ??1 ?0= ?0 ? ?? = ?0 ? (? ?) ??0

  26. www.mldawn.com @MLDawn2018 Mehran Bazargani The Joy of Finally Learning ML Dawn Course Available at: https://www.mldawn.com/course/the-birth-of-error-functions/ Demo s Code: https://www.mldawn.com/binary-classification-from-scratch-using-numpy/

Related


Introduction to Deep Learning: Neural Networks and Multilayer Perceptrons

Introduction to Deep Learning: Neural Networks and Multilayer Perceptrons

ellie
Graph Neural Networks

Graph Neural Networks

kornelia
Artificial Neural Networks From Scratch

Artificial Neural Networks From Scratch

inara
Back-Propagation Algorithm in Neural Networks

Back-Propagation Algorithm in Neural Networks

kamilah
Assistive Speech System for Individuals with Speech Impediments Using Neural Networks

Assistive Speech System for Individuals with Speech Impediments Using Neural Networks

albi
Neural Quantum States and Symmetries in Quantum Mechanics

Neural Quantum States and Symmetries in Quantum Mechanics

adelinda
Spiking Neurons and Spiking Neural Networks

Spiking Neurons and Spiking Neural Networks

kooper
Introduction to Neural Networks in IBM SPSS Modeler 14.2

Introduction to Neural Networks in IBM SPSS Modeler 14.2

anouk
Advanced Classifiers and Neural Networks

Advanced Classifiers and Neural Networks

elwynto
Artificial Neural Networks (ANN) and Perceptron in Machine Learning

Artificial Neural Networks (ANN) and Perceptron in Machine Learning

ubal_5
Machine Learning and Artificial Neural Networks for Face Verification: Overview and Applications

Machine Learning and Artificial Neural Networks for Face Verification: Overview and Applications

lipp195
Neural Network Watermarking Technologies

Neural Network Watermarking Technologies

alad
Deep Generative Bayesian Networks in Machine Learning

Deep Generative Bayesian Networks in Machine Learning

alr_bo
Adversarial Examples in Neural Networks

Adversarial Examples in Neural Networks

penrodo
Perceptron: Simplest Neural Network for Linear Pattern Classification

Perceptron: Simplest Neural Network for Linear Pattern Classification

klov_31
Introduction to Deep Learning: Concepts & Applications

Introduction to Deep Learning: Concepts & Applications

gmal
Shallow Artificial Neural Networks for Forecasting

Shallow Artificial Neural Networks for Forecasting

mgad
Spatial Resolution Optimization using Neural Networks

Spatial Resolution Optimization using Neural Networks

tdil
Neural Networks for Handwritten Digit Recognition

Neural Networks for Handwritten Digit Recognition

coventry_n
Unveiling the Power of Deep Learning in Computational Physics

Unveiling the Power of Deep Learning in Computational Physics

mais_8
Neural Networks: Theory, Architecture, and Applications

Neural Networks: Theory, Architecture, and Applications

akutn
Unveiling the Potential of Randomly Wired Networks in Neural Architecture Search

Unveiling the Potential of Randomly Wired Networks in Neural Architecture Search

har_s

More Related Content