文档介绍：Artificial Neural Networks
Simulate the biological neural system
The brain consists of neurons linked together
An artconverge for linearly separable classification problems.
If the problem is not linearly separable, the algorithm may not converge
14
Symbol
%Change Aug-Sept
Returns Sept.
Returns Oct.
ABC
34
-9
6
XYZ
-56
4
-11
PQ
20
-34
-8
ST
47
15
18
Application: Stock Prediction
15
features to use
labels
U
U
D
D
The network contains several layers
Intermediary layers: hidden layers
Nodes in hidden layers: hidden nodes
Feed Forward ANN: nodes in one layer are connected to nodes in the next layer only
Recurrent ANN: nodes additionally connect to nodes in same layer or previous layers
Multilayer ANN
16
Nodes may use activation functions other than the sign function
Multilayer ANN
17
XOR Example
18
Goal: find set of weights w that minimizes the error
is a function of w
Output of ANN (y) is nonlinear => difficult to optimize
Greedy algorithms:
Gradient descent efficient solution
Weight update formula dependent on algorithm
Model Learning
19
Determine the structure of the network
Number of nodes in the input layer:
one input node for each attribute
transform categorical into binary: one input node per value
Number of nodes in the output layer
1 node for a two class problem
k nodes for a k-class problem
The network topology: number of hidden layers, hidden nodes, links
Initialize the weights and bias parameters, usually at random
Training example with missing values should be removed or estimated
Design Issues
20
Multilayer neural networks with at least one hidden layer are universal approximators:
Can approximate any function
May suffer from overfitting
Can handle redundant features
Sensitive to noise
Training is time consuming
Classifying a test example is fast
Hard to interpret
Characteristics
21
Both decision boundaries classify all training points correctly
Which decision boundary is better?
Which