Author: Dhruv Shrivastava
In the field of mathematic modelling, a radial basis function network is an artificial neural network that uses radial basis functions as activation functions. Radial basis function neural network is distinguished from other neural networks due to their universal approximation and faster learning speed. Radial basis functions networks have many uses, including function approximation, time series prediction, classification, and system control.
Introduction:
Radial basis functions network was formulated by Broomhead and Lowe in 1988. Radial basis function (RBFs) has only one hidden layer and the convergence of optimization is much faster than other neural networks, and despite having one hidden layer. RBF network in its simplest form is a three-layer feedforward neural network consisting of first layer which corresponds to the inputs of the network, the second is hidden layer and the third layer is final output of the network.
This paper provides a detailed introduction to Radial Basis Function along with the mathematical and algorithmic development, then a comparison between RBF and MLP (multi-layer perceptron). We will also train RBF network to predict the customer’s response to subscription for term deposit.
Architecture of RBF:
Radial basis function (RBF) networks typically have three layers: an input layer, a hidden layer consists of hidden neurons, and activation function of these neurons is a Gaussian function. Hidden layer generates a signal which corresponds to an input vector in the input layer and corresponding to this signal the network generates a response output.
Design and Development:
To generate an output, neuron process the input signal through a function called activation function. In RBF activation function of hidden neuron is φ(X) i.e., for an input vector X output produced by the hidden neuron will be φ(X).
Fig-1.
Above figure represents a Gaussian neural activation for 1-D input x and with mean . Here φ(x) represents output of gaussian node given value of x.From above figure we can clearly see that the signal strength decreases because the input (X) moves far away from the middle.
The basic idea of this model is that the whole feature vector space is partitioned by Gaussian neural nodes, where each node generates a sign like an input vector, and strength of the signal produced by each neuron depends on the space between its center and therefore the input vector. Also, for inputs lying closer in Euclidian vector space, the output signals that are generated must be similar.
Here, is center of the neuron and X is response of the neuron corresponding to input X. In RBF architecture, weights connecting input vector to hidden neurons represents the center of the corresponding neuron. These weights are predetermined such that the entire space is covered by the receptive field of these neurons, whereas values of weights connecting hidden neurons to output neurons are determined to train network.
Fig-2.
It will good if vectors lying close in Euclidian space fall under the receptive area of same neuron therefore, we will determine the centre of hidden neuron using k-mean clustering.
K-means Clustering algorithm:
Select the number of cluster centres K.
Randomly select the k points in dataset and set them as K centroids of the data.
Determine the centroid closest to it for all data points.
Calculate the average of all points lying closest to the same centroid.
Change the value of all centroids to corresponding averages values.
Go to step 3 until convergence.
The range of receptive fields is chosen such that entire domain of input vector is covered by the receptive field of the neurons. So, the value of sigma is chosen according to maximum distance d between two hidden neurons.
Where d is the max distance and M is the number of hidden neurons.
Mathematical Development:
Let gij be an element of matrix G representing the output of j^th neuron for ithinput vector and Wij be an element of matrix W representing the weight connecting i^th output neuron to j^th hidden neuron. In Radial basis function, the activation function of output neuron is linear that is “g(z)=z” where z is the weighted summation of signals from hidden layer. Multiplying i^th row of G and j^th columns of W do the weighted summation of signals from hidden layer which is equal to signal produced by j^th output neuron.
GW =T
Where T is a column vector and ithrows contain the target value of ith training vector.
From above equation using pseudo inverse
where G^T is the transpose of matrix G.
Algorithm:
Define number of hidden neurons “K”.
Set the positions of RBF centres using K-means clustering algorithm.
Calculate using equation 2.
Calculate actions of RBF node using equation (1).
Train the output using equation (3).
RBF vs MLP:
MLPs (Multi Layered perceptron) are better than RBFs when the underlying characteristics feature of data is embedded deep inside very high dimensional sets for example, in image recognition etc.
RBFs have much faster convergence rate as compared to MLPs because RBFs have only one hidden layer.
RBF network is usually preferred over MLPs for low dimensional data where deep feature extraction is not required and results are directly correlated with the components of input vector.
Unlike most machine learning model RBF is a robust learning model.
Classification more time in RBF than MLP.
Fig-3: Classification in MLP vs RBF.
Implementation:
Predicting a client’s response from his/her characteristics details like age, job, education, marital status etc requires years of experience and learning. Here an RBF based AI implementation on bank marketing dataset is presented. We will try to predict whether or not a client will subscribe for a term deposit.
Classes: There is two “yes” and “no” corresponding to the response of client.
Step1: Import all necessary packages:
Step2: Read the data from the file and encode the label using LabelEncoder:
Step3: split the data into training and testing data:
Step4: Scale the data using StandardScaler:
Step5: Determine centres of the neurons using KMeans:
Step6: Determine
Step7: Set up matrix G:
Step8: Find weight matrix W to train the network:
Step9: Set up matrix G for the test data:
Step10: Analyse the accuracy of prediction on test set:
Conclusion:
With RBF we got a prediction accuracy pf 88% and so is the cause with MLP. But, the computational cost of training MLP is higher than RBF hence it is better to use RBF instead of MLP.
GitHub Link:
https://github.com/dhruv20032000/Radial-Basis-neural-network
References:
https://www.hackerearth.com/blog/developers/radial-basis-function-network/
https://towardsdatascience.com/radial-basis-functions-neural-networks-all-we-need-to-know-9a88cc053448
https://www.google.com/imgres?imgurl=https%3A%2F%2Fblog-c7ff.kxcdn.com%2Fblog%2Fwp-content%2Fuploads%2F2017%2F01%2FCapture-22.png&imgrefurl=https%3A%2F%2Fwww.hackerearth.com%2Fblog%2Fdevelopers%2Fradial-basis-function-network%2F&tbnid=Ec-EjF8cC2jANM&vet=12ahUKEwjR0b22nOfxAhW_s0sFHahBAvcQMygoegUIARCFAg..i&docid=K3wToKS1VWbIDM&w=511&h=306&q=radial%20basis%20functions%20neural%20networks&ved=2ahUKEwjR0b22nOfxAhW_s0sFHahBAvcQMygoegUIARCFAg
https://www.google.com/url?sa=i&url=https%3A%2F%2Fwww.researchgate.net%2Ffigure%2FArchitecture-of-the-RBF-neural-network-RBF-radial-basis-function_fig2_333469185&psig=AOvVaw2woGDCHYtcwjfoe6ve_6MO&ust=1626512246097000&source=images&cd=vfe&ved=0CAsQjRxqFwoTCJDq4Oac5_ECFQAAAAAdAAAAABAI
https://ieeexplore.ieee.org/abstract/document/1215400
https://www.google.com/url?sa=i&url=https%3A%2F%2Fslideplayer.com%2Fslide%2F5867042%2F&psig=AOvVaw0yx0kkLpJ5dvSk9_de4wSw&ust=1626522606565000&source=images&cd=vfe&ved=0CAsQjRxqFwoTCNCLi4PD5_ECFQAAAAAdAAAAABAD
Commentaires