FPGA-Based Fully Parallel PCA-ANN for Spectrum Sensing

The cognitive radio system is proposed as an optimal way to improve the frequency underutilization. Spectrum sensing is the first and the essential function in this approach. A cognitive user must sense his environment to detect the unused channels, and then he can use the free channel without causing any interference to the primary user. In this article, an innovative technique is proposed for spectrum sensing based on principal component analysis and neural networks in frequency domain. The designed blocks are described using VHSIC Hardware Description Language (VHDL). The suggested application consists of extracting features from the captured signals by PCA; the classification is done by a Multi-Layer Perceptron (MLP). Neural network training part and principal components are done on MATLAB environment; while the hardware implementations are created on an FPGA DE2-70board.


Introduction
The frequency spectrum use is becoming increasingly demanded (Datla, Rajbanshi, Wyglinski, & Minden, 2009), as a result of the rising needs of wireless technologies and their various services.The current management of the frequency spectrum is a static allocation which cannot support these growing needs.Within this context, the approach of cognitive radio was introduced by Mitola to end those problems and increase the spectrum efficiency.In this way, the spectrum management will shift from the classic appearance to a dynamic use (Mitola & Maguire Jr, 1999) and (Zeng, Liang, Hoang, & Zhang, 2010).The dynamism given to this management is based on separating users into two categories: PU: Primary User; that has a license to benefit from a given spectral band.SU: Secondary User; that does not have a license, but can temporarily profit from a free band.
Thus under the condition, the PU should not suffer harmful external interference, and does not make any modifications to allow coexistence with the SU.
Several methods for spectrum sensing are suggested in the literature, such as Matched filter (Fuchs, 2009), Cyclostationary test (Ning, Sohn, & Kim, 2009), and Energy Detection (Yucek & Arslan, 2009); but the energy detection remains the most chosen, not only for its simple implementation, but since it does not need to know any information in advance about the PU.In this method, the received signal energy is measured, and compared to a predetermined threshold, which presents the noise power of present on the channel, if the energy exceeds the threshold, we declare the PU presence, otherwise it is absent.
The uncertainty of noise, shadowing and the channel fading are problems that decrease the performance of an energy detector (Tandra & Sahai, 2005).In our context, we propose a hybrid architecture for spectrum sensing, which consists of three blocks FFT, PCA, and ANN.MATLAB is used to calculate PCA eigenvectors, besides training the neural network, with the intention of calculating weights and bias.The hardware implementation illustrates the scheme of a numerical system with three stages achieving FFT block with 1024 points, a matrix multiplication block to change the signal basis by PCA, and a feed-forward multilayer neural network to make the decision; the hole process is realized in Altera Cyclone II FPGA integrated inside the DE2-70 Board (TechnologiesTerasic).
The second section presents the adopted methodology and the used materials, then the third section deals with the different parts of implementation.In the fourth section, the implementation results are discussed.Finally, the fifth section concludes the paper.

Mathematical Formulation of Spectrum Sensing
In this context, we shall detect the presence of the PU that emits at a given time a signal x(t).The problem of the spectrum sensing can be formulated as follows: y(t): The received signal.x(t): The signal transmitted currently being deterministic or random, but unknown.n(t): Noise and it is supposed to be additive white Gaussian.
The hypothesis H presents the case where the received signal y (t)contains the noise alone.While the hypothesis H , refers to the ability of detecting the PU presence.
The performance of the detector is characterized by using the pair (P , P ) (Atapattu, Tellambura, & Jiang, 2010).
P : probability of detection P : probability of false alarm.

Dimensionality Reduction by PCA
The Principal Component Analysis (PCA) is commonly used in data exploration.When we have a quantitative database, wherein 'n' observations (the captured signals) are described by 'm' variables (samples), where 'm' is considerably high.It is impossible to understand the data structure and the proximity between the observations by a simple analysis or even a correlation matrix.The PCA can be considered a projection method, which projects the observations from the N-dimensional space to a K-dimensional space ( << ) (Burges, 2010), such that the maximum of information is retained on the K-first dimensions.
The purpose of PCA is condensing the original data into a new group, so that they have no correlation between them, and are ordered in terms of the percentage of variance contributed by each component.Thus, the first principal component includes the maximum variance, while the second principal component covers following variance, and the process is repeated until the last principal component.In this way, the information loss decreases from one step to the next.

Artificial Neural Networks
Hardware neural network systems (called neuro-hardware) have been developed rapidly over the last decade.Unlike the Von Neumann architecture -Classic -this is sequential in nature, artificial neural networks (ANN) benefit from the massively parallel processing.A variety of materials are designed to exploit the parallelism inherent in the neural network models.Despite the power growth of the numerical calculation in general processors, the approach of neuro-hardware is considered promising in specialized applications such as image processing, voice synthesis, pattern recognition…

Formal Neuron
Studying artificial neural network was inspired mainly from the biological learning system (Basheer & Hajmeer, 2000); the biological model is composed of complex layers of interconnected neurons.In effect, the human brain is composed of approximately 10 11 neurons, each one have an average of 10 3 connections.It is believed that the considerable calculation power of the brain is a result of the parallel and distributed processing performed by these neurons.
The artificial neuron has generally multiple inputs and a single output.Actions of excitatory synapses are illustrated by the coefficients called synaptic weights; these weights are coupled with all inputs.The numerical values of these coefficients are adjusted in the learning phase.The second figure illustrates the artificial neuron design, the formal neuron receives n inputs presented by the vector X , X … X , these inputs are assigned by their weights indicated such as W , W … W , and the presents the bias activation.Equation (4) gives the output of this model.
The activation function has a crucial role in the formal neuron architecture.In this paper, a sigmoid function is utilized the since its nonlinearity makes it possible to approximate any function.

Multi-Layer Perceptron
A Multi-Layer Perceptron (MLP) consists of a number of artificial neurons interconnected, this network is organized in the form of layers (Huang, 2003), such as the layer ′ ′ receive as input the outputs of the previous layer ′ − 1′ and feed its outputs to the next layer ′ + 1′.This model is called a direct neural network or a feed forward neural network.The first and last layers are called, respectively, the input and output layers.Layers that are neither input nor output are known as hidden layers., ( ) : Is the desired output for the i th neuron, and , ( ) is the measured output for the same neuron

Proposed Design
The design adopted in this paper is guided by the principle to realize an efficient detector based on neural networks, which will be able to determine whether the band being in the test is free or occupied, not only with a satisfactory precision, but also with a minimal number of inputs to the model to save

Implementation Platform
The detection system is implemented on an FPGA device, as it offers a high flexibility.

FFT Implementation
As a first block of our proposed architecture, we use the FFT MegaCore function which is an IP Core for FPGA from Altera (FFT MegaCore Function User Guide, August 2014), this function is able to calculate both forward and inverse Fourier transform.This hardware architecture is chosen to optimize the usage of resources, and has the four following parameters: Burst mode, Single output engine, Single instance of engine, 16-bit internal and data input/output precision widths.

PCA Implementation
The aim of using PCA in this work is to make possible the hardware implementation of the neural network detector.In this work, each captured signal contains 1024 samples.If we use the signal vector directly as an input of the neural network, there would be 1024 input neurons, which would form a very large network in terms of neurons number in the input and hidden layers, also it would increase the computational complexity (registers, multipliers, adders).So the architecture of the detector would be high in terms of hardware cost.In this respect, PCA is used to reduce ANN complexity; but we have to take into account that the classification rate must be kept at a satisfactory level.
The PCA training part, which concerns the eigenvectors matrix calculation, was developed in MATLAB.The hardware implementation has been restricted to features extraction following these two equations ( 8) and ( 11).
: Is the signal vector, while is the vector of variable means, and then present the variance of variables.

= .
Such as = , … , (9) X : Is the reduced vector, and P is the new basis u , … , u of the K-largest eigenvalues.
As described in the figure below, the architecture of PCA consists in three blocks: the first stage receives the signal vector already converted in the frequency space, then it subtracts the vector of variable means from the signal vector X, the vector has been stored in a Read-Only Memory (ROM).The second block is used to devise the first block output by the variance given by the equation ( 8).The third and final stage was used to project the signal into the reduced space using the equation ( 9).

Architecture of the Neuron
The hardware implementation of the decisional phase will be restricted in the trained neural network; the training phase has been done in MATLAB environment.As specified above, this implementation has been developed by VHDL language, and the design has been created in Quartus II environment.Figure 8 explains the block diagram of a neuron, it is consisted of a multiplier block which multiplies the weights by all parallel inputs, then an accumulator which gives the addition result of all the multiplications plus the bias, and finally the score is taken from the output of the activation function.

Activation Function
As cited in the section (2.3.1) the activation function has a critical impact in the neuron design, in the Multi-Layer perceptron model the sigmoid function is the most used.The sigmoid function consists of an infinite exponential series, which makes it inappropriate for a direct implementation on FPGA.To bypass this problem, we use the Piece-Wise Linear Approximation (Campo, Finker, Echanobe, & Basterretxea, 2013) , in that approximation the 'logsig' function presented by the equation 10, is divided into five intervals to consider it as linear function in each interval, as shown in the equation 11. Figure 9 illustrates the approximated sigmoid hardware design; this architecture provides an insight about the complexity degree for a hardware implementation of such function.

PCA Results
The PCA training is realized using MATLAB environment, the following graph gives the percentage of the first 35 eigenvalues.From these results, we can see that the first 10 components with a percentage of eigenvalue superior than 2%, together explain 98% of the total data-base variance.The first principal component describes the greater percentage 12.1%, while the second one illustrates about 10.6% of the total variance.

Neural Network Training Results
Knowing that the PCA block extracts 10 features, so in the input layer we use also 10 neurons, and in the output layer we use one neuron to differentiate between H 0 and H 1 .Each additional neuron in the hidden layer is used to take into account specific profiles of the input neurons.A larger number makes it possible to better match the data presented, but decreases the capacity of generalization in the network.Here again there is no general rule, but empirical rules (Zhang, Patuwo, & Hu, 2001).The size of the hidden layer must be: -Equal to that of the input layer.
-Equal to 75% of the input layer.
-Equal to the square root of the product between the number of neurons in the input and output layer.
Should be noted that the last choice reduces the number of freedom degrees left to the network, then the ability to adapt on the training samples in favor of a greater stability / ability to generalize.
In this work, we adopt a statistical approach, it consist in trying different architectures with different numbers of neurons in the hidden layer.The classification rate is calculated for these architectures.The following table summarizes the results of this statistical study: Analyzing the results shown in the table above, it is obvious that the high classification rate (98%) in testing is achieved by using 10 neurons in the hidden layer.

Neural Network Implementation Results
The implemented network architecture consists in using 10 neurons at the input layer.Thereafter, 10 neurons are used in the hidden layer, while we exploit one neuron in the output layer to differentiate between the two hypotheses and .The following figure shows the RTL design of the full network circuit: Figure 11.The RTL description of the network circuit

Full Detector Implementation Results
The subsequent diagram explains the manner in which we connect the two designed blocks (PCA and ANN), with the FFT MegaCore IP Core.At the beginning of this architecture, we receive the collected samples.The treatment begins by transforming the signal into frequency domain using the FFT IP core.Next, the PCA block extracts the features, this block takes ( ) with 1024 parameters and sets free ( ) with only 10 elements.Finally, the reduced vector is presented to the ANN block to make the decision about the channel stat.The 4 th table summarizes the proposed design characteristics, including the utilization of different resources; we notice that the proposed architecture takes 46% from the total logic elements in the FPGA, while it spends 70% from the embedded 9-bit Multipliers.

Conclusions
In this work, a new method of spectrum sensing based on PCA and ANN is designed and implemented on an FPGA device.A fully parallel MLP network is trained with a back-propagation algorithm, and the detector error is at a suitable level (2%).The numerical precision of the proposed detector is reasonable, and the inaccuracy of the ANN is very low.The sigmoid activation function is the most essential block of the neuron; it was realized in hardware using the Piece-Wise Linear Approximation with floating-point representation, and the results show that the activation block consumes the major part of the hardware resources.The final proposed network architecture is modular, making it possible to change the number of neurons and layers in a simple manner.In our future work, we will explore ways to integrate the learning phase for both PCA and ANN in the same design.
To do this, we have to reduce the FPGA resource usage, while maintaining the performance of detector at an acceptable level.

Figure 2 .
Figure 2. The formal neuron architecture

Figure 3 .
Figure 3. MLP model the training time and minimize the area of implementation.The proposed design consists of three stages:  FFT block to transform captured signals from time to frequency domain. Dimensionality reduction with PCA.Decision is done by a MLP neural network with two layers in which every layer is completely linked with its closest layers.

Figure 5 .
Figure 5. Measurement platform VHSIC Hardware Description Language (VHDL) is used to synthesize the design, with Altera Quartus II Web Edition as the place and-route tool.The target device is the Altera DE2-70.The following figure gives more details about the target board.

Figure 8 .
Figure 8. Block diagram of a neuron

Figure 10 .
Figure 10.Percentage of the first 35 eigenvalues

Figure 12 .
Figure 12.Full detector block architecture

Table 1 .
Data-base used in learning and testing

Table 2 .
Desired outputs in the learning step

Table 3 .
Learning and testing results for different hidden layer sizes

Table 4 .
The designed circuit results