PyRidge

This repository contains some supervised machine learning algorithms from the family of Ridge Classification,
also known as
Tikhonov regularization or
Extreme Learning Machine.
A nice discussion about these terms can be seen in this discussion in
StackExchange.

Although ELM is a polemic topic,
due to the accusations of plagiarism (see more here and
here),
some actual research is done by applying ensemble techniques to Ridge Classification ([3, 4]), thus some some papers
are used for implementing algorithms.

Main motivation of this repository is collecting the ML algorithms from my research while
I was doing my PhD in Data Science
in Universidad Loyola Andalucía.

Documentation and examples are in progress, but some part is available here.

Install

To install the library along with the dependencies,

python setup.py install

How to use a virtual environment

It is recommended to install the framework in a virtual environment

virtualenv -p python3.9 env

In order to activate the virtual environment

source env/bin/activate

To deactivate, just write deactivate.

Algorithms

List of algorithms implemented:

Kernel algorithms (or Ridge classification)

Kernel Ridge (KRidge) [1]

Neural algorithms (or ELM classification)

ELM [2]
AdaBoost ELM (AdaBoostELM) [3]
AdaBoost Negative Correlation (AdaBoostNCELM) [4]
Bagging ELM (BaggingELM) [5]
Diverse ELM (DiverseELM) [6]
Regularized Ensemble ELM (RegularizedEnsembleELM) [7]
PCA ELM [10]
PCA LDA ELM [11]
Sobol ELM [12]
Paralell Layer ELM [13]
Boosting Ridge ELM [14]

Negative Correlation algorithms

Negative Correlation ELM [8]
Negative Corelation with Neural Networks [9]

Code documentation

Documentation is published online. It can also be compiled locally, just having
sphinx installed. In the main directory, run the following code:

sphinx-build docs/source docs/

Data

Some data can be downloaded here for classification and
here for regression.
In order to run the tests, data and data_regression folders should be in main directory.

Also, repository uci-download-process
could help you to download some examples from UCI dataset.

An example

You can run a test for every algorithm, just simply

pytest

By default, logging level is set to INFO.

Bibliography

[1]: S. An, W. Liu and S. Venkatesh, "Face Recognition Using Kernel Ridge
Regression," 2007 IEEE Conference on Computer Vision and Pattern Recognition,
Minneapolis, MN, 2007, pp. 1-7.

[2]: G.-B. Huang, H. Zhou, X. Ding, and R. Zhang, “Extreme learning machine
for regression and multiclass classification,” IEEE Trans. Syst. man, Cybern.
Part B, Cybern., vol. 42, no. 2, pp. 513–29, 2012.

[3]: A. Riccardi, F. Fernández-Navarro, S. Carloni, F. Fernandez-Navarro,
and S. Carloni, “Cost-sensitive AdaBoost algorithm for ordinal regression
based on extreme learning machine,” IEEE Trans. Cybern., vol. 44, no. 10,
pp. 1898–1909, 2014.

[4]: Wang, S., Chen, H., & Yao, X. (2010, July). Negative correlation
learning for classification ensembles. In Neural Networks (IJCNN),
The 2010 International Joint Conference on (pp. 1-8). IEEE.

[5]: Tian, H., & Meng, B. (2010, September). A new modeling method based
on bagging ELM for day-ahead electricity price prediction. In Bio-Inspired
Computing: Theories and Applications (BIC-TA), 2010 IEEE Fifth
International Conference on (pp. 1076-1079). IEEE.

[6]: Perales-González, C., Carbonero-Ruz, M., Becerra-Alonso, D., &
Fernández-Navarro, F. (2018, June). A Preliminary Study of Diversity
in Extreme Learning Machines Ensembles. In International Conference
on Hybrid Artificial Intelligence Systems (pp. 302-314). Springer, Cham.

[7]: Perales-González, C., Carbonero-Ruz, M., Becerra-Alonso, D.,
Pérez-Rodríguez, F., & Fernández-Navarro, F.
(2019, June). Regularized Ensemble Neural Networks models in the
Extreme Learning Machine framework. In Neurocomputing (DOI: 10.1016/j.neucom.2019.06.040)

[8]: Perales-González, C., Carbonero-Ruz, M., Pérez-Rodríguez, J.,
Becerra-Alonso, D., & Fernández-Navarro, F.
(2020, March). Negative correlation learning in the extreme learning machine framework.
In Neural Comput & Applic (DOI: 10.1007/s00521-020-04788-9)

[9]: Wang, S., Chen, H., & Yao, X. (2010, July).
Negative correlation learning for classification ensembles.
In The 2010 International Joint Conference on Neural Networks (IJCNN) (pp. 1-8). IEEE.

[10]: Castaño, A., Fernández-Navarro, F., & Hervás-Martínez, C. (2013).
PCA-ELM: a robust and pruned extreme learning machine approach
based on principal component analysis.
Neural processing letters, 37(3), 377-392.

[11]: Castaño, A., Fernández-Navarro, F., Riccardi, A., & Hervás-Martínez, C. (2016).
Enforcement of the principal component analysis–extreme learning machine
algorithm by linear discriminant analysis.
Neural Computing and Applications, 27(6), 1749-1760.

[12]: Cervellera, C., & Macciò, D. (2015).
Low-discrepancy points for deterministic assignment of hidden
weights in extreme learning machines. IEEE transactions on neural networks
and learning systems, 27(4), 891-896.

[13]: Henríquez, P. A., & Ruz, G. A. (2017).
Extreme learning machine with a deterministic assignment of hidden
weights in two parallel layers. Neurocomputing, 226, 109-116.

[14]: Yangjun, R., Xiaoguang, S., Huyuan, S., Lijuan, S., & Xin, W. (2012, June).
Boosting ridge extreme learning machine.
2012 IEEE Symposium on Robotics and Applications (ISRA) (pp. 881-884). IEEE.

cperales/pyridge