contrast learning

 

https://builtin.com/machine-learning/contrastive-learning

self-supervised learning. 

machine learning model attrition challenge

 

machine learning model attrition challenge

https://mlmac.io/#submission

Federated learning (book)

Book series

Federated Learning (FL) requires an aggregator and parties to exchange model updates. (Page 285)

vulnerable to the inference of private data

System entities of the FL system

the attack surface is used to refer to the exposed parameters and data

against data leak

FL-specific attacks often take advantage of the information transmission during FL. 

Differential privacy: differential privacy at the party side or the aggregator side. 

For healthcare data and personal information, there are regulation and compliance requirements [14, 63]

page 285: In FL, training data is not explicitly shared. 

$13.3.1 Secure Aggregation 

interpretable machine learning Christo[h Molnar

 

https://christophm.github.io/interpretable-ml-book/

confusion matrix in machine learning

 

accuracy, F1

ICML

 

https://icml.cc/Sponsors/sponsorinfo

training set, validation set, and test test for machine learing / deep learning

 

from: https://machinelearningmastery.com/difference-test-validation-datasets/

– Training set: A set of examples used for learning, that is to fit the parameters of the classifier.

– Validation set: A set of examples used to tune the parameters of a classifier, for example to choose the number of hidden units in a neural network.

– Test set: A set of examples used only to assess the performance of a fully-specified classifier.

control theory and machine learning

 

https://sites.google.com/view/control-meets-learning

Hebbian, Oja learning

 14:16:01 From Hong Qin to Everyone:

https://en.wikipedia.org/wiki/Lasso_(statistics)

14:24:48 From Trevor Peyton to Everyone:

https://en.wikipedia.org/wiki/Hebbian_theory

14:25:25 From Trevor Peyton to Everyone:

https://en.wikipedia.org/wiki/Generalized_Hebbian_algorithm

14:26:08 From Trevor Peyton to Everyone:

https://en.wikipedia.org/wiki/Oja%27s_rule

universal approximation

 Oldies but goldies: A. Barron, Universal Approximation Bounds for Superpositions of a Sigmoidal Function, 1993. Proves that 1 hidden layer perceptrons break the curse of dimensionality to approximate a class of smooth functions. https://en.wikipedia.org/wiki/Universal_approximation_theorem… https://en.wikipedia.org/wiki/Multilayer_perceptron

https://twitter.com/gabrielpeyre/status/1384371246461329409

Galois_theory, Langlands program

 

https://en.wikipedia.org/wiki/Fundamental_theorem_of_Galois_theory

https://zh.wikipedia.org/wiki/%E4%BC%BD%E7%BD%97%E7%93%A6%E7%90%86%E8%AE%BA%E5%9F%BA%E6%9C%AC%E5%AE%9A%E7%90%86

我有一个直觉，工程上人们关于最优化方法的折腾路线图有点弱了，对那些年青人，我觉得可以以《群论与最优化》的主题去研究更具一般性，可以使数据科学的研究提升一个层次，一般来说，事物从某种状态任意变换为另一种状态路径的数目的庞大的，最直观的就是各种棋类问题，其实这些问题需要的思想与当年伽罗瓦解方程的思想沒本质区别（1百多年后，很少的人才能真正体会到它的真谛），如果人们能够熟练驾驭群的方法，人工智能的命运就会被改写，无需暴力方法。当年克莱因的纲领与现在朗兰兹的纲领，都逃不过伽罗瓦的群...

https://en.wikipedia.org/wiki/Langlands_program

https://en.wikipedia.org/wiki/Felix_Klein

https://en.wikipedia.org/wiki/Erlangen_program

regularization, L1 and L2 norm

What is the main purpose of using regularization?

Regularization helps to choose preferred model complexity, so that model is better at predicting. Regularization is nothing but adding a penalty term to the objective function and control the model complexity using that penalty term. It can be used for many machine learning algorithms.

https://towardsdatascience.com/regularization-in-machine-learning-76441ddcf99a

machine learning, hyper parameters

hyper-parameters, such as partition data into training, validation, and testing, iteractions, in general have no fixed way to pick theoretically.

hyper-parameters are those that need to be fixed before learning started.

hyperparameter optimization may be done by compare a tuple of hyperparameters, based on a predefined loss function on dependent data.

machine learning in Python

http://scikit-learn.org/stable/

machine learning, resources

李宏毅
https://www.youtube.com/channel/UC2ggjtuuWvxrHHHiaDH1dlQ/feed

applejack, install keras

See https://keras.io/#installation

sudo pip install keras

AI courese

http://course.fast.ai/ . Python notebook

http://wiki.fast.ai/index.php/How_to_use_the_Provided_Notebooks


https://github.com/fastai/courses/tree/master/deeplearning1/nbs

deep learning tools

Deep neural networks 

List of potential R packages for deep learning:

deeplearning https://cran.r-project.org/web/packages/deeplearning/index.html

mxnet (GPU) http://mxnet.io/get_started/setup.html#install-the-mxnet-package-for-r

h2o https://cran.r-project.org/web/packages/h2o/index.html

DeepNet

Deeplearning libraries in Python

Torch.ch, academic use, flexibility. deepSEA
Caffe, developed by the Berkely Vision and Learning Center

Theano, transparent use of GPU.

Keras https://keras.io/ . (theano + tensorflow)

Lasagne

Nolearn

Mocha

Wisconsin breast cancer diagnostic data set, machine learning analysis

This must an old data set

http://archive.ics.uci.edu/ml/datasets/Breast+Cancer+Wisconsin+%28Diagnostic%29


Gavin Brown. Diversity in Neural Network Ensembles. The University of Birmingham. 2004. [View Context].

Krzysztof Grabczewski and Wl/odzisl/aw Duch. Heterogeneous Forests of Decision Trees. ICANN. 2002. [View Context].

András Antos and Balázs Kégl and Tamás Linder and Gábor Lugosi. Data-dependent margin-based generalization bounds for classification. Journal of Machine Learning Research, 3. 2002. [View Context].

Kristin P. Bennett and Ayhan Demiriz and Richard Maclin. Exploiting unlabeled data in ensemble methods. KDD. 2002. [View Context].

Hussein A. Abbass. An evolutionary artificial neural networks approach for breast cancer diagnosis. Artificial Intelligence in Medicine, 25. 2002. [View Context].

Baback Moghaddam and Gregory Shakhnarovich. Boosted Dyadic Kernel Discriminants. NIPS. 2002. [View Context].

Robert Burbidge and Matthew Trotter and Bernard F. Buxton and Sean B. Holden. STAR - Sparsity through Automated Rejection. IWANN (1). 2001. [View Context].

Nikunj C. Oza and Stuart J. Russell. Experimental comparisons of online and batch versions of bagging and boosting. KDD. 2001. [View Context].

Endre Boros and Peter Hammer and Toshihide Ibaraki and Alexander Kogan and Eddy Mayoraz and Ilya B. Muchnik. An Implementation of Logical Analysis of Data. IEEE Trans. Knowl. Data Eng, 12. 2000. [View Context].

Yuh-Jeng Lee. Smooth Support Vector Machines. Preliminary Thesis Proposal Computer Sciences Department University of Wisconsin. 2000. [View Context].

Justin Bradley and Kristin P. Bennett and Bennett A. Demiriz. Constrained K-Means Clustering. Microsoft Research Dept. of Mathematical Sciences One Microsoft Way Dept. of Decision Sciences and Eng. Sys. 2000. [View Context].

Lorne Mason and Peter L. Bartlett and Jonathan Baxter. Improved Generalization Through Explicit Optimization of Margins. Machine Learning, 38. 2000. [View Context].

P. S and Bradley K. P and Bennett A. Demiriz. Constrained K-Means Clustering. Microsoft Research Dept. of Mathematical Sciences One Microsoft Way Dept. of Decision Sciences and Eng. Sys. 2000. [View Context].

Chun-Nan Hsu and Hilmar Schuschel and Ya-Ting Yang. The ANNIGMA-Wrapper Approach to Neural Nets Feature Selection for Knowledge Discovery and Data Mining. Institute of Information Science. 1999. [View Context].

Huan Liu and Hiroshi Motoda and Manoranjan Dash. A Monotonic Measure for Optimal Feature Selection. ECML. 1998. [View Context].

Lorne Mason and Peter L. Bartlett and Jonathan Baxter. Direct Optimization of Margins Improves Generalization in Combined Classifiers. NIPS. 1998. [View Context].

W. Nick Street. A Neural Network Model for Prognostic Prediction. ICML. 1998. [View Context].

Yk Huhtala and Juha Kärkkäinen and Pasi Porkka and Hannu Toivonen. Efficient Discovery of Functional and Approximate Dependencies Using Partitions. ICDE. 1998. [View Context].

. Prototype Selection for Composite Nearest Neighbor Classifiers. Department of Computer Science University of Massachusetts. 1997. [View Context].

Kristin P. Bennett and Erin J. Bredensteiner. A Parametric Optimization Method for Machine Learning. INFORMS Journal on Computing, 9. 1997. [View Context].

Rudy Setiono and Huan Liu. NeuroLinear: From neural networks to oblique decision rules. Neurocomputing, 17. 1997. [View Context].

Erin J. Bredensteiner and Kristin P. Bennett. Feature Minimization within Decision Trees. National Science Foundation. 1996. [View Context].

Ismail Taha and Joydeep Ghosh. Characterization of the Wisconsin Breast cancer Database Using a Hybrid Symbolic-Connectionist System. Proceedings of ANNIE. 1996. [View Context].

Jennifer A. Blue and Kristin P. Bennett. Hybrid Extreme Point Tabu Search. Department of Mathematical Sciences Rensselaer Polytechnic Institute. 1996. [View Context].

Geoffrey I. Webb. OPUS: An Efficient Admissible Algorithm for Unordered Search. J. Artif. Intell. Res. (JAIR, 3. 1995. [View Context].

Chotirat Ann and Dimitrios Gunopulos. Scaling up the Naive Bayesian Classifier: Using Decision Trees for Feature Selection. Computer Science Department University of California. [View Context].

Wl odzisl/aw Duch and Rudy Setiono and Jacek M. Zurada. Computational intelligence methods for rule-based data understanding. [View Context].

Rafael S. Parpinelli and Heitor S. Lopes and Alex Alves Freitas. An Ant Colony Based System for Data Mining: Applications to Medical Data. CEFET-PR, CPGEI Av. Sete de Setembro, 3165. [View Context].

Wl/odzisl/aw Duch and Rafal/ Adamczak Email:duchraad@phys. uni. torun. pl. Statistical methods for construction of neural networks. Department of Computer Methods, Nicholas Copernicus University. [View Context].

Rafael S. Parpinelli and Heitor S. Lopes and Alex Alves Freitas. PART FOUR: ANT COLONY OPTIMIZATION AND IMMUNE SYSTEMS Chapter X An Ant Colony Algorithm for Classification Rule Discovery. CEFET-PR, Curitiba. [View Context].

Adam H. Cannon and Lenore J. Cowen and Carey E. Priebe. Approximate Distance Classification. Department of Mathematical Sciences The Johns Hopkins University. [View Context].

Andrew I. Schein and Lyle H. Ungar. A-Optimality for Active Learning of Logistic Regression Classifiers. Department of Computer and Information Science Levine Hall. [View Context].

Bart Baesens and Stijn Viaene and Tony Van Gestel and J. A. K Suykens and Guido Dedene and Bart De Moor and Jan Vanthienen and Katholieke Universiteit Leuven. An Empirical Assessment of Kernel Type Performance for Least Squares Support Vector Machine Classifiers. Dept. Applied Economic Sciences. [View Context].

Adil M. Bagirov and Alex Rubinov and A. N. Soukhojak and John Yearwood. Unsupervised and supervised data classification via nonsmooth and global optimization. School of Information Technology and Mathematical Sciences, The University of Ballarat. [View Context].

Rudy Setiono and Huan Liu. Neural-Network Feature Selector. Department of Information Systems and Computer Science National University of Singapore. [View Context].

Huan Liu. A Family of Efficient Rule Generators. Department of Information Systems and Computer Science National University of Singapore. [View Context].

Rudy Setiono. Extracting M-of-N Rules from Trained Neural Networks. School of Computing National University of Singapore. [View Context].

Jarkko Salojarvi and Samuel Kaski and Janne Sinkkonen. Discriminative clustering in Fisher metrics. Neural Networks Research Centre Helsinki University of Technology. [View Context].

Wl odzisl and Rafal Adamczak and Krzysztof Grabczewski and Grzegorz Zal. A hybrid method for extraction of logical rules from data. Department of Computer Methods, Nicholas Copernicus University. [View Context].

Charles Campbell and Nello Cristianini. Simple Learning Algorithms for Training Support Vector Machines. Dept. of Engineering Mathematics. [View Context].