![](https://s3.amazonaws.com/media-p.slid.es/uploads/1241297/images/6922576/s2net.png)
A generalized semi-supervised elastic-net
Juan C. Laria
juancarlos.laria@uc3m.es
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1241297/images/6935839/logo-uc3m-3_1.png)
The semi-supervised framework
unsupervised
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1241297/images/6935895/dtulogo_white.png)
The semi-supervised framework
supervised
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1241297/images/6935839/logo-uc3m-3_1.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1241297/images/6935895/dtulogo_white.png)
The semi-supervised framework
semi-supervised
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1241297/images/6935839/logo-uc3m-3_1.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1241297/images/6935895/dtulogo_white.png)
The semi-supervised framework
semi-supervised
transfer-model
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1241297/images/6935839/logo-uc3m-3_1.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1241297/images/6935895/dtulogo_white.png)
Elastic-net regularization
lasso
ridge
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1241297/images/6935839/logo-uc3m-3_1.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1241297/images/6935895/dtulogo_white.png)
Joint trained elastic-net
Culp, M. (2013). On the semisupervised joint trained elastic net. Journal of Computational and Graphical Statistics 22 (2), 300–318.
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1241297/images/6935839/logo-uc3m-3_1.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1241297/images/6935895/dtulogo_white.png)
Extended linear joint trained framework
Søgaard Larsen, J. et. al (2020). Semi-supervised covariate shift modelling of spectroscopic data. (in-press)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1241297/images/6935839/logo-uc3m-3_1.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1241297/images/6935895/dtulogo_white.png)
Semi-supervised elastic-net
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1241297/images/6935839/logo-uc3m-3_1.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1241297/images/6935895/dtulogo_white.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1241297/images/6932374/nExtJT_JT.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1241297/images/6935839/logo-uc3m-3_1.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1241297/images/6935895/dtulogo_white.png)
Optimization
elastic-net with custom loss function
- Fast iterative shrinkage-thresholding algorithm (FISTA)
- Acelerated (block) gradient descent approach
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1241297/images/6935839/logo-uc3m-3_1.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1241297/images/6935895/dtulogo_white.png)
Implementation
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1241297/images/6935839/logo-uc3m-3_1.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1241297/images/6935860/arma.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1241297/images/6935865/rcpp.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1241297/images/6935867/Rlogo.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1241297/images/6935869/cpp.png)
-
We develop a flexible and fast implementation for s2net in R, written in C++ using RcppArmadillo and integrated into R via Rcpp modules.
-
The software is available in the s2net package
install.packages("s2net")
library(s2net)
vignette(package = "s2net")
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1241297/images/6935895/dtulogo_white.png)
Hyper-parameter tuning
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1241297/images/6935839/logo-uc3m-3_1.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1241297/images/6935895/dtulogo_white.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1241297/images/6980490/cloudml.png)
library(cloudml)
cloudml_train("main_script.R", config = "tuning.yml")
FLAGS <- flags(
flag_numeric("lambda1", 0.01, "Lasso weight"),
flag_numeric("lambda2", 0.01, "Ridge weight"),
flag_numeric("gamma1", 0.1, "s2net global weight"),
flag_numeric("gamma2", 100, "s2net covariance parameter"),
flag_numeric("gamma3", 0.5, "s2net shift parameter")
)
- Cloud based solution
- Easy to implement
- Blackbox grid/random search, bayesian optimization available
- Limited resources in the free tier
- Can be pricey
- Slow - not suitable for fast algorithms
- All dependencies should be available from CRAN - otherwise it is very difficult to make it work
Hyper-parameter tuning
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1241297/images/6935839/logo-uc3m-3_1.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1241297/images/6935895/dtulogo_white.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1241297/images/6980500/sparklyr-hex.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/1241297/images/6980503/spark-logo-trademark.png)
- Cloud based solution that runs locally
- Extremely fast, given the hardware
- Open source
- Very difficult to implement
- Needs local hardware, or renting a cluster in the cloud
- Native scala hyper-parameter tuning functions are only available for popular methods.
- You have to implement your search function that works with your method.
library(sparklyr)
result = spark_apply(grid, my_function, context = datos)
THANK you!
s2net
By Juan Carlos Laria
s2net
- 843