todo: Elastic net method

33 Zou, H. & Hastie, T. Regularization and variable selection via the elastic net. J

Roy Stat Soc B 67, 301-320, (2005).

34 Zou, H. & Zhang, H. H. On the Adaptive Elastic-Net with a Diverging Number of

Parameters. Ann Stat 37, 1733-1751, (2009).

https://www.r-bloggers.com/kickin-it-with-elastic-net-regression/

"Ridge regression is a really effective technique for thwarting overfitting. It does this by penalizing the L2 norm (euclidean distance) of the coefficient vector which results in “shrinking” the beta coefficients. The aggressiveness of the penalty is controlled by a parameter ."

"Lasso regression is a related regularization method. Instead of using the L2 norm, though, it penalizes the L1 norm (manhattan distance) of the coefficient vector."

"Elastic net regression is a hybrid approach that blends both penalization of the L2 and L1 norms."

robust statistics methods and measures

consistency

asymptotic bounds

cook's distance
https://en.wikipedia.org/wiki/Cook%27s_distance

M-estimator

Influence function and sensitivity curves

statistical tests on two arrays of dependent data (time series)

For two list of dependent data X and Y, we want to test X < Y.

The pairwised t-test is inappropriate because it violates independence assumption.

Bootstrap or permutation can only be used when there the list of observations are long.
For our H2O2-LOH dataset, we have only 10 time points (H2O2 doses), so bootstrap and permutation is inappropriate as well.

W Wu suggests that Fisher's exact test can be applied for each time point, and then apply Bonferroni correction. We think 3/4 LOH suggest a positive dependence between 1/2 and 1/4 LOH.

fisher exact test in R, test with proportion input data (not working)

#Q: Does R's default fisher's exact test function handle proportion input data?

mat = matrix(nrow=2,ncol=2)
mat[1,1]=30
mat[2,2]=45
mat[1,2]=5
mat[2,1]=6
mat;
fisher.test(mat)

mat = mat/sum(mat)
mat;
fisher.test(mat)

#conclusion: NO. R's defaulty fisher.test() does not handle proportion input.

time series comparison

http://stats.stackexchange.com/questions/25228/testing-for-statistically-significant-difference-in-time-series

https://en.wikipedia.org/wiki/Bland%E2%80%93Altman_plot

http://www.r-bloggers.com/how-to-compare-two-blackbox-timeseries-generators/

Difference DID?
https://www.researchgate.net/post/How_do_I_compare_two_time_series_trends_in_terms_of_magnitude_of_decline_or_increase

statistical power, GWAS, quantitative genetics

Genet Epidemiol. 2015 Oct 10. doi: 10.1002/gepi.21935. Evaluating the Calibration and Power of Three Gene-Based Association Tests of Rare Variants for the X Chromosome. Ma C1, Boehnke M1, Lee S1; GoT2D Investigators.

Calibrating the Performance of SNP Arrays for WholeGenome Association Studies Ke Hao, Eric E. Schadt, John D. Storey, 2008, PLOS Genetics


Family-based association tests for sequence data, and comparisons with population-based association tests, Iuliana Ionita-Laza1, Seunggeun Lee, Vladimir Makarov, Joseph D. Buxbaum, and Xihong Lin

change point analysis

http://www.r-bloggers.com/a-simple-intro-to-bayesian-change-point-analysis/


related to my previous study on intragenic spatial codon bias.

ROC curve

http://www.r-bloggers.com/illustrated-guide-to-roc-and-auc/

study notes on PCA, principal components, with R testing codes, princomp()

ResearchGate: Principal components are linear combinations of original variables x1, x2, etc. So when you do SVM on PCA decomposition you work with these combinations instead of original variables.

37:50 in Ng's video
https://www.youtube.com/watch?v=ey2PE5xi9-A
Ng showes to PCA (linear combinations of raw data) to reduce dimension of data. 

PCA is basically orthogonal transformation
http://en.wikipedia.org/wiki/Principal_component_analysis
"Principal component analysis (PCA) is a statistical procedure that uses an orthogonal transformation to convert a set of observations of possibly correlated variables into a set of values of linearly uncorrelated variables called principal components."
"PCA can be done by eigenvalue decomposition of a data covariance (or correlation) matrix or singular value decomposition of a data matrix, usually after mean centering (and normalizing or using Z-scores) the data matrix for each attribute. The results of a PCA are usually discussed in terms of component scores, sometimes called factor scores (the transformed variable values corresponding to a particular data point), and loadings (the weight by which each standardized original variable should be multiplied to get the component score)."


R:  princomp( )
pc.cr <- princomp(~ Murder + Assault + UrbanPop,
                  data = USArrests, na.action = na.exclude, cor = TRUE)
pc.cr$scores[1:5, ]  #scores probably are PCA results, based on wikipedia entry

I can use examples of linear combination to verify my guess. 


#######start of the R testing code and results #########
x1 = rnorm(100)
x2 = rnorm(100)
x3 = x1 + x2 + rnorm(100)/20
x4 = 2*x1 + rnorm(100)/20
X = data.frame(cbind(x1,x2,x3,x4))
pc <- princomp(X)
plot(pc)#only two major components, consistent

head(pc)

#######start of the R testing code and results #########

#######start of 2nd R testing code and results #########

set.seed(2014)

x1 = rnorm(100)

x2 = x1 + rnorm(100)/20

X = data.frame(cbind(x1,x2))

pc <- princomp(X, cor = TRUE)

head(pc)

pc$score[,1] - (0.707*x1 + 0.707*x2) #does this approach zero? 

summary( pc$score[,1] - (0.707*x1 + 0.707*x2) + mean( 0.707*x1 + 0.707*x2 ) )

#good, it approaches zero

summary(lm( pc$score[,1] ~ x1 ))

#######End of 2nd R testing code and results #########

E(X^2), expectation, variance, basic concepts

http://www.mathsrevision.net/advanced-level-maths-revision/statistics/expectation-and-variance

gradient decent for optimization

Notes from Andrew Ng's Machine learning, Coursera. 

combine p-value for multiple tests

To combine 6 phenotypic measures for each SNP, one way is the Fisher’s method. When null hypotheses are true for all association tests, they are independent by definition, and their p-value can be combined as 

 for K number of traits. To correct for multiple tests, phenotypic data will be permuted 1000 times, and lowest p-value from each test will be used to generate a distribution to determine proper genome-wide significance. 

It seems reasonable to extrapolate this to combine q-values for multiple tests. 

uniform distribution, fishers exact test and pFDR

Carlson, Heckerman, Shani, Microsoft

$\pi_0$ is the mixing parameter for H0.
It can be considered as $\pi_0$ is a prior that H0 is true, and $\pi_1$ is the prior that alternative H1 is true. pFDR is a posterior.

woman in science, slate, jane hu

http://www.slate.com/articles/double_x/doublex/2014/06/women_in_science_a_new_study_on_how_male_professors_discriminate_against.html

http://www.pnas.org/cgi/doi/10.1073/pnas.1403334111

Type I and type II erros

From wikipedia

FDR, false discovery rate

http://en.wikipedia.org/wiki/False_discovery_rate

http://www.totallab.com/products/samespots/support/faq/pq-values.aspx

common discrete distributions

see wikipedia

Bernoulli, binomial, uniform, genometric, hyptergenomeric, negative binomial, poisson

p-value, Nature, 2014

http://www.nature.com/polopoly_fs/1.14700!/menu/main/topColumns/topLeftColumn/pdf/506150a.pdf

Sample variance, n-1

http://www.youtube.com/watch?v=D1hgiAla3KI

Moments and their generating functions

Moments

http://www.youtube.com/watch?v=peIWy4N6HTA

moments generating functions
http://www.youtube.com/watch?v=kXIdNfvOWak