Wednesday, October 16, 2019

Jacobina, Hessian, gradient, maximum, minum, saddle point



The Jacobian matrix contains information about the local behavior of a function. The Jacobian matrix can be seen as a representation of some local factor of change. It consists of first order partial derivatives. If we take the partial derivatives from the first order partial derivatives, we get the second order partial derivatives, which are used in the Hessian matrix. The Hessian matrix is used for the Second Partial Derivative Test with which we can test, whether a point x is a local maximum, minimum or a so called saddle point .
With the Jacobian matrix we can convert from one coordinate system into another

NHANES National Youth Fitness Survey (NNYFS) 2012

https://wwwn.cdc.gov/nchs/nhanes/search/nnyfs12.aspx

Friday, October 11, 2019

graph isomorphism as encryption tool



As far as I know there is no cryptographic scheme based on Graph isomorphism. The following is the key reasons.
The security of a cryptographic scheme largely depend on one-wayness of the underlying function. For a function to be one-way it's not just need to be hard for few NP instances but must be hard for a random instance. In other words it is very easy to find problems that are hard for very instance but easy for majority of instances . Such problems may not come under P but they arn't one way functions either. One such good example is the encryption scheme based on subset-sum problem, which was eventually broken due to the above specified reason.

weighted adjacency matrix

Q: 0 means no link. but small value means a very close link.

In igraph, direction is from Column to row. The following example show arrow from 2nd and 3rd to 1st.



















In Yuan, network exact control paper, the directions are from row to columns. So, is the transpose of the igraph adjacency matrix.





smoking

smoking
CDC

https://chronicdata.cdc.gov/Survey-Data/Behavioral-Risk-Factor-Data-Tobacco-Use-2010-And-P/fpp2-pp25

Thursday, September 26, 2019

semantic segmentation (pixel wise classification)

semantic segmentation (pixel wise classification)

https://arxiv.org/abs/1511.00561

segnet :

SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation



Sunday, September 22, 2019

tensor

tensor is a general term:
2d tensor is matrix, 1D tensor is vector, 0D tensor is scalar.,
tensor can be more than 3D.


Friday, September 20, 2019

regularization, L1 and L2 norm


Regularization helps to choose preferred model complexity, so that model is better at predicting. Regularization is nothing but adding a penalty term to the objective function and control the model complexity using that penalty term. It can be used for many machine learning algorithms.

Wednesday, September 18, 2019

ssh with key


ssh -i ~/.ssh/private_key username@server.com 

openstack R tensorflow



qinlab3 instance

inside of Rstudio
install devtools from the drop down menu. This took some time.
ERROR:
openstack ubuntu libcurl was not found

See previous post for installing devetools in R
https://hongqinlab.blogspot.com/2017/11/microsoft-r-client-on-ubuntu-virtual.html

Apparently, default Ubuntu lack curl, openssl, xml2 etc.

1:45pm, finall be able to install keras on VirturalBox VM:

inside of R 3.4.x (installed through apt-get)
devtools::install_github("rstudio/keras")

> library(keras)
> install_keras()
Creating virtual environment '~/.virtualenvs/r-reticulate' ...
Using python: /home/hqin/anaconda3/bin/python3.7
Collecting pip
  Downloading https://files.pythonhosted.org/packages/30/db/9e38760b32e3e7f40cce46dd5fb107b8c73840df38f0046d8e6514e675a1/pip-19.2.3-py2.py3-none-any.whl (1.4MB)
    100% |████████████████████████████████| 1.4MB 21.5MB/s
Collecting wheel
  Downloading https://files.pythonhosted.org/packages/00/83/b4a77d044e78ad1a45610eb88f745be2fd2c6d658f9798a15e384b7d57c9/wheel-0.33.6-py2.py3-none-any.whl
Collecting setuptools
  Downloading https://files.pythonhosted.org/packages/b2/86/095d2f7829badc207c893dd4ac767e871f6cd547145df797ea26baea4e2e/setuptools-41.2.0-py2.py3-none-any.whl (576kB)
    100% |████████████████████████████████| 583kB 9.8MB/s


red-hat open stack will generate a key-pair, which save a private key on local computer, and leave a public key on the openstack server. The format of the key is *.pem.


Saturday, September 7, 2019

Monday, September 2, 2019

Sunday, September 1, 2019

disk.frame R data larger than RAM

R dealing with dataset larger than RAM
https://www.youtube.com/watch?v=3XMTyi_H4q4

tensorflow, keras

Keras tensorflow


























# Initialize x_1 and x_2
x_1 = Variable(6.0,float32)
x_2 = Variable(0.3,float32)

# Define the optimization operation
opt = keras.optimizers.SGD(learning_rate=0.01)

for j in range(100):
# Perform minimization using the loss function and x_1
opt.minimize(lambda: loss_function(x_1), var_list=[x_1])
# Perform minimization using the loss function and x_2
opt.minimize(lambda: loss_function(x_2), var_list=[x_2])

# Print x_1 and x_2 as numpy arrays
print(x_1.numpy(), x_2.numpy())























































































How to combine two model into a merged model. 


# Define the first dense layer
model.add(keras.layers.Dense(16, activation='sigmoid', input_shape=(784,)))

# Apply dropout to the first layer's output
model.add(keras.layers.Dropout(0.25))

# Define the output layer
model.add(keras.layers.Dense(4, activation='softmax'))

# Compile the model
model.compile('adam', loss='categorical_crossentropy')

How to merge 2 models with functional API

In [3]: # For model 1, pass the input layer to layer 1 and layer 1 to layer 2
        m1_layer1 = keras.layers.Dense(12, activation='sigmoid')(m1_inputs)
        m1_layer2 = keras.layers.Dense(4, activation='softmax')(m1_layer1)
        
        # For model 2, pass the input layer to layer 1 and layer 1 to layer 2
        m2_layer1 = keras.layers.Dense(12, activation='relu')(m2_inputs)
        m2_layer2 = keras.layers.Dense(4, activation='softmax')(m2_layer1)
        
        # Merge model outputs and define a functional model
        merged = keras.layers.add([m1_layer2, m2_layer2])
        model = keras.Model(inputs=[m1_inputs, m2_inputs], outputs=merged)
        
        # Print a model summary
        print(model.summary())
Model: "model"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_1 (InputLayer)            [(None, 784)]        0                                            
__________________________________________________________________________________________________
input_2 (InputLayer)            [(None, 784)]        0                                            
__________________________________________________________________________________________________
dense_5 (Dense)                 (None, 12)           9420        input_1[0][0]                    
__________________________________________________________________________________________________
dense_7 (Dense)                 (None, 12)           9420        input_2[0][0]                    
__________________________________________________________________________________________________
dense_6 (Dense)                 (None, 4)            52          dense_5[0][0]                    
__________________________________________________________________________________________________
dense_8 (Dense)                 (None, 4)            52          dense_7[0][0]                    
__________________________________________________________________________________________________
add_1 (Add)                     (None, 4)            0           dense_6[0][0]                    
                                                                 dense_8[0][0]                    
==================================================================================================
Total params: 18,944
Trainable params: 18,944
Non-trainable params: 0
__________________________________________________________________________________________________
None

<script.py> output:
    Model: "model"
    __________________________________________________________________________________________________
    Layer (type)                    Output Shape         Param #     Connected to                     
    ==================================================================================================
    input_1 (InputLayer)            [(None, 784)]        0                                            
    __________________________________________________________________________________________________
    input_2 (InputLayer)            [(None, 784)]        0                                            
    __________________________________________________________________________________________________
    dense (Dense)                   (None, 12)           9420        input_1[0][0]                    
    __________________________________________________________________________________________________
    dense_2 (Dense)                 (None, 12)           9420        input_2[0][0]                    
    __________________________________________________________________________________________________
    dense_1 (Dense)                 (None, 4)            52          dense[0][0]                      
    __________________________________________________________________________________________________
    dense_3 (Dense)                 (None, 4)            52          dense_2[0][0]                    
    __________________________________________________________________________________________________
    add (Add)                       (None, 4)            0           dense_1[0][0]                    
                                                                     dense_3[0][0]                    
    ==================================================================================================
    Total params: 18,944
    Trainable params: 18,944
    Non-trainable params: 0
    __________________________________________________________________________________________________
    None


How to perform validation






Keras: A metric is a function that is used to judge the performance of your model. Metric functions are to be supplied in the metrics parameter when a model is compiled.


 Define sequential model
        model = keras.Sequential()
        
        # Define the first layer
        model.add(keras.layers.Dense(32, activation='sigmoid', input_shape=(784,)))
        
        # Add activation function to classifier
        model.add(keras.layers.Dense(4, activation='softmax'))
        
        # Set the optimizer, loss function, and metrics
        model.compile(optimizer='rmsprop', loss='categorical_crossentropy', metrics=['accuracy'])
        
        # Add the number of epochs and the validation split
        model.fit(sign_language_features, sign_language_labels, epochs=10, validation_split=0.1)
Train on 1799 samples, validate on 200 samples
Epoch 1/10

  32/1799 [..............................] - ETA: 13s - loss: 1.5621 - accuracy: 0.1250
 384/1799 [=====>........................] - ETA: 1s - loss: 1.3521 - accuracy: 0.3151
 800/1799 [============>.................] - ETA: 0s - loss: 1.2557 - accuracy: 0.4425
1216/1799 [===================>..........] - ETA: 0s - loss: 1.1894 - accuracy: 0.5173
1792/1799 [============================>.] - ETA: 0s - loss: 1.1164 - accuracy: 0.5714
1799/1799 [==============================] - 1s 382us/sample - loss: 1.1150 - accuracy: 0.5725 - val_loss: 0.9990 - val_accuracy: 0.4700
Epoch 2/10

  32/1799 [..............................] - ETA: 0s - loss: 0.8695 - accuracy: 0.6562
 640/1799 [=========>....................] - ETA: 0s - loss: 0.8454 - accuracy: 0.7609
1184/1799 [==================>...........] - ETA: 0s - loss: 0.8061 - accuracy: 0.7753
1799/1799 [==============================] - 0s 97us/sample - loss: 0.7713 - accuracy: 0.7916 - val_loss: 0.6902 - val_accuracy: 0.7900

... ... 

  32/1799 [..............................] - ETA: 0s - loss: 0.2896 - accuracy: 0.8750
 672/1799 [==========>...................] - ETA: 0s - loss: 0.2077 - accuracy: 0.9717
1312/1799 [====================>.........] - ETA: 0s - loss: 0.2016 - accuracy: 0.9748
1799/1799 [==============================] - 0s 91us/sample - loss: 0.1943 - accuracy: 0.9739 - val_loss: 0.1634 - val_accuracy: 0.9800
Epoch 9/10

  32/1799 [..............................] - ETA: 0s - loss: 0.1352 - accuracy: 1.0000
 672/1799 [==========>...................] - ETA: 0s - loss: 0.1700 - accuracy: 0.9747
1312/1799 [====================>.........] - ETA: 0s - loss: 0.1612 - accuracy: 0.9809
1799/1799 [==============================] - 0s 89us/sample - loss: 0.1596 - accuracy: 0.9822 - val_loss: 0.1303 - val_accuracy: 0.9950
Epoch 10/10

  32/1799 [..............................] - ETA: 0s - loss: 0.1017 - accuracy: 1.0000
 704/1799 [==========>...................] - ETA: 0s - loss: 0.1478 - accuracy: 0.9858
1344/1799 [=====================>........] - ETA: 0s - loss: 0.1387 - accuracy: 0.9829
1799/1799 [==============================] - 0s 88us/sample - loss: 0.1358 - accuracy: 0.9817 - val_loss: 0.1126 - val_accuracy: 0.9850
Out[1]: <tensorflow.python.keras.callbacks.History at 0x7f7aab02fc18>

Overfitting
You will detect overfitting by checking whether the validation sample loss is substantially higher than the training sample loss and whether it increases with further training. With a small sample and a high learning rate, the model will struggle to converge on an optimum. You will set a low learning rate for the optimizer, which will make it easier to identify overfitting.
Excellent work! You may have noticed that the validation loss, val_loss, was substantially higher than the training loss, loss. Furthermore, if val_loss started to increase before the training process was terminated, then we may have overfitted. When this happens, you will want to try decreasing the number of epochs.

model.add(keras.layers.Dense(1024, activation='relu', input_shape=(784,)))

# Add activation function to classifier
model.add(keras.layers.Dense(4, activation='softmax'))

# Finish the model compilation
model.compile(optimizer=keras.optimizers.Adam(lr=0.01),
              loss='categorical_crossentropy', metrics=['accuracy'])

# Complete the model fit operation
model.fit(sign_language_features, sign_language_labels, epochs=200, validation_split=0.5)


In [1]: # Evaluate the small model using the train data
        small_train = small_model.evaluate(train_features, train_labels)
     
        # Evaluate the small model using the test data
        small_test = small_model.evaluate(test_features, test_labels)
     
        # Evaluate the large model using the train data
        large_train = large_model.evaluate(train_features, train_labels)
     
        # Evaluate the large model using the test data
        large_test = large_model.evaluate(test_features, test_labels)
     
        # Print losses
        print('\n Small - Train: {}, Test: {}'.format(small_train, small_test))
        print('Large - Train: {}, Test: {}'.format(large_train, large_test))

 32/100 [========>.....................] - ETA: 0s - loss: 0.9823
100/100 [==============================] - 0s 365us/sample - loss: 0.9452

 32/100 [========>.....................] - ETA: 0s - loss: 0.9657
100/100 [==============================] - 0s 57us/sample - loss: 1.0131

 32/100 [========>.....................] - ETA: 0s - loss: 0.0650
100/100 [==============================] - 0s 371us/sample - loss: 0.0487

 32/100 [========>.....................] - ETA: 0s - loss: 0.1011
100/100 [==============================] - 0s 61us/sample - loss: 0.2201

 Small - Train: 0.9452353072166443, Test: 1.0130866527557374
Large - Train: 0.04870099343359471, Test: 0.2201059103012085


Estimators: high level API


























































































Monday, August 26, 2019

Canvas gradebook, weighted final grade


Under "Assignments", "Groups" can be added, modified, deleted.

For weighted final grades,
"Assignment" -> "..." -> "weight final grade"


Changes in Cellular Architecture During Aging (R01)

Changes in Cellular Architecture During Aging (R01)

https://grants.nih.gov/grants/guide/pa-files/PA-16-442.html


algebraic multiplicity and geometric multiplicity

https://people.math.carleton.ca/~kcheung/math/notes/MATH1107/wk10/10_algebraic_and_geometric_multiplicities.html

The algebraic multiplicity of λ is the number of times λ is repeated as a root of the characteristic polynomial.

 Let A be an n × n matrix with eigenvalue λ. The geometric multiplicity of λ is the dimension of the eigenspace of λ.

In general, the algebraic multiplicity and geometric multiplicity of an eigenvalue can differ. However, the geometric multiplicity can never exceed the algebraic multiplicity.
It is a fact that summing up the algebraic multiplicities of all the eigenvalues of an n×n matrix A gives exactly nIf for every eigenvalue of A, the geometric multiplicity equals the algebraic multiplicity, then A is said to be diagonalizable. As we will see, it is relatively easy to compute powers of a diagonalizable matrix.

yeast PIN nD

controllability for yPIN with Dang's data



```{r eigen-spectrum}
e=eig$values;
summary(e)
digits = c(1:30)
zeros = digits
debug = 0
for ( i in 1:length(digits ) ) {
 tmp = sort(table(round(e, roundings[i])), decreasing = T)
 if (debug > 0) { print(tmp[1:3]) }
 zeros[i] = tmp[1]
}
cbind( digits , zeros)
```

   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
-59.919  -1.696   0.000   0.000   1.384  97.758 
      digits zeros
 [1,]      1   228
 [2,]      2   158
 [3,]      3   140
 [4,]      4   136
 [5,]      5   135
 [6,]      6   135
 [7,]      7   135
 [8,]      8   135
 [9,]      9   135
[10,]     10   135
[11,]     11   135
[12,]     12   135
[13,]     13   135
[14,]     14   135
[15,]     15   112
[16,]     16    45
[17,]     17     9
[18,]     18     3
[19,]     19     2
[20,]     20     1
[21,]     21     1
[22,]     22     1
[23,]     23     1
[24,]     24     1
[25,]     25     1
[26,]     26     1
[27,]     27     1
[28,]     28     1
[29,]     29     1
[30,]     30     1






























Conclusion: frequency of zero eigen values stabilize between 5 and 14 at 135. 


Friday, August 23, 2019

UTC work request


Everyone is able to put in a work request. Go here https://fpmis.utc.edu/. Put in your information and what room is having the problem.

Wednesday, August 21, 2019

Latex submission to BMC


All Latex files and template should all be listed as Manuscript


colony counting


sectioned colony counting is difficult.



Vanderbilt cancer heterogeneity workshop


=> Ken Lau, single cell RNAseq
https://www.mc.vanderbilt.edu/vumcdept/cellbio/laulab/research.html

scanpy
AnnData

get high-quality cells,
feature selection

dpFeature, select EDG between clusters identified by density peak clusting, Qiu 2017
SCENIC, coordianted regulated TF target, Aibar, 2018
NVR, neighborhood variance ratio, Welch 2016, Chen 2019 (Lau lab).

trajectory reconstruction, many algorithms,




phage PIN and aging simulation

first principle simulation of phage PIN: Can I regenerate the exponetial survival curves?

 2013 Dec;87(23):12745-55. doi: 10.1128/JVI.02495-13. Epub 2013 Sep 18.

The protein interaction network of bacteriophage lambda with its host, Escherichia coli.

https://www.ncbi.nlm.nih.gov/pubmed/24049175


 2011 Sep 26;11:213. doi: 10.1186/1471-2180-11-213.

The protein interaction map of bacteriophage lambda.



https://www.ncbi.nlm.nih.gov/pubmed/21943085