Sunday, November 8, 2020

video educators (to do)

 

https://twitter.com/CsEverything

Website: http://everythingcomputerscience.com/

https://www.youtube.com/channel/UCaV_0qp2NZd319K4_K8Z5SQ 


Barry Grant. 

https://www.youtube.com/channel/UCuop0rFv1hi84ZkONOZtdJA


Kenberley Norris Russell, facebook, playposit

https://www.facebook.com/groups/134847691291662/permalink/276908243752272/ 

R twitter with illustration


Kelly Bodin, decode your R code with flair

https://education.rstudio.com/blog/2020/05/flair/ 

computational complexity notes

 

Master Theorem

https://youtu.be/T68vN1FNY4o


Saturday, November 7, 2020

voter registration record data sets

 


https://data.pa.gov/Government-Efficiency-Citizen-Engagement/2020-General-Election-Mail-Ballot-Requests-Departm/mcba-yywm/data


data.gov

https://catalog.data.gov/organization/allegheny-county-city-of-pittsburgh-western-pa-regional-data-center



bit.ly/TFPlayground

 tensorflow Playground


bit.ly/TFPlayground

https://playground.tensorflow.org/









Python NetworkX

 

https://www.youtube.com/watch?v=yMSCWLma46s


RStudio freezing and QT Webengine eating CPU,

This bug was solved by re-install R. 

https://community.rstudio.com/t/rstudio-freezing-and-qt-webengine-eating-cpu/28019



tryCatch example in R

 

   tryCatch( info <- getMeta(lat = metaTb$Lat[i], lon = metaTb$Long_[i], plot=FALSE),   

       error=function(e)  {

         print( paste( "*** tryCaught error:", metaTb[i, ]) )

       } 

    )

    print(i)



Wednesday, November 4, 2020

Broad TERRA workspace

 

https://terra.bio/covid19

https://support.terra.bio/hc/en-us/articles/360041068771--COVID-19-workspaces-data-and-tools-in-Terra

https://terra.bio/covid19

Some broad sequences in SRA


Cellular and protein homeostasis webinars

 

Cellular and protein homeostasis webinars

Paolo De Los RiosÉcole polytechnique fédérale de Lausanne (EPFL)

Webinar series on cellular and protein homeostasis

Organised by: P. De Los Rios, N.B. Nillegoda, A. Barducci and P. Goloubinoff

https://tube.switch.ch/channels/4ed71569


Monday, November 2, 2020

medical image MNIST data set

 https://arxiv.org/abs/2010.14925 

"We present MedMNIST, a collection of 10 pre-processed medical open datasets. MedMNIST is standardized to perform classification tasks on lightweight 28 * 28 images, which requires no background knowledge. Covering the primary data modalities in medical image analysis, it is diverse on data scale (from 100 to 100,000) and tasks (binary/multi-class, ordinal regression and multi-label). MedMNIST could be used for educational purpose, rapid prototyping, multi-modal machine learning or AutoML in medical image analysis. Moreover, MedMNIST Classification Decathlon is designed to benchmark AutoML algorithms on all 10 datasets."


https://github.com/MedMNIST/MedMNIST 


Saturday, October 31, 2020

mixed model analysis

 


https://arbor-analytics.com/post/mixed-models-a-primer/


Fourier Neural Operator for Parametric Partial Differential Equations

Fourier Neural Operator for Parametric Partial Differential Equations


 Zongyi Li∗ , Nikola Kovachki∗ , Kamyar Azizzadenesheli† , Burigede Liu∗ , Kaushik Bhattacharya∗ , Andrew Stuart∗ , Anima Anandkumar∗ October 20, 2020


"The classical development of neural networks has primarily focused on learning mappings between finite-dimensional Euclidean spaces. Recently, this has been generalized to neural operators that learn mappings between function spaces. For partial differential equations (PDEs), neural operators directly learn the mapping from any functional parametric dependence to the solution. Thus, they learn an entire family of PDEs, in contrast to classical methods which solve one instance of the equation. In this work, we formulate a new neural operator by parameterizing the integral kernel directly in Fourier space, allowing for an expressive and efficient architecture. We perform experiments on Burgers’ equation, Darcy flow, and the Navier-Stokes equation (including the turbulent regime). Our Fourier neural operator shows state-of-the-art performance compared to existing neural network methodologies and it is up to three orders of magnitude faster compared to traditional PDE solvers."


 

https://www.technologyreview.com/2020/10/30/1011435/ai-fourier-neural-network-cracks-navier-stokes-and-partial-differential-equations/?utm_term=Autofeed&utm_campaign=site_visitor.unpaid.engagement&utm_medium=tr_social&utm_source=Facebook&fbclid=IwAR3HPDxmTVGrYLdfUzBFL7KeHiSlfN57dmZx2IStwA4dNpywqccY6Ip_9sk#Echobox=1604049241



ts conda create condaR403

 # Hong will install anaconda R403 in a conda environment on ts117. This strategy worked. 

conda create --name condaR403

  environment location: /home/hqin/.conda/envs/condaR403


-bash-4.2$ conda activate condaR403


conda install -c r r-base #??

# which R shows an R403 inside an conda environment. 


R

install.packages('tidyverse') #this seems worked. 

install.packages('EpiNow2')  #this run for a while

non-zero exit again due to V8. 


(condaR403) -bash-4.2$ conda install -c conda-forge libv8

conda install -c conda-forge r-randomcolor


R

install.packages('EpiNow2') #this worked!!!!

> library(EpiNow2)

> 


module load sge


qsub epinow2.pbs #this runs!!!!


(condaR403) -bash-4.2$ cat epinow2.pbs

#!/bin/bash -l

#$ -S /bin/bash

#$ -N epinow_job

#$ -V

#$ -cwd


. /etc/profile.d/modules.sh


module load anaconda/5.2.0


source activate condaR403


R -f batch_Rt_by_county.R --args 900 901 1 4/1/2020 5/1/2020



Friday, October 30, 2020

ts conda tsR403, tidyverse, EpiNow2 installation

# THIS DID NOT WORK

-bash-4.2$ module load anaconda/5.2.0 

-bash-4.2$ conda create --name tsR403

Collecting package metadata: done

Solving environment: done


## Package Plan ##


  environment location: /home/hqin/.conda/envs/tsR403




Proceed ([y]/n)? y  


Preparing transaction: done

Verifying transaction: done

Executing transaction: done

#

# To activate this environment, use

#

#     $ conda activate tsR403

#

# To deactivate an active environment, use

#

#     $ conda deactivate


-bash-4.2$ 

 -bash-4.2$ conda activate tsR403

(tsR403) -bash-4.2$ conda install -c conda-forge libv8

Collecting package metadata: done

Hong then run R

install.packages('tidyverse') #this seem to worked. 

-----------------------------[ ANTICONF ]-------------------------------

Configuration failed to find the libv8 engine library. Try installing:

 * deb: libv8-dev or libnode-dev (Debian / Ubuntu)

 * rpm: v8-devel (Fedora, EPEL)

 * brew: v8 (OSX)

 * csw: libv8_dev (Solaris)

To use a custom libv8, set INCLUDE_DIR and LIB_DIR manually via:

R CMD INSTALL --configure-vars='INCLUDE_DIR=... LIB_DIR=...'

---------------------------[ ERROR MESSAGE ]----------------------------

<stdin>:1:16: fatal error: v8.h: No such file or directory

compilation terminated.

-------------------------------------------------------


(tsR403) -bash-4.2$ conda install -c conda-forge r-randomcolor

Collecting package metadata: \ 

#this install many packages

Ref: https://github.com/iaconogi/bigSCale2/issues/19

I then tried:

R: install.packages('EpiNow2') #this seems to be running now. 

library(EpiNow2) #it worked!

install.packages('woldmet')

#The entire install seem to take almost 2 hours. 

Checked a few hour later, EpiNow2 installation did not work.