cajo Rt, gworkplace, t2m, ridgeside

 add Rt ~ mobility + weather test result to batch output. 

16:15, debug passed, 

16:50, execute the R code on ridgeside. 

17:51 Run three parallel R codes: 

(r2021nov) hqin@ridgeside:~/github/COVID19_transmission_MS$  
(r2021nov) hqin@ridgeside:~/github/COVID19_transmission_MS$ ps -ef | grep cajo
hqin      3938 23753 95 17:49 pts/3    00:00:33 /home/hqin/.conda/envs/r2021nov/lib/R/bin/exec/R -f auto_count_cajo_noargs-1K-2K.R
hqin      6046  2716 99 16:51 pts/0    01:37:38 /home/hqin/.conda/envs/r2021nov/lib/R/bin/exec/R -f auto_county_cajo_noargs.R
hqin     23702 23753  0 17:49 pts/3    00:00:00 grep cajo
hqin     30165 23753 99 17:49 pts/3    00:00:07 /home/hqin/.conda/envs/r2021nov/lib/R/bin/exec/R -f auto_count_cajo_noargs-2K-3151.R
hqin     31881 23753 99 17:48 pts/3    00:09:55 /home/hqin/.conda/envs/r2021nov/lib/R/bin/exec/R -f auto_count_cajo_noargs-1-1000.R

So, some counties in 1-1000 rows are taking a very long time. 

Next day, 2022-03-09, 15:11, The 1-1000 chunk R codes is still running after 22 hours. So, I need to find out the bottleneck, and output file piecewise. 

(r2021nov) hqin@ridgeside:~/github/COVID19_transmission_MS$ ps -ef | grep count
root       848     1  0 Mar03 ?        00:00:07 /usr/lib/accountsservice/accounts-daemon
hqin      6046  2716 99 Mar08 pts/0    23:09:38 /home/hqin/.conda/envs/r2021nov/lib/R/bin/exec/R -f auto_county_cajo_noargs.R
hqin     12325 23753  0 15:21 pts/3    00:00:00 grep count
hqin     31881 23753 99 Mar08 pts/3    21:51:50 /home/hqin/.conda/envs/r2021nov/lib/R/bin/exec/R -f auto_count_cajo_noargs-1-1000.R

So, I kill 31811 job.

Revised the R code to output file step by step. 

Surprisingly, this time the code finished shortly. 

(r2021nov) hqin@ridgeside:~/github/COVID19_transmission_MS$ ls -lth *csv  

-rw-r--r-- 1 hqin simctr 175K Mar  9 15:52 _autocajo_ridgeside1-1000:2020-03-15-to-2021-08-01.csv

So, the problem is stochastic. So, I killed the long-running job. 

synced the running results to github. 

done.