parse ERA5 GRIB for JHU counties on ts117, Nov 14-17, 7 days runs

 The template pbs is: 

-bash-4.2$ cat ts_grib_job_template.pbs 

#!/bin/bash -l

#$ -S /bin/bash

#$ -N pygrib_NUMBER

#$ -cwd

. /etc/profile.d/modules.sh

module load anaconda

source activate grib 

python parse_grib_4JHU-debug.py STARTINDEX  ENDINDEX 

-bash-4.2$ 

Qin used a python code to generate pbs jobs, and then submit them using shell script. 

-bash-4.2$ cat generate_PBS_bash.py 

import os

import re

filebatchshell = "submit_PBS.sh"

fshell = open(filebatchshell, "w")

step = 15

for i in range( 0, 600, step):

    fin = open("ts_grib_job_template.pbs", "r")

    LinesIn = fin.readlines() 

    fin.close()

    LinesIn[2] = "#$ -N pygrib_" + str(i) + "\n"

    LinesIn[7] = "python parse_grib_4JHU.py " + str(i) + " " + str(i + step) + "  \n"

            

    fileout = "ts_" + str(i) + ".pbs"

    fout = open(fileout, "w")

    for line in LinesIn: 

        fout.write(line)

    fout.close()

    buffer = "qsub " + fileout + "\n"

    fshell.write(buffer)

fshell.close()

To parse the 2m-dew point, I used generate_PBS_bash_dewpoint.py

This is a 7-day run. The last t2m output is on Nov 20, and the last dewpoint file output is on Nov 21. 

On Nov 21, I use github to backup running logs, and output csv files