firefly head node has no GPU.
Showing posts with label SimCenter. Show all posts
Showing posts with label SimCenter. Show all posts
Monday, August 1, 2022
Wednesday, July 27, 2022
alphafold firefly
Singularity container environment is now available on Firefly. And I prepared an image for running Alphafold and designed a customizable run script for interacting with the instance.
To quickly test Alphafold with default config and input, run the following:
$ bash /opt/ohpc/pub/singularity/test-scripts/alphafold-2/alphafold-singularity-run.sh
Here's a list of locations for relevant directories:
- Alphafold inference data: /scr/alphafold-data
- Singularity image Alphafold: /opt/ohpc/pub/singularity/images/alphafold-2.sif
- Sample run script and fasta file: /opt/ohpc/pub/singularity/test-scripts/alphafold-2
Thursday, December 16, 2021
Anaconda tensorflow on IBM powerpc architecture
This is for LOOKOUT cluster at UTC SimCenter
The key step is to specify the channel for IBM PowerPC. More details can be found at the following link.
The following link might also be helpful.
If you need additional information, please let me know.
Saturday, October 31, 2020
ts conda create condaR403
# Hong will install anaconda R403 in a conda environment on ts117. This strategy worked.
conda create --name condaR403
environment location: /home/hqin/.conda/envs/condaR403
-bash-4.2$ conda activate condaR403
conda install -c r r-base #??
# which R shows an R403 inside an conda environment.
R
install.packages('tidyverse') #this seems worked.
install.packages('EpiNow2') #this run for a while
non-zero exit again due to V8.
(condaR403) -bash-4.2$ conda install -c conda-forge libv8
$ conda install -c conda-forge r-randomcolor
R
install.packages('EpiNow2') #this worked!!!!
> library(EpiNow2)
>
module load sge
qsub epinow2.pbs #this runs!!!!
(condaR403) -bash-4.2$ cat epinow2.pbs
#!/bin/bash -l
#$ -S /bin/bash
#$ -N epinow_job
#$ -V
#$ -cwd
. /etc/profile.d/modules.sh
module load anaconda/5.2.0
source activate condaR403
R -f batch_Rt_by_county.R --args 900 901 1 4/1/2020 5/1/2020
Friday, October 30, 2020
ts conda tsR403, tidyverse, EpiNow2 installation
# THIS DID NOT WORK
-bash-4.2$ module load anaconda/5.2.0
-bash-4.2$ conda create --name tsR403
Collecting package metadata: done
Solving environment: done
## Package Plan ##
environment location: /home/hqin/.conda/envs/tsR403
Proceed ([y]/n)? y
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
# $ conda activate tsR403
#
# To deactivate an active environment, use
#
# $ conda deactivate
-bash-4.2$
-bash-4.2$ conda activate tsR403
(tsR403) -bash-4.2$ conda install -c conda-forge libv8
Collecting package metadata: done
Hong then run R
install.packages('tidyverse') #this seem to worked.
-----------------------------[ ANTICONF ]-------------------------------
Configuration failed to find the libv8 engine library. Try installing:
* deb: libv8-dev or libnode-dev (Debian / Ubuntu)
* rpm: v8-devel (Fedora, EPEL)
* brew: v8 (OSX)
* csw: libv8_dev (Solaris)
To use a custom libv8, set INCLUDE_DIR and LIB_DIR manually via:
R CMD INSTALL --configure-vars='INCLUDE_DIR=... LIB_DIR=...'
---------------------------[ ERROR MESSAGE ]----------------------------
<stdin>:1:16: fatal error: v8.h: No such file or directory
compilation terminated.
-------------------------------------------------------
(tsR403) -bash-4.2$ conda install -c conda-forge r-randomcolor
Collecting package metadata: \
#this install many packages
Ref: https://github.com/iaconogi/bigSCale2/issues/19
I then tried:
R: install.packages('EpiNow2') #this seems to be running now.
library(EpiNow2) #it worked!
install.packages('woldmet')
#The entire install seem to take almost 2 hours.
Checked a few hour later, EpiNow2 installation did not work.
Wednesday, October 28, 2020
ts117 usage
module load sge
$ module load sge/2011.11p1
For local R package installation, it seems to go to
info man
-bash-4.2$ ls /cm/shared/apps/R/R-3.4.3/share/info/
I ran R/3.5.3
install.packages('tidyverse') # it went to /cm/shared/apps/R-3.5.3
Friday, June 19, 2020
Sunday, June 14, 2020
ts anaconda, pandas, pygrib (qin log)
-bash-4.2$ module load anaconda
-bash-4.2$ conda creat --name grib python=3.7
CommandNotFoundError: No command 'conda creat'.
Did you mean 'conda create'?
-bash-4.2$ conda create --name grib python=3.7
Collecting package metadata: done
Solving environment: done
## Package Plan ##
environment location: /home/hqin/.conda/envs/grib
added / updated specs:
- python=3.7
The following packages will be downloaded:
package | build
---------------------------|-----------------
_libgcc_mutex-0.1 | main 3 KB
ca-certificates-2020.1.1 | 0 132 KB
certifi-2020.4.5.1 | py37_0 159 KB
ld_impl_linux-64-2.33.1 | h53a641e_7 645 KB
libedit-3.1.20181209 | hc058e9b_0 188 KB
libffi-3.3 | he6710b0_1 54 KB
libgcc-ng-9.1.0 | hdf63c60_0 8.1 MB
libstdcxx-ng-9.1.0 | hdf63c60_0 4.0 MB
ncurses-6.2 | he6710b0_1 1.1 MB
openssl-1.1.1g | h7b6447c_0 3.8 MB
pip-20.0.2 | py37_3 1.9 MB
python-3.7.7 | hcff3b4d_5 52.6 MB
readline-8.0 | h7b6447c_0 428 KB
setuptools-47.1.1 | py37_0 647 KB
sqlite-3.31.1 | h62c20be_1 2.0 MB
tk-8.6.8 | hbc83047_0 3.1 MB
wheel-0.34.2 | py37_0 49 KB
xz-5.2.5 | h7b6447c_0 438 KB
zlib-1.2.11 | h7b6447c_3 120 KB
------------------------------------------------------------
Total: 79.5 MB
The following NEW packages will be INSTALLED:
_libgcc_mutex pkgs/main/linux-64::_libgcc_mutex-0.1-main
ca-certificates pkgs/main/linux-64::ca-certificates-2020.1.1-0
certifi pkgs/main/linux-64::certifi-2020.4.5.1-py37_0
ld_impl_linux-64 pkgs/main/linux-64::ld_impl_linux-64-2.33.1-h53a641e_7
libedit pkgs/main/linux-64::libedit-3.1.20181209-hc058e9b_0
libffi pkgs/main/linux-64::libffi-3.3-he6710b0_1
libgcc-ng pkgs/main/linux-64::libgcc-ng-9.1.0-hdf63c60_0
libstdcxx-ng pkgs/main/linux-64::libstdcxx-ng-9.1.0-hdf63c60_0
ncurses pkgs/main/linux-64::ncurses-6.2-he6710b0_1
openssl pkgs/main/linux-64::openssl-1.1.1g-h7b6447c_0
pip pkgs/main/linux-64::pip-20.0.2-py37_3
python pkgs/main/linux-64::python-3.7.7-hcff3b4d_5
readline pkgs/main/linux-64::readline-8.0-h7b6447c_0
setuptools pkgs/main/linux-64::setuptools-47.1.1-py37_0
sqlite pkgs/main/linux-64::sqlite-3.31.1-h62c20be_1
tk pkgs/main/linux-64::tk-8.6.8-hbc83047_0
wheel pkgs/main/linux-64::wheel-0.34.2-py37_0
xz pkgs/main/linux-64::xz-5.2.5-h7b6447c_0
zlib pkgs/main/linux-64::zlib-1.2.11-h7b6447c_3
Proceed ([y]/n)?
Downloading and Extracting Packages
libgcc-ng-9.1.0 | 8.1 MB | ################################################################## | 100%
sqlite-3.31.1 | 2.0 MB | ################################################################## | 100%
readline-8.0 | 428 KB | ################################################################## | 100%
xz-5.2.5 | 438 KB | ################################################################## | 100%
ca-certificates-2020 | 132 KB | ################################################################## | 100%
ncurses-6.2 | 1.1 MB | ################################################################## | 100%
certifi-2020.4.5.1 | 159 KB | ################################################################## | 100%
python-3.7.7 | 52.6 MB | ################################################################## | 100%
libffi-3.3 | 54 KB | ################################################################## | 100%
ld_impl_linux-64-2.3 | 645 KB | ################################################################## | 100%
setuptools-47.1.1 | 647 KB | ################################################################## | 100%
libedit-3.1.20181209 | 188 KB | ################################################################## | 100%
pip-20.0.2 | 1.9 MB | ################################################################## | 100%
zlib-1.2.11 | 120 KB | ################################################################## | 100%
libstdcxx-ng-9.1.0 | 4.0 MB | ################################################################## | 100%
wheel-0.34.2 | 49 KB | ################################################################## | 100%
_libgcc_mutex-0.1 | 3 KB | ################################################################## | 100%
openssl-1.1.1g | 3.8 MB | ################################################################## | 100%
tk-8.6.8 | 3.1 MB | ################################################################## | 100%
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
# $ conda activate grib
#
# To deactivate an active environment, use
#
# $ conda deactivate
condo activate grib
condo activate grib
(grib) -bash-4.2$ conda install pandas
Collecting package metadata: done
Solving environment: done
## Package Plan ##
environment location: /home/hqin/.conda/envs/grib
added / updated specs:
- pandas
The following packages will be downloaded:
package | build
---------------------------|-----------------
blas-1.0 | mkl 6 KB
intel-openmp-2020.1 | 217 940 KB
libgfortran-ng-7.3.0 | hdf63c60_0 1.3 MB
mkl-2020.1 | 217 200.5 MB
mkl-service-2.3.0 | py37he904b0f_0 208 KB
mkl_fft-1.0.15 | py37ha843d7b_0 172 KB
mkl_random-1.1.1 | py37h0573a6f_0 375 KB
numpy-1.18.1 | py37h4f9e942_0 5 KB
numpy-base-1.18.1 | py37hde5b4d6_1 5.2 MB
pandas-1.0.3 | py37h0573a6f_0 11.1 MB
python-dateutil-2.8.1 | py_0 224 KB
pytz-2020.1 | py_0 239 KB
six-1.15.0 | py_0 13 KB
------------------------------------------------------------
Total: 220.2 MB
The following NEW packages will be INSTALLED:
blas pkgs/main/linux-64::blas-1.0-mkl
intel-openmp pkgs/main/linux-64::intel-openmp-2020.1-217
libgfortran-ng pkgs/main/linux-64::libgfortran-ng-7.3.0-hdf63c60_0
mkl pkgs/main/linux-64::mkl-2020.1-217
mkl-service pkgs/main/linux-64::mkl-service-2.3.0-py37he904b0f_0
mkl_fft pkgs/main/linux-64::mkl_fft-1.0.15-py37ha843d7b_0
mkl_random pkgs/main/linyux-64::mkl_random-1.1.1-py37h0573a6f_0
numpy pkgs/main/linux-64::numpy-1.18.1-py37h4f9e942_0
numpy-base pkgs/main/linux-64::numpy-base-1.18.1-py37hde5b4d6_1
pandas pkgs/main/linux-64::pandas-1.0.3-py37h0573a6f_0
python-dateutil pkgs/main/noarch::python-dateutil-2.8.1-py_0
pytz pkgs/main/noarch::pytz-2020.1-py_0
six pkgs/main/noarch::six-1.15.0-py_0
Proceed ([y]/n)? y
Downloading and Extracting Packages
python-dateutil-2.8. | 224 KB | ################################################################## | 100%
pytz-2020.1 | 239 KB | ################################################################## | 100%
six-1.15.0 | 13 KB | ################################################################## | 100%
numpy-1.18.1 | 5 KB | ################################################################## | 100%
mkl_fft-1.0.15 | 172 KB | ################################################################## | 100%
mkl-service-2.3.0 | 208 KB | ################################################################## | 100%
numpy-base-1.18.1 | 5.2 MB | ################################################################## | 100%
mkl_random-1.1.1 | 375 KB | ################################################################## | 100%
intel-openmp-2020.1 | 940 KB | ################################################################## | 100%
mkl-2020.1 | 200.5 MB | ################################################################## | 100%
blas-1.0 | 6 KB | ################################################################## | 100%
pandas-1.0.3 | 11.1 MB | ################################################################## | 100%
libgfortran-ng-7.3.0 | 1.3 MB | ################################################################## | 100%
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
(grib) -bash-4.2$ conda install -c conda-forge pygrib
Collecting package metadata: done
Solving environment: done
## Package Plan ##
environment location: /home/hqin/.conda/envs/grib
added / updated specs:
- pygrib
The following packages will be downloaded:
package | build
---------------------------|-----------------
bzip2-1.0.8 | h516909a_2 396 KB conda-forge
ca-certificates-2020.4.5.2 | hecda079_0 147 KB conda-forge
certifi-2020.4.5.2 | py37hc8dfbb8_0 152 KB conda-forge
curl-7.69.1 | h33f0ec9_0 137 KB conda-forge
eccodes-2.17.0 | hf05d9b7_2 3.9 MB conda-forge
hdf4-4.2.13 | hf30be14_1003 964 KB conda-forge
hdf5-1.10.6 |nompi_h3c11f04_100 3.0 MB conda-forge
jasper-1.900.1 | h07fcdf6_1006 286 KB conda-forge
jpeg-9d | h516909a_0 266 KB conda-forge
krb5-1.17.1 | h173b8e3_0 1.5 MB
libaec-1.0.4 | he1b5a44_1 32 KB conda-forge
libcurl-7.69.1 | hf7181ac_0 573 KB conda-forge
libnetcdf-4.7.4 |nompi_h84807e1_104 1.3 MB conda-forge
libpng-1.6.37 | hed695b0_1 308 KB conda-forge
libssh2-1.9.0 | hab1572f_2 298 KB conda-forge
libtiff-4.1.0 | hc7e4089_6 668 KB conda-forge
libwebp-base-1.1.0 | h516909a_3 845 KB conda-forge
lz4-c-1.9.2 | he1b5a44_1 226 KB conda-forge
openssl-1.1.1g | h516909a_0 2.1 MB conda-forge
proj-7.0.0 | h966b41f_4 3.7 MB conda-forge
pygrib-2.0.4 | py37h55c65ac_0 341 KB conda-forge
pyproj-2.6.1.post1 | py37h34dd122_0 438 KB conda-forge
python_abi-3.7 | 1_cp37m 4 KB conda-forge
zstd-1.4.4 | h6597ccf_3 991 KB conda-forge
------------------------------------------------------------
Total: 22.4 MB
The following NEW packages will be INSTALLED:
bzip2 conda-forge/linux-64::bzip2-1.0.8-h516909a_2
curl conda-forge/linux-64::curl-7.69.1-h33f0ec9_0
eccodes conda-forge/linux-64::eccodes-2.17.0-hf05d9b7_2
hdf4 conda-forge/linux-64::hdf4-4.2.13-hf30be14_1003
hdf5 conda-forge/linux-64::hdf5-1.10.6-nompi_h3c11f04_100
jasper conda-forge/linux-64::jasper-1.900.1-h07fcdf6_1006
jpeg conda-forge/linux-64::jpeg-9d-h516909a_0
krb5 pkgs/main/linux-64::krb5-1.17.1-h173b8e3_0
libaec conda-forge/linux-64::libaec-1.0.4-he1b5a44_1
libcurl conda-forge/linux-64::libcurl-7.69.1-hf7181ac_0
libnetcdf conda-forge/linux-64::libnetcdf-4.7.4-nompi_h84807e1_104
libpng conda-forge/linux-64::libpng-1.6.37-hed695b0_1
libssh2 conda-forge/linux-64::libssh2-1.9.0-hab1572f_2
libtiff conda-forge/linux-64::libtiff-4.1.0-hc7e4089_6
libwebp-base conda-forge/linux-64::libwebp-base-1.1.0-h516909a_3
lz4-c conda-forge/linux-64::lz4-c-1.9.2-he1b5a44_1
proj conda-forge/linux-64::proj-7.0.0-h966b41f_4
pygrib conda-forge/linux-64::pygrib-2.0.4-py37h55c65ac_0
pyproj conda-forge/linux-64::pyproj-2.6.1.post1-py37h34dd122_0
python_abi conda-forge/linux-64::python_abi-3.7-1_cp37m
zstd conda-forge/linux-64::zstd-1.4.4-h6597ccf_3
The following packages will be UPDATED:
ca-certificates pkgs/main::ca-certificates-2020.1.1-0 --> conda-forge::ca-certificates-2020.4.5.2-hecda079_0
certifi pkgs/main::certifi-2020.4.5.1-py37_0 --> conda-forge::certifi-2020.4.5.2-py37hc8dfbb8_0
The following packages will be SUPERSEDED by a higher-priority channel:
openssl pkgs/main::openssl-1.1.1g-h7b6447c_0 --> conda-forge::openssl-1.1.1g-h516909a_0
Proceed ([y]/n)? y
Downloading and Extracting Packages
python_abi-3.7 | 4 KB | ################################################################## | 100%
lz4-c-1.9.2 | 226 KB | ################################################################## | 100%
zstd-1.4.4 | 991 KB | ################################################################## | 100%
ca-certificates-2020 | 147 KB | ################################################################## | 100%
jpeg-9d | 266 KB | ################################################################## | 100%
openssl-1.1.1g | 2.1 MB | ################################################################## | 100%
eccodes-2.17.0 | 3.9 MB | ################################################################## | 100%
krb5-1.17.1 | 1.5 MB | ################################################################## | 100%
libnetcdf-4.7.4 | 1.3 MB | ################################################################## | 100%
libpng-1.6.37 | 308 KB | ################################################################## | 100%
libtiff-4.1.0 | 668 KB | ################################################################## | 100%
hdf4-4.2.13 | 964 KB | ################################################################## | 100%
certifi-2020.4.5.2 | 152 KB | ################################################################## | 100%
pygrib-2.0.4 | 341 KB | ################################################################## | 100%
proj-7.0.0 | 3.7 MB | ################################################################## | 100%
hdf5-1.10.6 | 3.0 MB | ################################################################## | 100%
pyproj-2.6.1.post1 | 438 KB | ################################################################## | 100%
libcurl-7.69.1 | 573 KB | ################################################################## | 100%
libwebp-base-1.1.0 | 845 KB | ################################################################## | 100%
curl-7.69.1 | 137 KB | ################################################################## | 100%
bzip2-1.0.8 | 396 KB | ################################################################## | 100%
libssh2-1.9.0 | 298 KB | ################################################################## | 100%
libaec-1.0.4 | 32 KB | ################################################################## | 100%
jasper-1.900.1 | 286 KB | ################################################################## | 100%
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
(grib) -bash-4.2$ python
Python 3.7.7 (default, May 7 2020, 21:25:33)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pygrib
>>> import pandas
>>>
Subscribe to:
Posts (Atom)