Thursday, January 31, 2019

pbs for 10K ms02 on ts117





#!/bin/bash -l
#$ -S /bin/bash
#$ -t 1-100
#$ -N MS02.test
#$ -cwd

. /etc/profile.d/modules.sh
module load shared
module load R/3.4.3

cmd="R -f ms02_driver_20180523.R --args Data/yeast.pin.csv  yeastMS02/ms02.$SGE_TASK_ID.csv 100 0.025 9"
$cmd

ridgesize running error, sda1 space out


I run 10 jobs on ridgesize, with debuging output to text file.  These text files run up to 450G in size, and take all spaces in /tmp/ in /dev/sda1/.

Wednesday, January 30, 2019

*** genomic databases, biobanks,

https://agingresearchbiobank.nia.nih.gov/studies/life/


repositive.io

include 23&me, 1000 genomes, Allen institute, arrayexpress, corpasome

danish diabetes study, human DNA methylation

dbGap

EMI metagenomics

Encylopedia of DNA elements

Estonis biocentre
Europoean Aggregation consoritum

Expression atlas

Babriel

GEO
Genome Austria
Genome Asia 100K
Genome in a bottle
Genomes Unzipped
Giant consoritum
Horizon
Human genome diversity project
Human knockout project
insilicoDB
integrative japanese genome variant project

IHGC, international headache genetics consortium

DDBJ

Kadoorie biobank

Majic consoritum
Methylome DB

Mike's Genome Mike Lin

MTB, mouse tumor biology database

NGSmethDB
NIAGADS, NIA Alzheimer's Disease

Open Humans
Personal genome project
Repositive
SRA
Simons genome diversity project
Singapore Genome Variation project
Steven's Keating's genome
TACCF
Texax biobank
GONL
Biobank Finland
UCSC
Cancer Methylome system
Expressomics

Tuesday, January 29, 2019

file transfer speed, simcenter to home


Speed ~7Mb / second
$ scp xx@xx.simcenter.utc.edu:./yeastMS02.tar.gz .

Monday, January 28, 2019

bash on ridgeside (simcenter)


Simcenter default shell is not bash. So, calling bash should use 'bash' not sh

mac startup list change, remove citrix receiver




applejack:LaunchAgents hqin$ pwd

/Library/LaunchAgents


com.adobe.AAM.Updater-1.0.plist
com.adobe.ARMDCHelper.cc24aef4a1b90ed56a725c38014c95072f92651fb65e1bf9c8e43c37a23d420d.plist
com.bradfordnetworks.bncsaui.plist
com.citrix.AuthManager_Mac.plist
com.citrix.ReceiverHelper.plist
com.citrix.ServiceRecords.plist
com.google.keystone.agent.plist
com.jamf.management.agent.plist
com.jamfsoftware.jamf.agent.plist
com.microsoft.update.agent.plist
com.oracle.java.Java-Updater.plist
com.paloaltonetworks.gp.pangpa.plist
com.paloaltonetworks.gp.pangps.plist
com.paragon-software.NTFS.fsnotify.agent.plist
com.paragon-software.facebook.agent.plist
com.pharos.notify.plist
com.pharos.popup.plist
org.chromium.chromoting.plist
org.macosforge.xquartz.startx.plist
applejack:LaunchAgents hqin$ less com.citrix.AuthManager_Mac.plist
applejack:LaunchAgents hqin$ grep vpn *
applejack:LaunchAgents hqin$ grep VPN *
applejack:LaunchAgents hqin$ pwd
/Library/LaunchAgents
applejack:LaunchAgents hqin$ 
applejack:LaunchAgents hqin$ 
applejack:LaunchAgents hqin$ pwd
/Library/LaunchAgents




 sudo nano -w com.citrix.AuthManager_Mac.plist
// change the key value to false, based on
https://superuser.com/questions/885273/how-can-i-stop-checkpoints-vpn-program-from-automatically-starting-on-my-mac-t

did not work, try this:

 sudo nano -w com.citrix.ReceiverHelper.plist

        <key>RunAtLoad</key>
        <false/> #change from true to false

still not work, try to remove software at /usr/local/libexec/ReceiverHelper.app

applejack:~ hqin$ cd /usr/local/libexec/
applejack:libexec hqin$ ls
AuthManager_Mac.app ReceiverHelper.app ServiceRecords.app
applejack:libexec hqin$ ll
total 0
drwxr-xr-x  3 root  wheel   102B May  1  2017 AuthManager_Mac.app
drwxr-xr-x  3 root  wheel   102B May  1  2017 ReceiverHelper.app
drwxr-xr-x  3 root  wheel   102B May  1  2017 ServiceRecords.app
applejack:libexec hqin$ less ReceiverHelper.app/
ReceiverHelper.app/ is a directory
applejack:libexec hqin$ cd ReceiverHelper.app/
applejack:ReceiverHelper.app hqin$ ls
Contents
applejack:ReceiverHelper.app hqin$ cd Contents/
applejack:Contents hqin$ ls
Frameworks MacOS Resources
Info.plist PkgInfo _CodeSignature
applejack:Contents hqin$ cd MacOS/
applejack:MacOS hqin$ ls
ReceiverHelper
applejack:MacOS hqin$ less ReceiverHelper 
"ReceiverHelper" may be a binary file.  See it anyway? 
applejack:MacOS hqin$ pwd
/usr/local/libexec/ReceiverHelper.app/Contents/MacOS
applejack:MacOS hqin$ sudo mv ReceiverHelper ReceiverHelper.trash
Password:
applejack:MacOS hqin$ pwd

/usr/local/libexec/ReceiverHelper.app/Contents/MacOS

Finally, this removed citrix receiver from startup.


Janssens elife 2015



Figure 2—source data 1

Table S2: The shotgun proteome data processing.
https://doi.org/10.7554/eLife.08527.011
Download elife-08527-fig2-data1-v2.xlsx
data have both mother and daughters, values provided by ORFs
Q: what does fitted mean?
Figure 2—source data 2
Table S3: The transcriptome data processing.
https://doi.org/10.7554/eLife.08527.012
Download elife-08527-fig2-data2-v2.xlsx
data have both mother and daughters, values provided by ORFs
Q: what does fitted mean? 
Figure 2—source data 3 (only mothers cells) 
Table S4: The final shotgun proteome data.
https://doi.org/10.7554/eLife.08527.013
Download elife-08527-fig2-data3-v2.xlsx

Figure 2—source data 4 (only mother cells)
Table S5: The final transcriptome data.
https://doi.org/10.7554/eLife.08527.014
Download elife-08527-fig2-data4-v2.xlsx

EMCS spaces


206 , lab class, 1420 sq ft
205G, student study room
205, suite,
205F, office,
204, graduate teaching

4th floor, office rooms,

Sunday, January 27, 2019

*** Socrative responses to Canvas grade book

Save all socrative report as xlsx files.

Rename all report files using the dates.

Using a Jupyter notebook to analyze NetworkdID in each file, summarize them in Pandas dataframe, output to Excel file.

Create assignment on Canvas for attendance by dates. (This step was skipped on 20190220).
Then Export gradebook from Canvas as csv.

Update the grade csv files. (Make sure the point row was preserved. This can be done to sort the grade on a separate sheet, and then copy the sorted and merged grade back).

Upload grade csv files to Canvas gradebook. Visually check some changes. Wait for grades change to be updated.

In [1]:
import os
import pandas as pd
debug = 1

load class roster from gradebook

In [2]:
mygradebook = '../grades/2019-01-26T2215_Grades-Data_Structures_and_Program_Design.csv'
In [3]:
import csv
#with open(mygradebook, newline='') as csvfile:
#    grades = csv.reader(csvfile, delimiter=' ', quotechar='|')
csvfile = open(mygradebook, newline='')
grades = csv.reader(csvfile)
name2id = {}
id2name = {}
for row in grades:
        print(row[0] +  "\t\t" + row[2])
        if (row[0] != "Student") & (row[0] != "Test Student") & (row[0] !="    Points Possible"):
            name2id[row[0]] = row[2]
            id2name[row[2]] = row[0]

In [5]:
id2name
Out[5]:

In [6]:
grades.line_num
Out[6]:
27
In [7]:
len(id2name)
Out[7]:
24
In [8]:
id2attendance = {} #attendance dictionary
In [9]:
report_df = pd.DataFrame(pd.Series(id2name), columns=["name"])
report_df
Out[9]:

read socrative reports

In [10]:
os.chdir("/Users/hqin/github/socrative/Socrative")
mypath = 'cpsc1110Sp2019_cleaned'
filenames = os.listdir(mypath) # returns list
filenames.sort()
print(filenames)
['01_08_2019.xlsx', '01_10_2019.xlsx', '01_15_2019.xlsx', '01_17_2019.xlsx', '01_22_2019.xlsx', '01_24_2019.xlsx']
In [11]:
from openpyxl import load_workbook
for i in range(len(filenames)): 
#for i in range(0,1): 
    myfilename = filenames[i]
    mycolumnlabel = myfilename.replace(".xlsx","")
    if (debug > 0):
                print("i=" , i , "myfilename =" , myfilename )
    wb = load_workbook(filename = mypath + "/" + myfilename)
    ws = wb["Sheet1"]

    unqiue_netids = set()
    for i in range(1,50):
        current_value = ws.cell(row=i, column=5).value
        if type(current_value) == type("x"):
            current_value = current_value.upper().replace(' ','') #remove white spaces
            #if (debug > 0):
                #print( current_value )
            unqiue_netids.add(current_value)
    
    for id in id2name: 
        if id in unqiue_netids: 
            id2attendance[ id ] =  1
        else:
            id2attendance[ id ] =  0   
    
    print(id2attendance)

    report_df[mycolumnlabel] = pd.Series(id2attendance,index=id2attendance.keys() )

In [12]:
report_df
Out[12]:
In [13]:
report_df.to_excel("socrative_summary.xlsx", sheet_name='scorative_report', index_label ="netid")  # doctest: +SKIP
In [ ]:

python pandas, dataframe, csv



https://medium.com/@kasiarachuta/importing-and-exporting-csv-files-in-python-7fa6e4d9f408


Friday, January 25, 2019

Zombie Aqua die, viability assay

Zombie Aqua die, viability assay

Cell Lines and Drugs Cultured P12 and U937 cell lines were grown to confluence. Two million cells/mL were plated into a 24 well tissue culture plate with or without 20 mg/mL of mitomycin-c for 48 hrs at 37°Celsius in complete-RPMI medium + 5% fetal calf serum. Viability Staining & ViCell Counting Treated cells were washed in 10 times the volume of PBS, specifically chosen since it lacks protein, at 400g for 5  minutes. An aliquot of each cell line and treatment condition was counted on the ViCell. One million cells from each cell line and treatment condition were stained in 100 µL final volume with either 1 µL of Zombie Yellow, 1 µL of Zombie Aqua, or 20 µL of 7-AAD. All conditions were stained for 30 minutes. Then, all tubes were washed in 3 mL of growth media and spun at 400g for 5 minutes. Cell pellets were resuspended in PBS and acquired on the CytoFLEX. Results Of the three viability dyes tested, 7-AAD produced the highest signal to noise (S:N) ratio (Figure 1). Each cell line and treatment group showed similar trends in percentage viable cells with the addition of mitomycin-c or Ly-294. Additionally, each viability dye correlated to the results generated by the ViCell (Figure 2). Although the percentages of viable cells differed slightly between the viability dyes tested, it is possible that this is due to the altering mechanisms of actions of the viability dye binding.

These data suggest that it is necessary to be consistent in individual experiments with a singular viability dye of choice.

References 1- Perfetto SP, Chattopadhyay PK, Lamoreaux L, Nguyen R, Ambrozak D, Roup RA, Roederer M. Amine-reactive dyes for dead cell discrimination in fixed samples. Curr. Protoc. Cytom. Chapter 9: Unit 9.34, 2010.

Reagent Details Reagent Supplier Catalog No. 7-AAD Beckman Coulter A07704 Mitomycin-C Sigma M4287 Ly-294 Sigma L9908 Zombie Yellow BioLegend 423103 Zombie Aqua BioLegend 423101

Wednesday, January 23, 2019

Opentron OT2


Please follow these tutorials for setting up the OT-2: Unboxing the OT-2Unlocking the OT-2

You will also want to download our app and get started by running a basic protocol, e.g. Opentrons Logo Protocol and Customizable serial dilution.

The support website is the best place to answer most common questions that come with getting started. If you have any technical questions, please reach out to technical support over Intercom, the little blue chat button on the bottom-righthand corner of the OT-2 app and our website.

Thank you and let me know if you have any questions!

Saturday, January 19, 2019

sample lab websites at GitHub

Sample lab websites at GitHub

https://cellgeometry.ucsf.edu/
https://github.com/cellgeometry/cellgeometry.github.io

https://fraserlab.com/
https://github.com/fraser-lab/fraser-lab.github.io


http://zunderlab.com/
https://github.com/zunderlab/zunderlab.github.io

https://hallettlab.netlify.com/ 
https://github.com/HallettLab/lab-website


customerized hclustfun or distfun in heatmap


for( mymethod in mymethods ) {
 hd =  hclust( dist(ctb2), mymethod); 
 # plot( hd, main="hamming distance, ward linkage" )
 coat.cat = cutree(hd, numclus )  ###<=== change is here
 col.palette = c("red","brown","blue","green");
 coat.color = col.palette[coat.cat]

 library(RColorBrewer);
 #hmcol = colorRampPalette(brewer.pal(10,"RdBu"))(256);
 hmcol = colorRampPalette(brewer.pal(5,"RdBu"))(16);

 #heatmap( ctb2, col=hmcol, scale="none", margins = c(5,10) );
 heatmap( ctb2, col=hmcol, scale="none", margins = c(5,10), 
 RowSideColors=coat.color, ColSideColors = spec.colors,
 hclustfun = function(c) hclust( c, method=mymethod),

 distfun = function(c) as.dist(hamming.distance(c)) #Hamming is less pleasant than Euclidean 

 main = mymethod
 );
}

CR prot GO results, top 0.5% of Dang's ratio


These data can be used set up simulation studies.



Showing 142 to 160 of 160 entries