Pipeline run code and environment:
- Command:
/scratch/jps3dp/tools/databio//peppro/pipelines/peppro.py --sample-name H9_PRO-seq_1 --genome hg38 --input /project/shefflab/data//guertin/fastq/H9_DMSO_rep1_PE1.fastq.gz --single-or-paired PAIRED -O /project/shefflab/processed//peppro/paper/6.11.2020/results_pipeline -P 12 -M 12000 --input2 /project/shefflab/data//guertin/fastq/H9_DMSO_rep1_PE2.fastq.gz --protocol PRO --umi-len 8 --scale --prealignments human_rDNA --dirty -R
- Compute host: udc-ba25-34c0
- Working dir: /sfs/lustre/bahamut/scratch/jps3dp/tools/databio/ppqc
- Outfolder: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/
- Pipeline started at: (06-15 07:17:16) elapsed: 1.0 TIME
Version log:
- Python version: 3.6.6
- Pypiper dir:
/sfs/qumulo/qhome/jps3dp/.local/lib/python3.6/site-packages/pypiper
- Pypiper version: 0.12.1
- Pipeline dir:
/sfs/lustre/bahamut/scratch/jps3dp/tools/databio/peppro/pipelines
- Pipeline version: 0.9.8
- Pipeline hash: 887a9a85408962dc3ca070dc954e59a5d4d73a10
- Pipeline branch: * master
- Pipeline date: 2020-06-09 11:36:49 -0400
- Pipeline diff: 40 files changed, 16413 insertions(+), 3702 deletions(-)
Arguments passed to pipeline:
TSS_name
:None
adapter
:cutadapt
anno_name
:None
complexity
:False
config_file
:peppro.yaml
cores
:12
coverage
:False
dedup
:seqkit
dirty
:True
ensembl_gene_body
:None
ensembl_tss
:None
exon_name
:None
force_follow
:False
genome_assembly
:hg38
input
:['/project/shefflab/data//guertin/fastq/H9_DMSO_rep1_PE1.fastq.gz']
input2
:['/project/shefflab/data//guertin/fastq/H9_DMSO_rep1_PE2.fastq.gz']
intron_name
:None
keep
:False
logdev
:False
max_len
:-1
mem
:12000
new_start
:False
no_fifo
:False
output_parent
:/project/shefflab/processed//peppro/paper/6.11.2020/results_pipeline
paired_end
:True
pre_name
:None
prealignments
:['human_rDNA']
prioritize
:False
protocol
:PRO
recover
:True
sample_name
:H9_PRO-seq_1
scale
:True
search_file
:None
silent
:False
single_or_paired
:PAIRED
sob
:False
testmode
:False
trimmer
:seqtk
umi_len
:8
verbosity
:None
Local input file: /project/shefflab/data//guertin/fastq/H9_DMSO_rep1_PE1.fastq.gz Local input file: /project/shefflab/data//guertin/fastq/H9_DMSO_rep1_PE2.fastq.gz
File_mb
2469.73 PEPPRO RES
Read_type
PAIRED PEPPRO RES
Genome
hg38 PEPPRO RES Detected PRO input
Merge/link and fastq conversion: (06-15 07:17:17) elapsed: 2.0 TIME
Number of input file sets: 2
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/raw/H9_PRO-seq_1_R1.fastq.gz
ln -sf /project/shefflab/data//guertin/fastq/H9_DMSO_rep1_PE1.fastq.gz /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/raw/H9_PRO-seq_1_R1.fastq.gz
(261655)Command completed. Elapsed time: 0:00:00. Running peak memory: 0GB.
PID: 261655; Command: ln; Return code: 0; Memory used: 0.0GB
Local input file: '/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/raw/H9_PRO-seq_1_R1.fastq.gz'
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/raw/H9_PRO-seq_1_R2.fastq.gz
ln -sf /project/shefflab/data//guertin/fastq/H9_DMSO_rep1_PE2.fastq.gz /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/raw/H9_PRO-seq_1_R2.fastq.gz
(261657)Command completed. Elapsed time: 0:00:00. Running peak memory: 0.0GB.
PID: 261657; Command: ln; Return code: 0; Memory used: 0.0GB
Local input file: '/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/raw/H9_PRO-seq_1_R2.fastq.gz'
Found .fastq.gz file
Found .fastq.gz file
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R1.fastq
,/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R2.fastq
pigz -f -p 12 -d -c /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/raw/H9_PRO-seq_1_R1.fastq.gz > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R1.fastq
(261662)Command completed. Elapsed time: 0:02:40. Running peak memory: 0.002GB.
PID: 261662; Command: pigz; Return code: 0; Memory used: 0.002GB
pigz -f -p 12 -d -c /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/raw/H9_PRO-seq_1_R2.fastq.gz > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R2.fastq
(41970)Command completed. Elapsed time: 0:01:34. Running peak memory: 0.002GB.
PID: 41970; Command: pigz; Return code: 0; Memory used: 0.002GB
Raw_reads
97729860 PEPPRO RES
Fastq_reads
97729860 PEPPRO RES ['/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/raw/H9_PRO-seq_1_R1.fastq.gz', '/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/raw/H9_PRO-seq_1_R2.fastq.gz']
FASTQ processing: (06-15 07:23:33) elapsed: 376.0 TIME
cutadapt --version
Target to produce:/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R1_processed.fastq
(cutadapt -j 12 -m 10 -O 1 -a TGGAATTCTCGGGTGCCAAGG /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R1.fastq -o /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R1_noadap.fastq --too-short-output /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R1_short.fastq ) > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/cutadapt/H9_PRO-seq_1_R1_cutadapt.txt
(233666)Command completed. Elapsed time: 0:01:19. Running peak memory: 3.252GB.
PID: 233666; Command: cutadapt; Return code: 0; Memory used: 3.252GB
seqtk trimfq -b 8 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R1_noadap.fastq | seqtk seq -L 10 -r - > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R1_processed.fastq
(233763,233764)Command completed. Elapsed time: 0:01:10. Running peak memory: 3.252GB.
PID: 233763; Command: seqtk; Return code: 0; Memory used: 0.001GB
PID: 233764; Command: seqtk; Return code: 0; Memory used: 0.002GB
Evaluating read trimming
Trimmed_reads
30221343 PEPPRO RES
Trim_loss_rate
69.08 PEPPRO RES Targetless command, running...
fastqc --noextract --outdir /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastqc /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R1_processed.fastq
(234065)Got SIGTERM. Failing gracefully... (06-15 12:57:44) elapsed: 20051.0 TIME Child process 234065 (fastqc) terminated after 0 sec. Setting dynamic recover file: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/recover.lock.fastq__H9_PRO-seq_1_R1_processed.fastq Setting dynamic recover file: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/recover.lock.trimmed_fastqc
Pipeline failed at: (06-15 12:57:44) elapsed: 0.0 TIME
Total time: 5:40:29 Failure reason: SIGTERM Pipeline aborted.
Pipeline run code and environment:
- Command:
/scratch/jps3dp/tools/databio//peppro/pipelines/peppro.py --sample-name H9_PRO-seq_1 --genome hg38 --input /project/shefflab/data//guertin/fastq/H9_DMSO_rep1_PE1.fastq.gz --single-or-paired PAIRED -O /project/shefflab/processed//peppro/paper/6.11.2020/results_pipeline -P 12 -M 12000 --input2 /project/shefflab/data//guertin/fastq/H9_DMSO_rep1_PE2.fastq.gz --protocol PRO --umi-len 8 --scale --prealignments human_rDNA --dirty -R
- Compute host: udc-ba25-12c0
- Working dir: /sfs/lustre/bahamut/scratch/jps3dp/tools/databio/ppqc
- Outfolder: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/
- Pipeline started at: (06-15 12:59:45) elapsed: 36.0 TIME
Version log:
- Python version: 3.6.6
- Pypiper dir:
/sfs/qumulo/qhome/jps3dp/.local/lib/python3.6/site-packages/pypiper
- Pypiper version: 0.12.1
- Pipeline dir:
/sfs/lustre/bahamut/scratch/jps3dp/tools/databio/peppro/pipelines
- Pipeline version: 0.9.8
- Pipeline hash: 887a9a85408962dc3ca070dc954e59a5d4d73a10
- Pipeline branch: * master
- Pipeline date: 2020-06-09 11:36:49 -0400
- Pipeline diff: 40 files changed, 16413 insertions(+), 3702 deletions(-)
Arguments passed to pipeline:
TSS_name
:None
adapter
:cutadapt
anno_name
:None
complexity
:False
config_file
:peppro.yaml
cores
:12
coverage
:False
dedup
:seqkit
dirty
:True
ensembl_gene_body
:None
ensembl_tss
:None
exon_name
:None
force_follow
:False
genome_assembly
:hg38
input
:['/project/shefflab/data//guertin/fastq/H9_DMSO_rep1_PE1.fastq.gz']
input2
:['/project/shefflab/data//guertin/fastq/H9_DMSO_rep1_PE2.fastq.gz']
intron_name
:None
keep
:False
logdev
:False
max_len
:-1
mem
:12000
new_start
:False
no_fifo
:False
output_parent
:/project/shefflab/processed//peppro/paper/6.11.2020/results_pipeline
paired_end
:True
pre_name
:None
prealignments
:['human_rDNA']
prioritize
:False
protocol
:PRO
recover
:True
sample_name
:H9_PRO-seq_1
scale
:True
search_file
:None
silent
:False
single_or_paired
:PAIRED
sob
:False
testmode
:False
trimmer
:seqtk
umi_len
:8
verbosity
:None
Local input file: /project/shefflab/data//guertin/fastq/H9_DMSO_rep1_PE1.fastq.gz Local input file: /project/shefflab/data//guertin/fastq/H9_DMSO_rep1_PE2.fastq.gz
File_mb
2469.73 PEPPRO RES
Read_type
PAIRED PEPPRO RES
Genome
hg38 PEPPRO RES Detected PRO input
Merge/link and fastq conversion: (06-15 12:59:46) elapsed: 1.0 TIME
Number of input file sets: 2
Target exists: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/raw/H9_PRO-seq_1_R1.fastq.gz
Local input file: '/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/raw/H9_PRO-seq_1_R1.fastq.gz'
Target exists: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/raw/H9_PRO-seq_1_R2.fastq.gz
Local input file: '/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/raw/H9_PRO-seq_1_R2.fastq.gz'
Found .fastq.gz file
Found .fastq.gz file
Target exists: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R1.fastq
Target exists: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R2.fastq
['/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/raw/H9_PRO-seq_1_R1.fastq.gz', '/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/raw/H9_PRO-seq_1_R2.fastq.gz']
FASTQ processing: (06-15 12:59:46) elapsed: 0.0 TIME
cutadapt --version
Found lock file: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/lock.fastq__H9_PRO-seq_1_R1_processed.fastq Overwriting target... Target to produce:/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R1_processed.fastq
(cutadapt -j 12 -m 10 -O 1 -a TGGAATTCTCGGGTGCCAAGG /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R1.fastq -o /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R1_noadap.fastq --too-short-output /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R1_short.fastq ) > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/cutadapt/H9_PRO-seq_1_R1_cutadapt.txt
(14926)Command completed. Elapsed time: 0:00:59. Running peak memory: 3.258GB.
PID: 14926; Command: cutadapt; Return code: 0; Memory used: 3.258GB
seqtk trimfq -b 8 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R1_noadap.fastq | seqtk seq -L 10 -r - > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R1_processed.fastq
(15390,15391)Command completed. Elapsed time: 0:00:33. Running peak memory: 3.258GB.
PID: 15390; Command: seqtk; Return code: 0; Memory used: 0.001GB
PID: 15391; Command: seqtk; Return code: 0; Memory used: 0.002GB
Evaluating read trimming
Trimmed_reads
30221343 PEPPRO RES
Trim_loss_rate
69.08 PEPPRO RES Found lock file: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/lock.trimmed_fastqc Overwriting target... Targetless command, running...
fastqc --noextract --outdir /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastqc /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R1_processed.fastq
(15447)Started analysis of H9_PRO-seq_1_R1_processed.fastq Approx 5% complete for H9_PRO-seq_1_R1_processed.fastq Approx 10% complete for H9_PRO-seq_1_R1_processed.fastq Approx 15% complete for H9_PRO-seq_1_R1_processed.fastq Approx 20% complete for H9_PRO-seq_1_R1_processed.fastq Approx 25% complete for H9_PRO-seq_1_R1_processed.fastq Approx 30% complete for H9_PRO-seq_1_R1_processed.fastq Approx 35% complete for H9_PRO-seq_1_R1_processed.fastq Approx 40% complete for H9_PRO-seq_1_R1_processed.fastq Approx 45% complete for H9_PRO-seq_1_R1_processed.fastq Approx 50% complete for H9_PRO-seq_1_R1_processed.fastq Approx 55% complete for H9_PRO-seq_1_R1_processed.fastq Approx 60% complete for H9_PRO-seq_1_R1_processed.fastq Approx 65% complete for H9_PRO-seq_1_R1_processed.fastq Approx 70% complete for H9_PRO-seq_1_R1_processed.fastq Approx 75% complete for H9_PRO-seq_1_R1_processed.fastq Approx 80% complete for H9_PRO-seq_1_R1_processed.fastq Approx 85% complete for H9_PRO-seq_1_R1_processed.fastq Approx 90% complete for H9_PRO-seq_1_R1_processed.fastq Approx 95% complete for H9_PRO-seq_1_R1_processed.fastq Analysis complete for H9_PRO-seq_1_R1_processed.fastqCommand completed. Elapsed time: 0:00:57. Running peak memory: 3.258GB.
PID: 15447; Command: fastqc; Return code: 0; Memory used: 0.182GB
FastQC report r1
fastqc/H9_PRO-seq_1_R1_processed_fastqc.html FastQC report r1 None PEPPRO OBJ Target to produce:/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R1_trimmed.fastq
seqkit rmdup --threads 12 --by-seq --ignore-case -o /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R1_dedup.fastq /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R1_noadap.fastq
(15527)[INFO][0m 3516802 duplicated records removedCommand completed. Elapsed time: 0:00:56. Running peak memory: 3.258GB.
PID: 15527; Command: seqkit; Return code: 0; Memory used: 2.002GB
seqtk trimfq -b 8 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R1_dedup.fastq | seqtk seq -L 10 -r - > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R1_trimmed.fastq
(15609,15610)Command completed. Elapsed time: 0:00:24. Running peak memory: 3.258GB.
PID: 15609; Command: seqtk; Return code: 0; Memory used: 0.001GB
PID: 15610; Command: seqtk; Return code: 0; Memory used: 0.002GB
grep 'Reads with adapters:' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/cutadapt/H9_PRO-seq_1_R1_cutadapt.txt | awk '{print $(NF-1)}'
Reads_with_adapter
34822522.0 PEPPRO RES
grep 'Total basepairs processed:' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/cutadapt/H9_PRO-seq_1_R1_cutadapt.txt | awk '{print $(NF-1)}'
awk '{sum+=$1*$2} END {printf "%.0f", sum}' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/cutadapt/H9_PRO-seq_1_R1_cutadapt.txt
wc -l /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R1_short.fastq | awk '{print $1}'
Uninformative_adapter_reads
17998987.0 PEPPRO RES
Duplicate_reads
3516802.0 PEPPRO RES
Pct_uninformative_adapter_reads
36.8342 PEPPRO RES Target to produce:/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/processed_R1.flag
touch /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/processed_R1.flag
(15713)Command completed. Elapsed time: 0:00:00. Running peak memory: 3.258GB.
PID: 15713; Command: touch; Return code: 0; Memory used: 0.0GB
cutadapt --version
Target to produce:/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R2_trimmed.fastq
(cutadapt -j 12 -m 10 -O 1 -a GATCGTCGGACTGTAGAACTCTGAAC /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R2.fastq -o /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R2_noadap.fastq --too-short-output /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R2_short.fastq ) > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/cutadapt/H9_PRO-seq_1_R2_cutadapt.txt
(15715)Command completed. Elapsed time: 0:00:54. Running peak memory: 3.432GB.
PID: 15715; Command: cutadapt; Return code: 0; Memory used: 3.432GB
seqtk trimfq -b 8 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R2_noadap.fastq | seqtk seq -L 10 -r - > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R2_trimmed.fastq
(15801,15802)Command completed. Elapsed time: 0:00:26. Running peak memory: 3.432GB.
PID: 15801; Command: seqtk; Return code: 0; Memory used: 0.001GB
PID: 15802; Command: seqtk; Return code: 0; Memory used: 0.002GB
Evaluating read trimming
Trimmed_reads
60442686 PEPPRO RES
Trim_loss_rate
38.15 PEPPRO RES Targetless command, running...
fastqc --noextract --outdir /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastqc /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R1_processed.fastq
(16066)Started analysis of H9_PRO-seq_1_R1_processed.fastq Approx 5% complete for H9_PRO-seq_1_R1_processed.fastq Approx 10% complete for H9_PRO-seq_1_R1_processed.fastq Approx 15% complete for H9_PRO-seq_1_R1_processed.fastq Approx 20% complete for H9_PRO-seq_1_R1_processed.fastq Approx 25% complete for H9_PRO-seq_1_R1_processed.fastq Approx 30% complete for H9_PRO-seq_1_R1_processed.fastq Approx 35% complete for H9_PRO-seq_1_R1_processed.fastq Approx 40% complete for H9_PRO-seq_1_R1_processed.fastq Approx 45% complete for H9_PRO-seq_1_R1_processed.fastq Approx 50% complete for H9_PRO-seq_1_R1_processed.fastq Approx 55% complete for H9_PRO-seq_1_R1_processed.fastq Approx 60% complete for H9_PRO-seq_1_R1_processed.fastq Approx 65% complete for H9_PRO-seq_1_R1_processed.fastq Approx 70% complete for H9_PRO-seq_1_R1_processed.fastq Approx 75% complete for H9_PRO-seq_1_R1_processed.fastq Approx 80% complete for H9_PRO-seq_1_R1_processed.fastq Approx 85% complete for H9_PRO-seq_1_R1_processed.fastq Approx 90% complete for H9_PRO-seq_1_R1_processed.fastq Approx 95% complete for H9_PRO-seq_1_R1_processed.fastq Analysis complete for H9_PRO-seq_1_R1_processed.fastq
A fatal error has been detected by the Java Runtime Environment:
SIGSEGV (0xb) at pc=0x00007f999e503631, pid=16066, tid=0x00007f9983bfb700
JRE version: Java(TM) SE Runtime Environment (8.0_171-b11) (build 1.8.0_171-b11)
Java VM: Java HotSpot(TM) 64-Bit Server VM (25.171-b11 mixed mode linux-amd64 compressed oops)
Problematic frame:
V [libjvm.so+0x4b3631] CompileBroker::invoke_compiler_on_method(CompileTask*)+0x5a1
Core dump written. Default location: /sfs/lustre/bahamut/scratch/jps3dp/tools/databio/ppqc/core or core.16066
An error report file with more information is saved as:
/sfs/lustre/bahamut/scratch/jps3dp/tools/databio/ppqc/hs_err_pid16066.log
If you would like to submit a bug report, please visit:
http://bugreport.java.com/bugreport/crash.jsp
Command completed. Elapsed time: 0:00:58. Running peak memory: 3.432GB.
PID: 16066; Command: fastqc; Return code: -6; Memory used: 0.18GB
Subprocess returned nonzero result. Check above output for details ERROR: Subprocess returned nonzero result, but pipeline is continuing because nofail=True
FastQC report r1
fastqc/H9_PRO-seq_1_R1_processed_fastqc.html FastQC report r1 None PEPPRO OBJ Targetless command, running...
fastqc --noextract --outdir /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastqc /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R2_trimmed.fastq
(16137)Started analysis of H9_PRO-seq_1_R2_trimmed.fastq Approx 5% complete for H9_PRO-seq_1_R2_trimmed.fastq Approx 10% complete for H9_PRO-seq_1_R2_trimmed.fastq Approx 15% complete for H9_PRO-seq_1_R2_trimmed.fastq Approx 20% complete for H9_PRO-seq_1_R2_trimmed.fastq Approx 25% complete for H9_PRO-seq_1_R2_trimmed.fastq Approx 30% complete for H9_PRO-seq_1_R2_trimmed.fastq Approx 35% complete for H9_PRO-seq_1_R2_trimmed.fastq Approx 40% complete for H9_PRO-seq_1_R2_trimmed.fastq Approx 45% complete for H9_PRO-seq_1_R2_trimmed.fastq Approx 50% complete for H9_PRO-seq_1_R2_trimmed.fastq Approx 55% complete for H9_PRO-seq_1_R2_trimmed.fastq Approx 60% complete for H9_PRO-seq_1_R2_trimmed.fastq Approx 65% complete for H9_PRO-seq_1_R2_trimmed.fastq Approx 70% complete for H9_PRO-seq_1_R2_trimmed.fastq Approx 75% complete for H9_PRO-seq_1_R2_trimmed.fastq Approx 80% complete for H9_PRO-seq_1_R2_trimmed.fastq Approx 85% complete for H9_PRO-seq_1_R2_trimmed.fastq Approx 90% complete for H9_PRO-seq_1_R2_trimmed.fastq Approx 95% complete for H9_PRO-seq_1_R2_trimmed.fastq Analysis complete for H9_PRO-seq_1_R2_trimmed.fastqCommand completed. Elapsed time: 0:00:54. Running peak memory: 3.432GB.
PID: 16137; Command: fastqc; Return code: 0; Memory used: 0.17GB
FastQC report r2
fastqc/H9_PRO-seq_1_R2_trimmed_fastqc.html FastQC report r2 None PEPPRO OBJ Target to produce:/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/cutadapt/H9_PRO-seq_1.hist
,/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/cutadapt/H9_PRO-seq_1.histogram
fastq_pair -t 87956874 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R1_noadap.fastq /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R2_noadap.fastq
(16204)Left paired: 30692275 Right paired: 30692275 Left single: 173668 Right single: 1557501 Writing the paired reads to /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R1_noadap.fastq.paired.fq and /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R2_noadap.fastq.paired.fq. Writing the single reads to /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R1_noadap.fastq.single.fq and /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R2_noadap.fastq.single.fqCommand completed. Elapsed time: 0:02:42. Running peak memory: 6.12GB.
PID: 16204; Command: fastq_pair; Return code: 0; Memory used: 6.12GB
flash -q -t 12 --compress-prog=pigz --suffix=gz /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R1_noadap.fastq.paired.fq /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R2_noadap.fastq.paired.fq -o H9_PRO-seq_1 -d /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/cutadapt
(16448)Command completed. Elapsed time: 0:00:59. Running peak memory: 6.12GB.
PID: 16448; Command: flash; Return code: 0; Memory used: 0.094GB
Plot adapter insertion distribution (06-15 13:10:55) elapsed: 669.0 TIME
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/cutadapt/H9_PRO-seq_1_adapter_insertion_distribution.pdf
Rscript /scratch/jps3dp/tools/databio//peppro/tools/PEPPRO.R adapt -i /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/cutadapt/H9_PRO-seq_1.hist -o /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/cutadapt -u 8
(16944)Adapter insertion distribution plot completed!
Command completed. Elapsed time: 0:00:05. Running peak memory: 6.12GB.
PID: 16944; Command: Rscript; Return code: 0; Memory used: 0.311GB
Adapter insertion distribution
cutadapt/H9_PRO-seq_1_adapter_insertion_distribution.pdf Adapter insertion distribution cutadapt/H9_PRO-seq_1_adapter_insertion_distribution.png PEPPRO OBJ Missing stat 'Peak_adapter_insertion_size'
awk 'NR>2 {print prev} {prev=$0}' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/cutadapt/H9_PRO-seq_1.hist | awk 'BEGIN{max= 0; max_len=0; len=0}{if ($2>0+max) {max=$2; len=$1}; max_len=$1} END{print len-8}'
Peak_adapter_insertion_size
20 PEPPRO RES Missing stat 'Degradation_ratio'
Calculating degradation ratio (06-15 13:11:00) elapsed: 5.0 TIME
awk '{ if (($1-8) == 10) {status = 1}} END {if (status) {print status} else {print 0}}' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/cutadapt/H9_PRO-seq_1.hist
awk '{ if (($1-8) == 20) {status = 1}} END {if (status) {print status} else {print 0}}' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/cutadapt/H9_PRO-seq_1.hist
awk '{ if (($1-8) == 30) {status = 1}} END {if (status) {print status} else {print 0}}' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/cutadapt/H9_PRO-seq_1.hist
awk '{ if (($1-8) == 40) {status = 1}} END {if (status) {print status} else {print 0}}' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/cutadapt/H9_PRO-seq_1.hist
awk '(($1-8 ) <= 20 && ($1-8 ) >= 10){degradedSum += $2}; (($1-8 ) >= 30 && ($1-8) <= 40){intactSum += $2} END {if (intactSum < 1) {intactSum = 1} print degradedSum/intactSum}' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/cutadapt/H9_PRO-seq_1.hist
Degradation_ratio
0.8896 PEPPRO RES Target to produce:/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R2_trimmed_dups.fastq
cp /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R2_trimmed.fastq /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R2_trimmed_dups.fastq
(16973)Command completed. Elapsed time: 0:00:21. Running peak memory: 6.12GB.
PID: 16973; Command: cp; Return code: 0; Memory used: 0.002GB
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/processed_R2.flag
touch /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/processed_R2.flag
(17341)Command completed. Elapsed time: 0:00:00. Running peak memory: 6.12GB.
PID: 17341; Command: touch; Return code: 0; Memory used: 0.0GB
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/repaired.flag
fastq_pair -t 87956874 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R1_processed.fastq /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R2_trimmed.fastq
(17348)Left paired: 30003416 Right paired: 30003416 Left single: 217927 Right single: 1602455 Writing the paired reads to /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R1_processed.fastq.paired.fq and /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R2_trimmed.fastq.paired.fq. Writing the single reads to /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R1_processed.fastq.single.fq and /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R2_trimmed.fastq.single.fqCommand completed. Elapsed time: 0:02:32. Running peak memory: 6.178GB.
PID: 17348; Command: fastq_pair; Return code: 0; Memory used: 6.178GB
mv /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R1_processed.fastq.paired.fq /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R1_processed.fastq
(17984)Command completed. Elapsed time: 0:00:01. Running peak memory: 6.178GB.
PID: 17984; Command: mv; Return code: 0; Memory used: 0.002GB
mv /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R2_trimmed.fastq.paired.fq /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R2_trimmed.fastq
(17986)Command completed. Elapsed time: 0:00:00. Running peak memory: 6.178GB.
PID: 17986; Command: mv; Return code: 0; Memory used: 0.001GB
touch /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/repaired.flag
(17988)Command completed. Elapsed time: 0:00:00. Running peak memory: 6.178GB.
PID: 17988; Command: touch; Return code: 0; Memory used: 0.0GB
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/dups_repaired.flag
fastq_pair -t 87956874 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R1_trimmed.fastq /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R2_trimmed_dups.fastq
(17989)Left paired: 26674489 Right paired: 26674489 Left single: 126830 Right single: 4931382 Writing the paired reads to /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R1_trimmed.fastq.paired.fq and /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R2_trimmed_dups.fastq.paired.fq. Writing the single reads to /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R1_trimmed.fastq.single.fq and /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R2_trimmed_dups.fastq.single.fqCommand completed. Elapsed time: 0:01:59. Running peak memory: 6.178GB.
PID: 17989; Command: fastq_pair; Return code: 0; Memory used: 6.154GB
mv /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R1_trimmed.fastq.paired.fq /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R1_trimmed.fastq
(19059)Command completed. Elapsed time: 0:00:00. Running peak memory: 6.178GB.
PID: 19059; Command: mv; Return code: 0; Memory used: 0.002GB
mv /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R2_trimmed_dups.fastq.paired.fq /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R2_trimmed_dups.fastq
(19060)Command completed. Elapsed time: 0:00:01. Running peak memory: 6.178GB.
PID: 19060; Command: mv; Return code: 0; Memory used: 0.001GB
touch /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/dups_repaired.flag
(19063)Command completed. Elapsed time: 0:00:00. Running peak memory: 6.178GB.
PID: 19063; Command: touch; Return code: 0; Memory used: 0.0GB
Prealignments (06-15 13:15:55) elapsed: 295.0 TIME
Prealignment assemblies: ['human_rDNA']
Map to human_rDNA (06-15 13:15:55) elapsed: 0.0 TIME
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/prealignments/human_rDNA_bt2
mkfifo /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/prealignments/human_rDNA_bt2
(19064)Command completed. Elapsed time: 0:00:00. Running peak memory: 6.178GB.
PID: 19064; Command: mkfifo; Return code: 0; Memory used: 0.0GB
File not added to cleanup: prealignments/human_rDNA_bt2
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/prealignments/H9_PRO-seq_1_human_rDNA_unmap_R2.fq.gz
perl /scratch/jps3dp/tools/databio//peppro/tools/filter_paired_fq.pl /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/prealignments/human_rDNA_bt2 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R1_processed.fastq /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R2_trimmed.fastq /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/prealignments/H9_PRO-seq_1_human_rDNA_unmap_R1.fq /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/prealignments/H9_PRO-seq_1_human_rDNA_unmap_R2.fq
(19065)
(bowtie2 -p 12 -k 1 -D 20 -R 3 -N 1 -L 20 -i S,1,0.50 -x /project/shefflab/genomes/human_rDNA/bowtie2_index/default/human_rDNA --rg-id H9_PRO-seq_1 -U /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R1_processed.fastq --un /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/prealignments/human_rDNA_bt2 2>&1 > /dev/null)
not gzipping output File not added to cleanup: prealignments/human_rDNA_bt2 Missing stat 'Aligned_reads_human_rDNA' 30003416 reads; of these: 30003416 (100.00%) were unpaired; of these: 27299076 (90.99%) aligned 0 times 2704340 (9.01%) aligned exactly 1 time 0 (0.00%) aligned >1 times 9.01% overall alignment rate
Aligned_reads_human_rDNA
5408680.0 PEPPRO RES
Alignment_rate_human_rDNA
8.95 PEPPRO RES
Map to human_rDNA (06-15 13:19:19) elapsed: 204.0 TIME
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/prealignments/human_rDNA_dups_bt2
mkfifo /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/prealignments/human_rDNA_dups_bt2
(19503)Command completed. Elapsed time: 0:00:00. Running peak memory: 6.178GB.
PID: 19503; Command: mkfifo; Return code: 0; Memory used: 0.002GB
File not added to cleanup: prealignments/human_rDNA_dups_bt2
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/prealignments/H9_PRO-seq_1_human_rDNA_unmap_R2.fq.gz
perl /scratch/jps3dp/tools/databio//peppro/tools/filter_paired_fq.pl /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/prealignments/human_rDNA_dups_bt2 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R1_trimmed.fastq /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R2_trimmed_dups.fastq /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/prealignments/H9_PRO-seq_1_human_rDNA_unmap_dups_R1.fq /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/prealignments/H9_PRO-seq_1_human_rDNA_unmap_dups_R2.fq
(19504)
(bowtie2 -p 12 -k 1 -D 20 -R 3 -N 1 -L 20 -i S,1,0.50 -x /project/shefflab/genomes/human_rDNA/bowtie2_index/default/human_rDNA --rg-id H9_PRO-seq_1 -U /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/fastq/H9_PRO-seq_1_R1_trimmed.fastq --un /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/prealignments/human_rDNA_dups_bt2 2>&1 > /dev/null)
not gzipping output 2704340 reads skipped 0 reads lost File not added to cleanup: prealignments/human_rDNA_dups_bt2
Map to genome (06-15 13:22:29) elapsed: 189.0 TIME
No files match cleanup pattern: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_fail_qc_dups.bam
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_sort.bam
bowtie2 -p 12 --very-sensitive -X 2000 --rg-id H9_PRO-seq_1 -x /project/shefflab/genomes/hg38/bowtie2_index/default/hg38 --rf -1 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/prealignments/H9_PRO-seq_1_human_rDNA_unmap_R1.fq -2 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/prealignments/H9_PRO-seq_1_human_rDNA_unmap_R2.fq | samtools view -bS - -@ 1 | samtools sort - -@ 1 -T /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/tmp909od977 -o /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_temp.bam
(20319,20320,20321)2176982 reads skipped 0 reads lost 27299076 reads; of these: 27299076 (100.00%) were paired; of these: 9566070 (35.04%) aligned concordantly 0 times 14769056 (54.10%) aligned concordantly exactly 1 time 2963950 (10.86%) aligned concordantly >1 times ---- 9566070 pairs aligned concordantly 0 times; of these: 2129010 (22.26%) aligned discordantly 1 time ---- 7437060 pairs aligned 0 times concordantly or discordantly; of these: 14874120 mates make up the pairs; of these: 6258301 (42.08%) aligned 0 times 3282618 (22.07%) aligned exactly 1 time 5333201 (35.86%) aligned >1 times 88.54% overall alignment rate [bam_sort_core] merging from 15 files and 1 in-memory blocks...Command completed. Elapsed time: 0:46:17. Running peak memory: 6.178GB.
PID: 20320; Command: samtools; Return code: 0; Memory used: 0.004GB
PID: 20319; Command: bowtie2; Return code: 0; Memory used: 3.802GB
PID: 20321; Command: samtools; Return code: 0; Memory used: 0.897GB
samtools view -q 10 -b -@ 12 -U /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_fail_qc.bam /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_temp.bam > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_sort.bam
(24875)Command completed. Elapsed time: 0:01:24. Running peak memory: 6.178GB.
PID: 24875; Command: samtools; Return code: 0; Memory used: 0.018GB
samtools depth -b /project/shefflab/genomes/hg38/refgene_anno/default/hg38_pre-mRNA.bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_sort.bam | awk '{counter++;sum+=$3}END{print sum/counter}'
Mapped_reads
48339851 PEPPRO RES
QC_filtered_reads
27585022 PEPPRO RES
Aligned_reads
20754829.0 PEPPRO RES
Alignment_rate
34.34 PEPPRO RES
Total_efficiency
21.24 PEPPRO RES
Read_depth
3.26 PEPPRO RES Target to produce:/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_sort_dups.bam
bowtie2 -p 12 --very-sensitive -X 2000 --rg-id H9_PRO-seq_1 -x /project/shefflab/genomes/hg38/bowtie2_index/default/hg38 --rf -1 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/prealignments/H9_PRO-seq_1_human_rDNA_unmap_dups_R1.fq -2 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/prealignments/H9_PRO-seq_1_human_rDNA_unmap_dups_R2.fq | samtools view -bS - -@ 1 | samtools sort - -@ 1 -T /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/tmp909od977 -o /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_temp_dups.bam
(26249,26255,26256)24497507 reads; of these: 24497507 (100.00%) were paired; of these: 8334941 (34.02%) aligned concordantly 0 times 13482466 (55.04%) aligned concordantly exactly 1 time 2680100 (10.94%) aligned concordantly >1 times ---- 8334941 pairs aligned concordantly 0 times; of these: 1950931 (23.41%) aligned discordantly 1 time ---- 6384010 pairs aligned 0 times concordantly or discordantly; of these: 12768020 mates make up the pairs; of these: 5376104 (42.11%) aligned 0 times 3000379 (23.50%) aligned exactly 1 time 4391537 (34.39%) aligned >1 times 89.03% overall alignment rate [bam_sort_core] merging from 14 files and 1 in-memory blocks...Command completed. Elapsed time: 0:41:14. Running peak memory: 6.178GB.
PID: 26249; Command: bowtie2; Return code: 0; Memory used: 3.767GB
PID: 26255; Command: samtools; Return code: 0; Memory used: 0.004GB
PID: 26256; Command: samtools; Return code: 0; Memory used: 0.897GB
samtools view -q 10 -b -@ 12 -U /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_fail_qc_dups.bam /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_temp_dups.bam > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_sort_dups.bam
(30214)Command completed. Elapsed time: 0:01:17. Running peak memory: 6.178GB.
PID: 30214; Command: samtools; Return code: 0; Memory used: 0.018GB
Compress all unmapped read files (06-15 15:05:53) elapsed: 6204.0 TIME
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/prealignments/H9_PRO-seq_1_human_rDNA_unmap_R2.fq.gz
pigz -f -p 12 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/prealignments/H9_PRO-seq_1_human_rDNA_unmap_R2.fq
(30427)Command completed. Elapsed time: 0:00:21. Running peak memory: 6.178GB.
PID: 30427; Command: pigz; Return code: 0; Memory used: 0.009GB
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/prealignments/H9_PRO-seq_1_human_rDNA_unmap_R1.fq.gz
pigz -f -p 12 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/prealignments/H9_PRO-seq_1_human_rDNA_unmap_R1.fq
(30459)Command completed. Elapsed time: 0:00:25. Running peak memory: 6.178GB.
PID: 30459; Command: pigz; Return code: 0; Memory used: 0.009GB
Missing stat 'Mitochondrial_reads'
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_temp.bam.bai
samtools index /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_temp.bam
(30497)Command completed. Elapsed time: 0:00:38. Running peak memory: 6.178GB.
PID: 30497; Command: samtools; Return code: 0; Memory used: 0.01GB
samtools idxstats /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_temp.bam | grep -we 'chrM' -we 'chrMT' -we 'M' -we 'MT' -we 'rCRSd' -we 'rCRSd_3k'| cut -f 3
Mitochondrial_reads
674956 PEPPRO RES Target to produce:/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_noMT.bam
samtools index /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_sort.bam
(30537)Command completed. Elapsed time: 0:00:29. Running peak memory: 6.178GB.
PID: 30537; Command: samtools; Return code: 0; Memory used: 0.012GB
samtools idxstats /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_sort.bam | cut -f 1-2 | awk '{print $1, 0, $2}' | grep -vwe 'chrM' -vwe 'chrMT' -vwe 'M' -vwe 'MT' -vwe 'rCRSd' -vwe 'rCRSd_3k' > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/chr_sizes.bed
(30565,30566,30567,30568)Command completed. Elapsed time: 0:00:00. Running peak memory: 6.178GB.
PID: 30565; Command: samtools; Return code: 0; Memory used: 0.003GB
PID: 30567; Command: awk; Return code: 0; Memory used: 0.001GB
PID: 30566; Command: cut; Return code: 0; Memory used: 0.001GB
PID: 30568; Command: grep; Return code: 0; Memory used: 0.003GB
samtools view -L /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/chr_sizes.bed -b -@ 12 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_sort.bam > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_noMT.bam
(30570)Command completed. Elapsed time: 0:00:41. Running peak memory: 6.178GB.
PID: 30570; Command: samtools; Return code: 0; Memory used: 0.019GB
mv /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_noMT.bam /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_sort.bam
(30622)Command completed. Elapsed time: 0:00:00. Running peak memory: 6.178GB.
PID: 30622; Command: mv; Return code: 0; Memory used: 0.002GB
samtools index /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_sort.bam
(30623)Command completed. Elapsed time: 0:00:29. Running peak memory: 6.178GB.
PID: 30623; Command: samtools; Return code: 0; Memory used: 0.011GB
Split BAM file (06-15 15:08:57) elapsed: 184.0 TIME
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_PE1.bam
,/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_PE2.bam
samtools view -b -f 64 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_sort.bam | samtools sort - -@ 12 > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_PE1.bam
(30649,30650)[bam_sort_core] merging from 0 files and 12 in-memory blocks...Command completed. Elapsed time: 0:02:46. Running peak memory: 6.178GB.
PID: 30649; Command: samtools; Return code: 0; Memory used: 0.004GB
PID: 30650; Command: samtools; Return code: 0; Memory used: 5.83GB
samtools view -b -f 128 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_sort.bam | samtools sort - -@ 12 > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_PE2.bam
(31033,31034)[bam_sort_core] merging from 0 files and 12 in-memory blocks...Command completed. Elapsed time: 0:02:27. Running peak memory: 6.178GB.
PID: 31034; Command: samtools; Return code: 0; Memory used: 4.57GB
PID: 31033; Command: samtools; Return code: 0; Memory used: 0.004GB
Missing stat 'Maximum_read_length'
samtools stats /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_PE1.bam | grep '^SN' | cut -f 2- | grep 'maximum length:' | cut -f 2-
Maximum_read_length
38 PEPPRO RES Missing stat 'Genome_size'
awk '{sum+=$2} END {printf "%.0f", sum}' /project/shefflab/genomes/hg38/fasta/default/hg38.chrom.sizes
Genome_size
3099922541 PEPPRO RES Target to produce:/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_temp_dups.bam.bai
samtools index /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_temp_dups.bam
(31393)Command completed. Elapsed time: 0:00:33. Running peak memory: 6.178GB.
PID: 31393; Command: samtools; Return code: 0; Memory used: 0.01GB
samtools idxstats /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_temp_dups.bam | grep -we 'chrM' -we 'chrMT' -we 'M' -we 'MT' -we 'rCRSd' -we 'rCRSd_3k'| cut -f 3
Target exists:/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_sort_dups.bam
No files match cleanup pattern: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_sort_dups.bam.bai Target to produce:/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_dups_PE1.bam
,/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_dups_PE2.bam
samtools view -b -f 64 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_sort_dups.bam | samtools sort - -@ 12 > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_dups_PE1.bam
(31436,31437)[bam_sort_core] merging from 0 files and 12 in-memory blocks...Command completed. Elapsed time: 0:02:43. Running peak memory: 6.178GB.
PID: 31436; Command: samtools; Return code: 0; Memory used: 0.004GB
PID: 31437; Command: samtools; Return code: 0; Memory used: 5.399GB
samtools view -b -f 128 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_sort_dups.bam | samtools sort - -@ 12 > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_dups_PE2.bam
(31655,31656)[bam_sort_core] merging from 0 files and 12 in-memory blocks...Command completed. Elapsed time: 0:02:22. Running peak memory: 6.178GB.
PID: 31656; Command: samtools; Return code: 0; Memory used: 4.278GB
PID: 31655; Command: samtools; Return code: 0; Memory used: 0.004GB
Calculate library complexity (06-15 15:20:47) elapsed: 710.0 TIME
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_preseq_out.txt
preseq c_curve -v -o /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_preseq_out.txt -B /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_dups_PE1.bam
(32034)BAM_INPUT TOTAL READS = 20330230 COUNTS_SUM = 20330230 DISTINCT READS = 1.66542e+07 DISTINCT COUNTS = 262 MAX COUNT = 16193 COUNTS OF 1 = 1.47277e+07 OBSERVED COUNTS (16194) 1 14727710 2 1349104 3 301309 4 112637 5 55354 6 31165 7 18956 8 12770 9 8912 10 6479 11 4906 12 3639 13 2910 14 2335 15 1909 16 1594 17 1295 18 1055 19 904 20 791 21 668 22 611 23 529 24 477 25 427 26 388 27 364 28 323 29 242 30 234 31 263 32 232 33 207 34 182 35 165 36 158 37 167 38 119 39 133 40 114 41 111 42 122 43 105 44 82 45 73 46 82 47 75 48 75 49 76 50 60 51 49 52 59 53 63 54 55 55 46 56 48 57 54 58 37 59 41 60 42 61 47 62 48 63 35 64 35 65 32 66 27 67 30 68 32 69 27 70 18 71 21 72 23 73 16 74 16 75 21 76 17 77 15 78 13 79 24 80 11 81 16 82 21 83 21 84 11 85 11 86 10 87 23 88 12 89 16 90 16 91 11 92 12 93 10 94 8 95 9 96 12 97 16 98 6 99 14 100 3 101 5 102 7 103 9 104 5 105 11 106 10 107 7 108 5 109 7 110 7 111 7 112 4 113 7 114 3 115 4 116 11 117 9 118 4 119 5 122 7 123 3 124 3 125 8 126 3 127 4 128 4 129 3 130 3 131 7 132 6 133 4 134 4 135 7 136 7 137 2 138 5 139 4 140 3 141 1 142 4 143 4 144 2 145 3 146 4 147 5 148 2 149 4 150 1 151 1 152 1 154 1 155 5 156 3 158 1 161 1 162 3 163 2 164 1 165 3 166 3 167 1 169 1 170 2 171 1 172 1 173 2 174 1 175 1 176 3 178 2 179 4 181 1 182 1 183 1 184 1 186 1 187 2 188 1 190 1 192 2 193 1 196 1 198 1 199 1 200 1 202 1 203 2 204 2 205 1 206 1 207 1 208 1 209 1 212 1 214 1 216 3 220 1 221 1 222 1 223 1 224 2 227 1 228 1 232 1 233 1 235 1 237 2 241 1 249 1 251 1 253 1 254 1 255 1 256 1 257 1 258 2 262 1 264 1 266 1 268 1 269 1 275 1 277 1 280 1 296 1 299 1 303 1 306 1 313 1 315 1 317 2 325 1 345 1 348 1 406 1 412 1 430 1 433 1 436 1 437 1 493 1 494 1 540 1 547 1 606 2 611 1 612 1 734 1 787 1 789 1 848 1 927 1 943 1 1376 1 1390 1 1414 1 2008 1 4768 1 5683 1 6998 1 14724 1 16193 1
sample size: 1000000
sample size: 2000000
sample size: 3000000
sample size: 4000000
sample size: 5000000
sample size: 6000000
sample size: 7000000
sample size: 8000000
sample size: 9000000
sample size: 10000000
sample size: 11000000
sample size: 12000000
sample size: 13000000
sample size: 14000000
sample size: 15000000
sample size: 16000000
sample size: 17000000
sample size: 18000000
sample size: 19000000
sample size: 20000000
Command completed. Elapsed time: 0:01:41. Running peak memory: 6.178GB.
PID: 32034; Command: preseq; Return code: 0; Memory used: 0.005GB
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_preseq_yield.txt
preseq lc_extrap -v -o /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_preseq_yield.txt -B /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_dups_PE1.bam
(32142)BAM_INPUT TOTAL READS = 20330230 DISTINCT READS = 1.66542e+07 DISTINCT COUNTS = 262 MAX COUNT = 16193 COUNTS OF 1 = 1.47277e+07 MAX TERMS = 100 OBSERVED COUNTS (16194) 1 14727710 2 1349104 3 301309 4 112637 5 55354 6 31165 7 18956 8 12770 9 8912 10 6479 11 4906 12 3639 13 2910 14 2335 15 1909 16 1594 17 1295 18 1055 19 904 20 791 21 668 22 611 23 529 24 477 25 427 26 388 27 364 28 323 29 242 30 234 31 263 32 232 33 207 34 182 35 165 36 158 37 167 38 119 39 133 40 114 41 111 42 122 43 105 44 82 45 73 46 82 47 75 48 75 49 76 50 60 51 49 52 59 53 63 54 55 55 46 56 48 57 54 58 37 59 41 60 42 61 47 62 48 63 35 64 35 65 32 66 27 67 30 68 32 69 27 70 18 71 21 72 23 73 16 74 16 75 21 76 17 77 15 78 13 79 24 80 11 81 16 82 21 83 21 84 11 85 11 86 10 87 23 88 12 89 16 90 16 91 11 92 12 93 10 94 8 95 9 96 12 97 16 98 6 99 14 100 3 101 5 102 7 103 9 104 5 105 11 106 10 107 7 108 5 109 7 110 7 111 7 112 4 113 7 114 3 115 4 116 11 117 9 118 4 119 5 122 7 123 3 124 3 125 8 126 3 127 4 128 4 129 3 130 3 131 7 132 6 133 4 134 4 135 7 136 7 137 2 138 5 139 4 140 3 141 1 142 4 143 4 144 2 145 3 146 4 147 5 148 2 149 4 150 1 151 1 152 1 154 1 155 5 156 3 158 1 161 1 162 3 163 2 164 1 165 3 166 3 167 1 169 1 170 2 171 1 172 1 173 2 174 1 175 1 176 3 178 2 179 4 181 1 182 1 183 1 184 1 186 1 187 2 188 1 190 1 192 2 193 1 196 1 198 1 199 1 200 1 202 1 203 2 204 2 205 1 206 1 207 1 208 1 209 1 212 1 214 1 216 3 220 1 221 1 222 1 223 1 224 2 227 1 228 1 232 1 233 1 235 1 237 2 241 1 249 1 251 1 253 1 254 1 255 1 256 1 257 1 258 2 262 1 264 1 266 1 268 1 269 1 275 1 277 1 280 1 296 1 299 1 303 1 306 1 313 1 315 1 317 2 325 1 345 1 348 1 406 1 412 1 430 1 433 1 436 1 437 1 493 1 494 1 540 1 547 1 606 2 611 1 612 1 734 1 787 1 789 1 848 1 927 1 943 1 1376 1 1390 1 1414 1 2008 1 4768 1 5683 1 6998 1 14724 1 16193 1
[ESTIMATING YIELD CURVE]
[BOOTSTRAPPING HISTOGRAM]
....................................................................................................
[COMPUTING CONFIDENCE INTERVALS]
[WRITING OUTPUT]
Command completed. Elapsed time: 0:01:47. Running peak memory: 6.178GB.
PID: 32142; Command: preseq; Return code: 0; Memory used: 0.011GB
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_preseq_counts.txt
echo '/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_preseq_yield.txt '$(samtools view -c -F 4 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_dups_PE1.bam)' '$(samtools view -c -F 4 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_PE1.bam) > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_preseq_counts.txt
(32237)Command completed. Elapsed time: 0:00:29. Running peak memory: 6.178GB.
PID: 32237; Command: echo; Return code: 0; Memory used: 0.006GB
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_preseq_plot.pdf
,/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_preseq_plot.png
Rscript /scratch/jps3dp/tools/databio//peppro/tools/PEPPRO.R preseq -i /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_preseq_yield.txt -r /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_preseq_counts.txt -o /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_preseq_plot
(32272)Processing H9_PRO-seq_1 INFO: Found real counts for H9_PRO-seq_1 - Total (M): 21.966708 Unique (M): 20.33023
Library complexity plot completed!
Command completed. Elapsed time: 0:00:06. Running peak memory: 6.178GB.
PID: 32272; Command: Rscript; Return code: 0; Memory used: 0.211GB
Library complexity
QC_hg38/H9_PRO-seq_1_preseq_plot.pdf Library complexity QC_hg38/H9_PRO-seq_1_preseq_plot.png PEPPRO OBJ Missing stat 'Frac_exp_unique_at_10M'
grep -w '10000000' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_preseq_yield.txt | awk '{print $2}'
Frac_exp_unique_at_10M
0.8774 PEPPRO RES
Calculate NRF, PBC1, and PBC2 (06-15 15:24:50) elapsed: 243.0 TIME
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_PE1.bam.bai
samtools index /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_PE1.bam
(32292)Command completed. Elapsed time: 0:00:18. Running peak memory: 6.178GB.
PID: 32292; Command: samtools; Return code: 0; Memory used: 0.014GB
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_bamQC.tsv
/scratch/jps3dp/tools/databio//peppro/tools/bamQC.py --silent -i /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_PE1.bam -c 12 -o /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_bamQC.tsv
(32424)Configured logger 'root' using pararead v0.6 Registering input file: '/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_PE1.bam' Temporary files will be stored in: '/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/tmp_H9_PRO-seq_1_PE1_b_xgcpnk' Processing with 12 cores... Discarding 96 chunk(s) of reads: ['chrM', 'chr1_KI270709v1_random', 'chr1_KI270710v1_random', 'chr2_KI270715v1_random', 'chr9_KI270720v1_random', 'chr14_KI270724v1_random', 'chr22_KI270735v1_random', 'chr22_KI270739v1_random', 'chrY_KI270740v1_random', 'chrUn_KI270302v1', 'chrUn_KI270304v1', 'chrUn_KI270303v1', 'chrUn_KI270305v1', 'chrUn_KI270322v1', 'chrUn_KI270320v1', 'chrUn_KI270310v1', 'chrUn_KI270316v1', 'chrUn_KI270315v1', 'chrUn_KI270312v1', 'chrUn_KI270311v1', 'chrUn_KI270317v1', 'chrUn_KI270412v1', 'chrUn_KI270411v1', 'chrUn_KI270414v1', 'chrUn_KI270419v1', 'chrUn_KI270418v1', 'chrUn_KI270420v1', 'chrUn_KI270424v1', 'chrUn_KI270417v1', 'chrUn_KI270422v1', 'chrUn_KI270423v1', 'chrUn_KI270425v1', 'chrUn_KI270429v1', 'chrUn_KI270465v1', 'chrUn_KI270468v1', 'chrUn_KI270510v1', 'chrUn_KI270509v1', 'chrUn_KI270518v1', 'chrUn_KI270508v1', 'chrUn_KI270516v1', 'chrUn_KI270512v1', 'chrUn_KI270522v1', 'chrUn_KI270511v1', 'chrUn_KI270507v1', 'chrUn_KI270517v1', 'chrUn_KI270529v1', 'chrUn_KI270528v1', 'chrUn_KI270530v1', 'chrUn_KI270538v1', 'chrUn_KI270544v1', 'chrUn_KI270548v1', 'chrUn_KI270583v1', 'chrUn_KI270580v1', 'chrUn_KI270581v1', 'chrUn_KI270590v1', 'chrUn_KI270582v1', 'chrUn_KI270588v1', 'chrUn_KI270591v1', 'chrUn_KI270330v1', 'chrUn_KI270329v1', 'chrUn_KI270334v1', 'chrUn_KI270335v1', 'chrUn_KI270338v1', 'chrUn_KI270340v1', 'chrUn_KI270336v1', 'chrUn_KI270363v1', 'chrUn_KI270364v1', 'chrUn_KI270378v1', 'chrUn_KI270379v1', 'chrUn_KI270389v1', 'chrUn_KI270390v1', 'chrUn_KI270387v1', 'chrUn_KI270395v1', 'chrUn_KI270396v1', 'chrUn_KI270388v1', 'chrUn_KI270394v1', 'chrUn_KI270386v1', 'chrUn_KI270391v1', 'chrUn_KI270383v1', 'chrUn_KI270393v1', 'chrUn_KI270384v1', 'chrUn_KI270392v1', 'chrUn_KI270381v1', 'chrUn_KI270385v1', 'chrUn_KI270382v1', 'chrUn_KI270376v1', 'chrUn_KI270374v1', 'chrUn_KI270372v1', 'chrUn_KI270373v1', 'chrUn_KI270375v1', 'chrUn_KI270371v1', 'chrUn_KI270448v1', 'chrUn_GL000226v1', 'chrUn_GL000213v1', 'chrUn_KI270754v1', 'chrUn_KI270755v1'] Keeping 99 chunk(s) of reads: ['chr1', 'chr2', 'chr3', 'chr4', 'chr5', 'chr6', 'chr7', 'chr8', 'chr9', 'chr10', 'chr11', 'chr12', 'chr13', 'chr14', 'chr15', 'chr16', 'chr17', 'chr18', 'chr19', 'chr20', 'chr21', 'chr22', 'chrX', 'chrY', 'chr1_KI270706v1_random', 'chr1_KI270707v1_random', 'chr1_KI270708v1_random', 'chr1_KI270711v1_random', 'chr1_KI270712v1_random', 'chr1_KI270713v1_random', 'chr1_KI270714v1_random', 'chr2_KI270716v1_random', 'chr3_GL000221v1_random', 'chr4_GL000008v2_random', 'chr5_GL000208v1_random', 'chr9_KI270717v1_random', 'chr9_KI270718v1_random', 'chr9_KI270719v1_random', 'chr11_KI270721v1_random', 'chr14_GL000009v2_random', 'chr14_GL000225v1_random', 'chr14_KI270722v1_random', 'chr14_GL000194v1_random', 'chr14_KI270723v1_random', 'chr14_KI270725v1_random', 'chr14_KI270726v1_random', 'chr15_KI270727v1_random', 'chr16_KI270728v1_random', 'chr17_GL000205v2_random', 'chr17_KI270729v1_random', 'chr17_KI270730v1_random', 'chr22_KI270731v1_random', 'chr22_KI270732v1_random', 'chr22_KI270733v1_random', 'chr22_KI270734v1_random', 'chr22_KI270736v1_random', 'chr22_KI270737v1_random', 'chr22_KI270738v1_random', 'chrUn_KI270442v1', 'chrUn_KI270466v1', 'chrUn_KI270467v1', 'chrUn_KI270435v1', 'chrUn_KI270438v1', 'chrUn_KI270519v1', 'chrUn_KI270515v1', 'chrUn_KI270539v1', 'chrUn_KI270587v1', 'chrUn_KI270579v1', 'chrUn_KI270589v1', 'chrUn_KI270584v1', 'chrUn_KI270593v1', 'chrUn_KI270333v1', 'chrUn_KI270337v1', 'chrUn_KI270362v1', 'chrUn_KI270366v1', 'chrUn_KI270521v1', 'chrUn_GL000195v1', 'chrUn_GL000219v1', 'chrUn_GL000220v1', 'chrUn_GL000224v1', 'chrUn_KI270741v1', 'chrUn_KI270743v1', 'chrUn_KI270744v1', 'chrUn_KI270745v1', 'chrUn_KI270746v1', 'chrUn_KI270747v1', 'chrUn_KI270748v1', 'chrUn_KI270749v1', 'chrUn_KI270750v1', 'chrUn_KI270751v1', 'chrUn_KI270752v1', 'chrUn_KI270753v1', 'chrUn_KI270756v1', 'chrUn_KI270757v1', 'chrUn_GL000214v1', 'chrUn_KI270742v1', 'chrUn_GL000216v2', 'chrUn_GL000218v1', 'chrEBV']Command completed. Elapsed time: 0:00:24. Running peak memory: 6.178GB.
PID: 32424; Command: /scratch/jps3dp/tools/databio//peppro/tools/bamQC.py; Return code: 0; Memory used: 1.899GB
awk '{ for (i=1; i<=NF; ++i) { if ($i ~ "NRF") c=i } getline; print $c }' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_bamQC.tsv
awk '{ for (i=1; i<=NF; ++i) { if ($i ~ "PBC1") c=i } getline; print $c }' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_bamQC.tsv
awk '{ for (i=1; i<=NF; ++i) { if ($i ~ "PBC2") c=i } getline; print $c }' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_bamQC.tsv
NRF
1.0 PEPPRO RES
PBC1
10983354.0 PEPPRO RES
PBC2
10983354.0 PEPPRO RES Target to produce:/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_unmap.bam
samtools view -b -@ 12 -f 12 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_temp.bam > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_unmap.bam
(32480)Command completed. Elapsed time: 0:00:09. Running peak memory: 6.178GB.
PID: 32480; Command: samtools; Return code: 0; Memory used: 0.009GB
samtools view -c -f 4 -@ 12 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_temp.bam
Unmapped_reads
6258301 PEPPRO RES
Split BAM by strand (06-15 15:25:48) elapsed: 58.0 TIME
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_plus.bam
,/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_minus.bam
samtools view -bh -F 20 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_PE1.bam > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_plus.bam
(32525)Command completed. Elapsed time: 0:01:10. Running peak memory: 6.178GB.
PID: 32525; Command: samtools; Return code: 0; Memory used: 0.006GB
samtools view -bh -f 16 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_PE1.bam > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_minus.bam
(32589)Command completed. Elapsed time: 0:01:08. Running peak memory: 6.178GB.
PID: 32589; Command: samtools; Return code: 0; Memory used: 0.006GB
Calculate TSS enrichment (06-15 15:28:07) elapsed: 139.0 TIME
Missing stat 'TSS_non-coding_score'
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/plus_TSS.tsv
,/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/minus_TSS.tsv
sed -n -e '/[[:space:]]+/w /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/plus_TSS.tsv' -e '/[[:space:]]-/w /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/minus_TSS.tsv' /project/shefflab/genomes/hg38/refgene_anno/default/hg38_TSS.bed
(32652)Command completed. Elapsed time: 0:00:00. Running peak memory: 6.178GB.
PID: 32652; Command: sed; Return code: 0; Memory used: 0.002GB
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_plus_TssEnrichment.txt
/scratch/jps3dp/tools/databio//peppro/tools/pyTssEnrichment.py -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_PE1.bam -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/plus_TSS.tsv -p ends -c 12 -z -v -s 6 -o /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_plus_TssEnrichment.txt
(32653)Command completed. Elapsed time: 0:00:07. Running peak memory: 6.178GB.
PID: 32653; Command: /scratch/jps3dp/tools/databio//peppro/tools/pyTssEnrichment.py; Return code: 0; Memory used: 0.793GB
TSS_coding_score
26.7 PEPPRO RES Target to produce:/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_minus_TssEnrichment.txt
/scratch/jps3dp/tools/databio//peppro/tools/pyTssEnrichment.py -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_PE1.bam -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/minus_TSS.tsv -p ends -c 12 -z -v -s 6 -o /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_minus_TssEnrichment.txt
(32686)Command completed. Elapsed time: 0:00:06. Running peak memory: 6.178GB.
PID: 32686; Command: /scratch/jps3dp/tools/databio//peppro/tools/pyTssEnrichment.py; Return code: 0; Memory used: 0.925GB
TSS_non-coding_score
10.1 PEPPRO RES Target to produce:/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_TSSenrichment.pdf
Rscript /scratch/jps3dp/tools/databio//peppro/tools/PEPPRO.R tss -i /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_plus_TssEnrichment.txt /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_minus_TssEnrichment.txt
(32721)
Generating TSS plot with /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_plus_TssEnrichment.txt and /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_minus_TssEnrichment.txt
geom_smooth()
using formula 'y ~ x'
geom_smooth()
using formula 'y ~ x'
geom_smooth()
using formula 'y ~ x'
geom_smooth()
using formula 'y ~ x'
TSS enrichment plot completed!
Command completed. Elapsed time: 0:00:06. Running peak memory: 6.178GB.
PID: 32721; Command: Rscript; Return code: 0; Memory used: 0.317GB
TSS enrichment
QC_hg38/H9_PRO-seq_1_TSSenrichment.pdf TSS enrichment QC_hg38/H9_PRO-seq_1_TSSenrichment.png PEPPRO OBJ Target to produce:/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/chr_order.txt
samtools view -H /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_PE1.bam | grep 'SN:' | awk -F':' '{print $2,$3}' | awk -F' ' -v OFS=' ' '{print $1,$3}' > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/chr_order.txt
(32742,32743,32744,32745)Command completed. Elapsed time: 0:00:00. Running peak memory: 6.178GB.
PID: 32742; Command: samtools; Return code: 0; Memory used: 0.0GB
PID: 32744; Command: awk; Return code: 0; Memory used: 0.0GB
PID: 32743; Command: grep; Return code: 0; Memory used: 0.0GB
PID: 32745; Command: awk; Return code: 0; Memory used: 0.0GB
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/chr_keep.txt
cut -f 1 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/chr_keep.txt
(32747)Command completed. Elapsed time: 0:00:00. Running peak memory: 6.178GB.
PID: 32747; Command: cut; Return code: 0; Memory used: 0.002GB
Calculate Pause Index (PI) (06-15 15:28:26) elapsed: 19.0 TIME
Missing stat 'Pause_index'
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/hg38_ensembl_tss.bed
,/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/hg38_ensembl_gene_body.bed
grep -wf /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/chr_keep.txt /project/shefflab/genomes/hg38/ensembl_gtf/default/hg38_ensembl_TSS.bed | bedtools sort -i stdin -faidx /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/hg38_ensembl_tss.bed
(32750,32751)Command completed. Elapsed time: 0:00:02. Running peak memory: 6.178GB.
PID: 32750; Command: grep; Return code: 0; Memory used: 0.002GB
PID: 32751; Command: bedtools; Return code: 0; Memory used: 0.046GB
grep -wf /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/chr_keep.txt /project/shefflab/genomes/hg38/ensembl_gtf/default/hg38_ensembl_gene_body.bed | bedtools sort -i stdin -faidx /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/hg38_ensembl_gene_body.bed
(32755,32756)Command completed. Elapsed time: 0:00:00. Running peak memory: 6.178GB.
PID: 32755; Command: grep; Return code: 0; Memory used: 0.003GB
PID: 32756; Command: bedtools; Return code: 0; Memory used: 0.022GB
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_TSS_density.bed
bedtools coverage -sorted -counts -s -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/hg38_ensembl_tss.bed -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_PE1.bam -g /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/chr_order.txt | awk '$7>0' | sort -k4,4 -k7,7nr | sort -k4,4 -u > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_TSS_density.bed
(32758,32759,32760,32761)Command completed. Elapsed time: 0:00:27. Running peak memory: 6.178GB.
PID: 32759; Command: awk; Return code: 0; Memory used: 0.001GB
PID: 32761; Command: sort; Return code: 0; Memory used: 0.002GB
PID: 32758; Command: bedtools; Return code: 0; Memory used: 0.009GB
PID: 32760; Command: sort; Return code: 0; Memory used: 0.004GB
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_gene_body_density.bed
bedtools coverage -sorted -counts -s -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/hg38_ensembl_gene_body.bed -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_PE1.bam -g /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/chr_order.txt | awk '$7>0' | sort -k4 > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_gene_body_density.bed
(32785,32786,32787)Command completed. Elapsed time: 0:00:31. Running peak memory: 6.178GB.
PID: 32786; Command: awk; Return code: 0; Memory used: 0.001GB
PID: 32785; Command: bedtools; Return code: 0; Memory used: 0.063GB
PID: 32787; Command: sort; Return code: 0; Memory used: 0.005GB
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_pause_index.bed
join --nocheck-order -j4 -o 1.1 1.2 1.3 1.4 1.6 1.7 2.2 2.3 2.7 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_TSS_density.bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_gene_body_density.bed | awk -v OFS=' ' '{ if ($5 == "+"){print $1, $2, $8, $4, sqrt((($6+$9)/sqrt(($8-$2)^2))^2), ($6/sqrt(($3-$2)^2))/($9/sqrt(($8-$7)^2)), $5} else {print $1, $2, $8, $4, sqrt((($6+$9)/sqrt(($3-$7)^2))^2),($6/sqrt(($3-$2)^2))/($9/sqrt(($8-$7)^2)), $5}}' | env LC_COLLATE=C sort -k1,1 -k2,2n > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/tmp6bum7qxi
(32822,32823,32824)Command completed. Elapsed time: 0:00:00. Running peak memory: 6.178GB.
PID: 32822; Command: join; Return code: 0; Memory used: 0.001GB
PID: 32824; Command: env; Return code: 0; Memory used: 0.004GB
PID: 32823; Command: awk; Return code: 0; Memory used: 0.001GB
awk '{print $5}' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/tmp6bum7qxi | sort -n | awk 'BEGIN{i=0} {s[i]=$1; i++;} END{print s[int(NR*0.5-0.5)]}'
Target to produce:/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_pause_index.bed
awk -v OFS=' ' '{ if ($5 > 0.0210697) {print $1, $2, $3, $4, $6, $7}}' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/tmp6bum7qxi > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_pause_index.bed
(32831)Command completed. Elapsed time: 0:00:00. Running peak memory: 6.178GB.
PID: 32831; Command: awk; Return code: 0; Memory used: 0.002GB
sort -k5,5n /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_pause_index.bed | awk ' { a[i++]=$5; } END { x=int((i+1)/2); if (x < (i+1)/2) print (a[x-1]+a[x])/2; else print a[x-1]; }'
Pause_index
26.45 PEPPRO RES Target to produce:/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_pause_index.pdf
Rscript /scratch/jps3dp/tools/databio//peppro/tools/PEPPRO.R pi --annotate -i /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_pause_index.bed
(32836)Pause index plot completed!
Command completed. Elapsed time: 0:00:05. Running peak memory: 6.178GB.
PID: 32836; Command: Rscript; Return code: 0; Memory used: 0.317GB
Pause index
QC_hg38/H9_PRO-seq_1_pause_index.pdf Pause index QC_hg38/H9_PRO-seq_1_pause_index.png PEPPRO OBJ Target to produce:/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_pause_index.bed.gz
pigz -f -p 12 -f /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_pause_index.bed
(32857)Command completed. Elapsed time: 0:00:00. Running peak memory: 6.178GB.
PID: 32857; Command: pigz; Return code: 0; Memory used: 0.002GB
Calculate Fraction of Reads in pre-mature mRNA (06-15 15:29:32) elapsed: 65.0 TIME
Missing stat 'Plus_FRiP'
samtools view -@ 4 -c -L /project/shefflab/genomes/hg38/refgene_anno/default/hg38_pre-mRNA.bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_plus.bam
20754829.0 7959339
Plus_FRiP
0.38 PEPPRO RES Missing stat 'Minus_FRiP'
samtools view -@ 4 -c -L /project/shefflab/genomes/hg38/refgene_anno/default/hg38_pre-mRNA.bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_minus.bam
20754829.0 7448642
Minus_FRiP
0.36 PEPPRO RES Target to produce:/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/signal_hg38/H9_PRO-seq_1_gene_coverage.bed
grep -wf /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/chr_keep.txt /project/shefflab/genomes/hg38/refgene_anno/default/hg38_pre-mRNA.bed | bedtools sort -i stdin -faidx /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/hg38_gene_sort.bed
(33122,33123)Command completed. Elapsed time: 0:00:01. Running peak memory: 6.178GB.
PID: 33122; Command: grep; Return code: 0; Memory used: 0.004GB
PID: 33123; Command: bedtools; Return code: 0; Memory used: 0.006GB
bedtools coverage -sorted -counts -s -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/hg38_gene_sort.bed -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_PE1.bam -g /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/signal_hg38/H9_PRO-seq_1_gene_coverage.bed
(33126)Command completed. Elapsed time: 0:00:31. Running peak memory: 6.178GB.
PID: 33126; Command: bedtools; Return code: 0; Memory used: 0.069GB
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/raw/hg38_annotations.bed
ln -sf /project/shefflab/genomes/hg38/feat_annotation/default/hg38_annotations.bed.gz /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/raw/hg38_annotations.bed.gz
(33158)Command completed. Elapsed time: 0:00:00. Running peak memory: 6.178GB.
PID: 33158; Command: ln; Return code: 0; Memory used: 0.002GB
pigz -f -p 12 -d -c /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/raw/hg38_annotations.bed.gz > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/raw/hg38_annotations.bed
(33159)Command completed. Elapsed time: 0:00:00. Running peak memory: 6.178GB.
PID: 33159; Command: pigz; Return code: 0; Memory used: 0.002GB
Calculate cumulative and terminal fraction of reads in features (cFRiF/FRiF) (06-15 15:30:43) elapsed: 71.0 TIME
cut -f 4 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/raw/hg38_annotations.bed | sort -u
Target to produce:/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/Enhancer
awk -F' ' '{print>"/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/"$4}' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/raw/hg38_annotations.bed
(33168)Command completed. Elapsed time: 0:00:01. Running peak memory: 6.178GB.
PID: 33168; Command: awk; Return code: 0; Memory used: 0.002GB
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/Enhancer_sort.bed
cut -f 1 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/chr_order.txt | grep -wf - /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/Enhancer | cut -f 1-3 | bedtools sort -i stdin -faidx /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/Enhancer_sort.bed
(33170,33171,33172,33173)Command completed. Elapsed time: 0:00:01. Running peak memory: 6.178GB.
PID: 33170; Command: cut; Return code: 0; Memory used: 0.0GB
PID: 33171; Command: grep; Return code: 0; Memory used: 0.002GB
PID: 33173; Command: bedtools; Return code: 0; Memory used: 0.044GB
PID: 33172; Command: cut; Return code: 0; Memory used: 0.001GB
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_Enhancer_plus_coverage.bed
bedtools coverage -sorted -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/Enhancer_sort.bed -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_plus.bam -g /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_Enhancer_plus_coverage.bed
(33175)Command completed. Elapsed time: 0:00:13. Running peak memory: 6.178GB.
PID: 33175; Command: bedtools; Return code: 0; Memory used: 0.007GB
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_Enhancer_minus_coverage.bed
bedtools coverage -sorted -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/Enhancer_sort.bed -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_minus.bam -g /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_Enhancer_minus_coverage.bed
(33193)Command completed. Elapsed time: 0:00:13. Running peak memory: 6.178GB.
PID: 33193; Command: bedtools; Return code: 0; Memory used: 0.01GB
Target exists: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/Promoter
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/Promoter_sort.bed
cut -f 1 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/chr_order.txt | grep -wf - /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/Promoter | cut -f 1-3 | bedtools sort -i stdin -faidx /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/Promoter_sort.bed
(33209,33210,33211,33212)Command completed. Elapsed time: 0:00:00. Running peak memory: 6.178GB.
PID: 33209; Command: cut; Return code: 0; Memory used: 0.0GB
PID: 33210; Command: grep; Return code: 0; Memory used: 0.003GB
PID: 33212; Command: bedtools; Return code: 0; Memory used: 0.008GB
PID: 33211; Command: cut; Return code: 0; Memory used: 0.001GB
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_Promoter_plus_coverage.bed
bedtools coverage -sorted -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/Promoter_sort.bed -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_plus.bam -g /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_Promoter_plus_coverage.bed
(33214)Command completed. Elapsed time: 0:00:14. Running peak memory: 6.178GB.
PID: 33214; Command: bedtools; Return code: 0; Memory used: 0.024GB
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_Promoter_minus_coverage.bed
bedtools coverage -sorted -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/Promoter_sort.bed -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_minus.bam -g /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_Promoter_minus_coverage.bed
(33234)Command completed. Elapsed time: 0:00:13. Running peak memory: 6.178GB.
PID: 33234; Command: bedtools; Return code: 0; Memory used: 0.017GB
Target exists: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/Promoter Flanking Region
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/Promoter_Flanking_Region
mv "/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/Promoter Flanking Region" "/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/Promoter_Flanking_Region"
(33251)Command completed. Elapsed time: 0:00:00. Running peak memory: 6.178GB.
PID: 33251; Command: mv; Return code: 0; Memory used: 0.002GB
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/Promoter_Flanking_Region_sort.bed
cut -f 1 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/chr_order.txt | grep -wf - /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/Promoter_Flanking_Region | cut -f 1-3 | bedtools sort -i stdin -faidx /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/Promoter_Flanking_Region_sort.bed
(33252,33253,33254,33255)Command completed. Elapsed time: 0:00:01. Running peak memory: 6.178GB.
PID: 33252; Command: cut; Return code: 0; Memory used: 0.0GB
PID: 33254; Command: cut; Return code: 0; Memory used: 0.001GB
PID: 33253; Command: grep; Return code: 0; Memory used: 0.002GB
PID: 33255; Command: bedtools; Return code: 0; Memory used: 0.01GB
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_Promoter_Flanking_Region_plus_coverage.bed
bedtools coverage -sorted -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/Promoter_Flanking_Region_sort.bed -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_plus.bam -g /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_Promoter_Flanking_Region_plus_coverage.bed
(33257)Command completed. Elapsed time: 0:00:14. Running peak memory: 6.178GB.
PID: 33257; Command: bedtools; Return code: 0; Memory used: 0.01GB
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_Promoter_Flanking_Region_minus_coverage.bed
bedtools coverage -sorted -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/Promoter_Flanking_Region_sort.bed -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_minus.bam -g /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_Promoter_Flanking_Region_minus_coverage.bed
(33271)Command completed. Elapsed time: 0:00:13. Running peak memory: 6.178GB.
PID: 33271; Command: bedtools; Return code: 0; Memory used: 0.01GB
Target exists: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/5' UTR
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/5_UTR
mv "/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/5' UTR" "/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/5_UTR"
(33286)Command completed. Elapsed time: 0:00:00. Running peak memory: 6.178GB.
PID: 33286; Command: mv; Return code: 0; Memory used: 0.0GB
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/5_UTR_sort.bed
cut -f 1 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/chr_order.txt | grep -wf - /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/5_UTR | cut -f 1-3 | bedtools sort -i stdin -faidx /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/5_UTR_sort.bed
(33287,33288,33289,33290)Command completed. Elapsed time: 0:00:01. Running peak memory: 6.178GB.
PID: 33287; Command: cut; Return code: 0; Memory used: 0.0GB
PID: 33288; Command: grep; Return code: 0; Memory used: 0.003GB
PID: 33290; Command: bedtools; Return code: 0; Memory used: 0.03GB
PID: 33289; Command: cut; Return code: 0; Memory used: 0.001GB
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_5_UTR_plus_coverage.bed
bedtools coverage -sorted -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/5_UTR_sort.bed -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_plus.bam -g /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_5_UTR_plus_coverage.bed
(33292)Command completed. Elapsed time: 0:00:13. Running peak memory: 6.178GB.
PID: 33292; Command: bedtools; Return code: 0; Memory used: 0.009GB
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_5_UTR_minus_coverage.bed
bedtools coverage -sorted -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/5_UTR_sort.bed -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_minus.bam -g /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_5_UTR_minus_coverage.bed
(33309)Command completed. Elapsed time: 0:00:13. Running peak memory: 6.178GB.
PID: 33309; Command: bedtools; Return code: 0; Memory used: 0.008GB
Target exists: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/3' UTR
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/3_UTR
mv "/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/3' UTR" "/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/3_UTR"
(33326)Command completed. Elapsed time: 0:00:00. Running peak memory: 6.178GB.
PID: 33326; Command: mv; Return code: 0; Memory used: 0.001GB
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/3_UTR_sort.bed
cut -f 1 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/chr_order.txt | grep -wf - /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/3_UTR | cut -f 1-3 | bedtools sort -i stdin -faidx /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/3_UTR_sort.bed
(33327,33328,33329,33330)Command completed. Elapsed time: 0:00:01. Running peak memory: 6.178GB.
PID: 33327; Command: cut; Return code: 0; Memory used: 0.0GB
PID: 33328; Command: grep; Return code: 0; Memory used: 0.003GB
PID: 33330; Command: bedtools; Return code: 0; Memory used: 0.009GB
PID: 33329; Command: cut; Return code: 0; Memory used: 0.001GB
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_3_UTR_plus_coverage.bed
bedtools coverage -sorted -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/3_UTR_sort.bed -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_plus.bam -g /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_3_UTR_plus_coverage.bed
(33333)Command completed. Elapsed time: 0:00:13. Running peak memory: 6.178GB.
PID: 33333; Command: bedtools; Return code: 0; Memory used: 0.012GB
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_3_UTR_minus_coverage.bed
bedtools coverage -sorted -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/3_UTR_sort.bed -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_minus.bam -g /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_3_UTR_minus_coverage.bed
(33349)Command completed. Elapsed time: 0:00:13. Running peak memory: 6.178GB.
PID: 33349; Command: bedtools; Return code: 0; Memory used: 0.01GB
Target exists: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/Exon
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/Exon_sort.bed
cut -f 1 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/chr_order.txt | grep -wf - /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/Exon | cut -f 1-3 | bedtools sort -i stdin -faidx /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/Exon_sort.bed
(33364,33365,33366,33367)Command completed. Elapsed time: 0:00:03. Running peak memory: 6.178GB.
PID: 33364; Command: cut; Return code: 0; Memory used: 0.0GB
PID: 33365; Command: grep; Return code: 0; Memory used: 0.004GB
PID: 33367; Command: bedtools; Return code: 0; Memory used: 0.158GB
PID: 33366; Command: cut; Return code: 0; Memory used: 0.001GB
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_Exon_plus_coverage.bed
bedtools coverage -sorted -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/Exon_sort.bed -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_plus.bam -g /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_Exon_plus_coverage.bed
(33373)Command completed. Elapsed time: 0:00:15. Running peak memory: 6.178GB.
PID: 33373; Command: bedtools; Return code: 0; Memory used: 0.025GB
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_Exon_minus_coverage.bed
bedtools coverage -sorted -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/Exon_sort.bed -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_minus.bam -g /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_Exon_minus_coverage.bed
(33390)Command completed. Elapsed time: 0:00:14. Running peak memory: 6.178GB.
PID: 33390; Command: bedtools; Return code: 0; Memory used: 0.022GB
Target exists: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/Intron
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/Intron_sort.bed
cut -f 1 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/chr_order.txt | grep -wf - /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/Intron | cut -f 1-3 | bedtools sort -i stdin -faidx /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/Intron_sort.bed
(33408,33409,33410,33411)Command completed. Elapsed time: 0:00:01. Running peak memory: 6.178GB.
PID: 33408; Command: cut; Return code: 0; Memory used: 0.0GB
PID: 33410; Command: cut; Return code: 0; Memory used: 0.001GB
PID: 33409; Command: grep; Return code: 0; Memory used: 0.002GB
PID: 33411; Command: bedtools; Return code: 0; Memory used: 0.077GB
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_Intron_plus_coverage.bed
bedtools coverage -sorted -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/Intron_sort.bed -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_plus.bam -g /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_Intron_plus_coverage.bed
(33415)Command completed. Elapsed time: 0:00:16. Running peak memory: 6.178GB.
PID: 33415; Command: bedtools; Return code: 0; Memory used: 0.056GB
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_Intron_minus_coverage.bed
bedtools coverage -sorted -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/Intron_sort.bed -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_minus.bam -g /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_Intron_minus_coverage.bed
(33468)Command completed. Elapsed time: 0:00:15. Running peak memory: 6.178GB.
PID: 33468; Command: bedtools; Return code: 0; Memory used: 0.039GB
Plot cFRiF/FRiF (06-15 15:34:04) elapsed: 201.0 TIME
samtools view -@ 12 -q 10 -c -F4 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_plus.bam
Target to produce:/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_cFRiF.pdf
Rscript /scratch/jps3dp/tools/databio//peppro/tools/PEPPRO.R frif -s H9_PRO-seq_1 -z 3099922541 -n 11202246 -y cfrif --reads -o /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_cFRiF.pdf --bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_Enhancer_plus_coverage.bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_Promoter_plus_coverage.bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_Promoter_Flanking_Region_plus_coverage.bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_5_UTR_plus_coverage.bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_3_UTR_plus_coverage.bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_Exon_plus_coverage.bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_Intron_plus_coverage.bed
(33502)Cumulative cfrif plot completed!
Command completed. Elapsed time: 0:00:33. Running peak memory: 6.178GB.
PID: 33502; Command: Rscript; Return code: 0; Memory used: 0.501GB
cFRiF
QC_hg38/H9_PRO-seq_1_cFRiF.pdf cFRiF QC_hg38/H9_PRO-seq_1_cFRiF.png PEPPRO OBJ Target to produce:/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_FRiF.pdf
Rscript /scratch/jps3dp/tools/databio//peppro/tools/PEPPRO.R frif -s H9_PRO-seq_1 -z 3099922541 -n 11202246 -y frif --reads -o /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_FRiF.pdf --bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_Enhancer_plus_coverage.bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_Promoter_plus_coverage.bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_Promoter_Flanking_Region_plus_coverage.bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_5_UTR_plus_coverage.bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_3_UTR_plus_coverage.bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_Exon_plus_coverage.bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_Intron_plus_coverage.bed
(33548)Cumulative frif plot completed!
Command completed. Elapsed time: 0:00:24. Running peak memory: 6.178GB.
PID: 33548; Command: Rscript; Return code: 0; Memory used: 0.501GB
FRiF
QC_hg38/H9_PRO-seq_1_FRiF.pdf FRiF QC_hg38/H9_PRO-seq_1_FRiF.png PEPPRO OBJ
Calculate mRNA contamination (06-15 15:35:02) elapsed: 58.0 TIME
Missing stat 'mRNA_contamination'
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/hg38_exons_sort.bed
,/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/hg38_introns_sort.bed
grep -wf /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/chr_keep.txt /project/shefflab/genomes/hg38/refgene_anno/default/hg38_exons.bed | bedtools sort -i stdin -faidx /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/hg38_exons_sort.bed
(33698,33699)Command completed. Elapsed time: 0:00:05. Running peak memory: 6.178GB.
PID: 33698; Command: grep; Return code: 0; Memory used: 0.004GB
PID: 33699; Command: bedtools; Return code: 0; Memory used: 0.093GB
grep -wf /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/chr_keep.txt /project/shefflab/genomes/hg38/refgene_anno/default/hg38_introns.bed | bedtools sort -i stdin -faidx /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/chr_order.txt | bedtools sort -i stdin -faidx /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/hg38_introns_sort.bed
(33705,33706,33707)Command completed. Elapsed time: 0:00:05. Running peak memory: 6.178GB.
PID: 33705; Command: grep; Return code: 0; Memory used: 0.005GB
PID: 33707; Command: bedtools; Return code: 0; Memory used: 0.005GB
PID: 33706; Command: bedtools; Return code: 0; Memory used: 0.033GB
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_exons_coverage.bed
,/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_introns_coverage.bed
bedtools coverage -sorted -counts -s -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/hg38_exons_sort.bed -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_PE1.bam -g /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_exons_coverage.bed
(33719)Command completed. Elapsed time: 0:00:26. Running peak memory: 6.178GB.
PID: 33719; Command: bedtools; Return code: 0; Memory used: 0.013GB
bedtools coverage -sorted -counts -s -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/hg38_introns_sort.bed -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_PE1.bam -g /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_introns_coverage.bed
(33747)Command completed. Elapsed time: 0:00:29. Running peak memory: 6.178GB.
PID: 33747; Command: bedtools; Return code: 0; Memory used: 0.055GB
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_exons_rpkm.bed
awk -v OFS=' ' '{chrom[$4] = $1; if($4!=prev4) {chromStart[$4] = $2} strand[$4] = $6; readCount[$4] += $7; exonCount[$4] += 1; geneSizeKB[$4] += (sqrt(($3-$2+0.00000001)^2)/1000); gene[$4] = $4; chromEnd[$4]=$3; prev4=$4} END { for (a in readCount) { print chrom[a], chromStart[a], chromEnd[a], gene[a], (readCount[a]/20.754829)/geneSizeKB[a], strand[a]}}' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_exons_coverage.bed | awk '$5>0' | sort -k4 > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_exons_rpkm.bed
(33772,33773,33774)Command completed. Elapsed time: 0:00:01. Running peak memory: 6.178GB.
PID: 33772; Command: awk; Return code: 0; Memory used: 0.008GB
PID: 33774; Command: sort; Return code: 0; Memory used: 0.003GB
PID: 33773; Command: awk; Return code: 0; Memory used: 0.001GB
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_introns_rpkm.bed
awk -v OFS=' ' '{chrom[$4] = $1; if($4!=prev4) {chromStart[$4] = $2} strand[$4] = $6; readCount[$4] += $7; exonCount[$4] += 1; geneSizeKB[$4] += (sqrt(($3-$2+0.00000001)^2)/1000); gene[$4] = $4; chromEnd[$4]=$3; prev4=$4} END { for (a in readCount) { print chrom[a], chromStart[a], chromEnd[a], gene[a], (readCount[a]/20.754829)/geneSizeKB[a], strand[a]}}' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_introns_coverage.bed | awk '$5>0' | sort -k4 > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_introns_rpkm.bed
(33776,33777,33778)Command completed. Elapsed time: 0:00:01. Running peak memory: 6.178GB.
PID: 33776; Command: awk; Return code: 0; Memory used: 0.007GB
PID: 33778; Command: sort; Return code: 0; Memory used: 0.004GB
PID: 33777; Command: awk; Return code: 0; Memory used: 0.001GB
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_exon_intron_ratios.bed
join --nocheck-order -a1 -a2 -j4 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_introns_rpkm.bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_exons_rpkm.bed | awk -v OFS=' ' 'NF==11 {print $7, $8, $9, $1, ($10/$5), $11}' | sort -k1,1 -k2,2n > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_exon_intron_ratios.bed
(33781,33782,33783)Command completed. Elapsed time: 0:00:00. Running peak memory: 6.178GB.
PID: 33781; Command: join; Return code: 0; Memory used: 0.001GB
PID: 33783; Command: sort; Return code: 0; Memory used: 0.004GB
PID: 33782; Command: awk; Return code: 0; Memory used: 0.001GB
awk '{print $5}' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_exon_intron_ratios.bed | sort -n | awk ' { a[i++]=$1; } END { x=int((i+1)/2); if (x < (i+1)/2) print (a[x-1]+a[x])/2; else print a[x-1]; }'
mRNA_contamination
1.23 PEPPRO RES Target to produce:/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_mRNA_contamination.pdf
Rscript /scratch/jps3dp/tools/databio//peppro/tools/PEPPRO.R mrna -i /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_exon_intron_ratios.bed --annotate
(33789)mRNA contamination plot completed!
Command completed. Elapsed time: 0:00:05. Running peak memory: 6.178GB.
PID: 33789; Command: Rscript; Return code: 0; Memory used: 0.317GB
mRNA contamination
QC_hg38/H9_PRO-seq_1_mRNA_contamination.pdf mRNA contamination QC_hg38/H9_PRO-seq_1_mRNA_contamination.png PEPPRO OBJ Target to produce:/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_exon_intron_ratios.bed.gz
pigz -f -p 12 -f /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/QC_hg38/H9_PRO-seq_1_exon_intron_ratios.bed
(33810)Command completed. Elapsed time: 0:00:00. Running peak memory: 6.178GB.
PID: 33810; Command: pigz; Return code: 0; Memory used: 0.003GB
Produce bigWig files (06-15 15:36:13) elapsed: 71.0 TIME
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/signal_hg38/H9_PRO-seq_1_plus_exact_body_0-mer.bw
,/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/signal_hg38/H9_PRO-seq_1_plus_smooth_body_0-mer.bw
samtools index /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_plus.bam
(33818)Command completed. Elapsed time: 0:00:10. Running peak memory: 6.178GB.
PID: 33818; Command: samtools; Return code: 0; Memory used: 0.008GB
/scratch/jps3dp/tools/databio//peppro/tools/bamSitesToWig.py -i /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_plus.bam -c /project/shefflab/genomes/hg38/fasta/default/hg38.chrom.sizes -o /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/signal_hg38/H9_PRO-seq_1_plus_exact_body_0-mer.bw -w /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/signal_hg38/H9_PRO-seq_1_plus_smooth_body_0-mer.bw -p 8 --variable-step --tail-edge --scale 20754829.0
(33829)Cutting parallel chroms in half to accommodate two tracks. Registering input file: '/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_plus.bam' Temporary files will be stored in: 'tmp_H9_PRO-seq_1_plus_cuttrace_exvi9dtb' Processing with 4 cores... stdin is empty of data Discarding 115 chunk(s) of reads: ['chrM', 'chr1_KI270709v1_random', 'chr1_KI270710v1_random', 'chr1_KI270711v1_random', 'chr2_KI270715v1_random', 'chr2_KI270716v1_random', 'chr5_GL000208v1_random', 'chr9_KI270718v1_random', 'chr9_KI270720v1_random', 'chr14_KI270724v1_random', 'chr22_KI270735v1_random', 'chr22_KI270737v1_random', 'chr22_KI270739v1_random', 'chrY_KI270740v1_random', 'chrUn_KI270302v1', 'chrUn_KI270304v1', 'chrUn_KI270303v1', 'chrUn_KI270305v1', 'chrUn_KI270322v1', 'chrUn_KI270320v1', 'chrUn_KI270310v1', 'chrUn_KI270316v1', 'chrUn_KI270315v1', 'chrUn_KI270312v1', 'chrUn_KI270311v1', 'chrUn_KI270317v1', 'chrUn_KI270412v1', 'chrUn_KI270411v1', 'chrUn_KI270414v1', 'chrUn_KI270419v1', 'chrUn_KI270418v1', 'chrUn_KI270420v1', 'chrUn_KI270424v1', 'chrUn_KI270417v1', 'chrUn_KI270422v1', 'chrUn_KI270423v1', 'chrUn_KI270425v1', 'chrUn_KI270429v1', 'chrUn_KI270465v1', 'chrUn_KI270467v1', 'chrUn_KI270435v1', 'chrUn_KI270468v1', 'chrUn_KI270510v1', 'chrUn_KI270509v1', 'chrUn_KI270518v1', 'chrUn_KI270508v1', 'chrUn_KI270516v1', 'chrUn_KI270512v1', 'chrUn_KI270519v1', 'chrUn_KI270522v1', 'chrUn_KI270511v1', 'chrUn_KI270507v1', 'chrUn_KI270517v1', 'chrUn_KI270529v1', 'chrUn_KI270528v1', 'chrUn_KI270530v1', 'chrUn_KI270538v1', 'chrUn_KI270544v1', 'chrUn_KI270548v1', 'chrUn_KI270583v1', 'chrUn_KI270580v1', 'chrUn_KI270581v1', 'chrUn_KI270579v1', 'chrUn_KI270590v1', 'chrUn_KI270582v1', 'chrUn_KI270588v1', 'chrUn_KI270593v1', 'chrUn_KI270591v1', 'chrUn_KI270330v1', 'chrUn_KI270329v1', 'chrUn_KI270334v1', 'chrUn_KI270333v1', 'chrUn_KI270335v1', 'chrUn_KI270338v1', 'chrUn_KI270340v1', 'chrUn_KI270336v1', 'chrUn_KI270337v1', 'chrUn_KI270363v1', 'chrUn_KI270364v1', 'chrUn_KI270362v1', 'chrUn_KI270366v1', 'chrUn_KI270378v1', 'chrUn_KI270379v1', 'chrUn_KI270389v1', 'chrUn_KI270390v1', 'chrUn_KI270387v1', 'chrUn_KI270395v1', 'chrUn_KI270396v1', 'chrUn_KI270388v1', 'chrUn_KI270394v1', 'chrUn_KI270386v1', 'chrUn_KI270391v1', 'chrUn_KI270383v1', 'chrUn_KI270393v1', 'chrUn_KI270384v1', 'chrUn_KI270392v1', 'chrUn_KI270381v1', 'chrUn_KI270385v1', 'chrUn_KI270382v1', 'chrUn_KI270376v1', 'chrUn_KI270374v1', 'chrUn_KI270372v1', 'chrUn_KI270373v1', 'chrUn_KI270375v1', 'chrUn_KI270371v1', 'chrUn_KI270448v1', 'chrUn_KI270521v1', 'chrUn_GL000226v1', 'chrUn_GL000213v1', 'chrUn_KI270746v1', 'chrUn_KI270749v1', 'chrUn_KI270752v1', 'chrUn_KI270754v1', 'chrUn_KI270755v1', 'chrUn_KI270756v1'] Keeping 80 chunk(s) of reads: ['chr1', 'chr2', 'chr3', 'chr4', 'chr5', 'chr6', 'chr7', 'chr8', 'chr9', 'chr10', 'chr11', 'chr12', 'chr13', 'chr14', 'chr15', 'chr16', 'chr17', 'chr18', 'chr19', 'chr20', 'chr21', 'chr22', 'chrX', 'chrY', 'chr1_KI270706v1_random', 'chr1_KI270707v1_random', 'chr1_KI270708v1_random', 'chr1_KI270712v1_random', 'chr1_KI270713v1_random', 'chr1_KI270714v1_random', 'chr3_GL000221v1_random', 'chr4_GL000008v2_random', 'chr9_KI270717v1_random', 'chr9_KI270719v1_random', 'chr11_KI270721v1_random', 'chr14_GL000009v2_random', 'chr14_GL000225v1_random', 'chr14_KI270722v1_random', 'chr14_GL000194v1_random', 'chr14_KI270723v1_random', 'chr14_KI270725v1_random', 'chr14_KI270726v1_random', 'chr15_KI270727v1_random', 'chr16_KI270728v1_random', 'chr17_GL000205v2_random', 'chr17_KI270729v1_random', 'chr17_KI270730v1_random', 'chr22_KI270731v1_random', 'chr22_KI270732v1_random', 'chr22_KI270733v1_random', 'chr22_KI270734v1_random', 'chr22_KI270736v1_random', 'chr22_KI270738v1_random', 'chrUn_KI270442v1', 'chrUn_KI270466v1', 'chrUn_KI270438v1', 'chrUn_KI270515v1', 'chrUn_KI270539v1', 'chrUn_KI270587v1', 'chrUn_KI270589v1', 'chrUn_KI270584v1', 'chrUn_GL000195v1', 'chrUn_GL000219v1', 'chrUn_GL000220v1', 'chrUn_GL000224v1', 'chrUn_KI270741v1', 'chrUn_KI270743v1', 'chrUn_KI270744v1', 'chrUn_KI270745v1', 'chrUn_KI270747v1', 'chrUn_KI270748v1', 'chrUn_KI270750v1', 'chrUn_KI270751v1', 'chrUn_KI270753v1', 'chrUn_KI270757v1', 'chrUn_GL000214v1', 'chrUn_KI270742v1', 'chrUn_GL000216v2', 'chrUn_GL000218v1', 'chrEBV'] Reduce step (merge files)... Merging 80 files into output file: '/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/signal_hg38/H9_PRO-seq_1_plus_exact_body_0-mer.bw' Merging 80 files into output file: '/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/signal_hg38/H9_PRO-seq_1_plus_smooth_body_0-mer.bw'Command completed. Elapsed time: 0:06:18. Running peak memory: 6.178GB.
PID: 33829; Command: /scratch/jps3dp/tools/databio//peppro/tools/bamSitesToWig.py; Return code: 0; Memory used: 3.346GB
Target to produce: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/signal_hg38/H9_PRO-seq_1_minus_exact_body_0-mer.bw
,/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/signal_hg38/H9_PRO-seq_1_minus_smooth_body_0-mer.bw
samtools index /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_minus.bam
(35275)Command completed. Elapsed time: 0:00:08. Running peak memory: 6.178GB.
PID: 35275; Command: samtools; Return code: 0; Memory used: 0.009GB
/scratch/jps3dp/tools/databio//peppro/tools/bamSitesToWig.py -i /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_minus.bam -c /project/shefflab/genomes/hg38/fasta/default/hg38.chrom.sizes -o /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/signal_hg38/H9_PRO-seq_1_minus_exact_body_0-mer.bw -w /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/signal_hg38/H9_PRO-seq_1_minus_smooth_body_0-mer.bw -p 8 --variable-step --tail-edge --scale 20754829.0
(35283)Cutting parallel chroms in half to accommodate two tracks. Registering input file: '/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/aligned_hg38/H9_PRO-seq_1_minus.bam' Temporary files will be stored in: 'tmp_H9_PRO-seq_1_minus_cuttrace_j2gcnfml' Processing with 4 cores... stdin is empty of data stdin is empty of data Discarding 109 chunk(s) of reads: ['chrM', 'chr1_KI270707v1_random', 'chr1_KI270709v1_random', 'chr1_KI270710v1_random', 'chr2_KI270715v1_random', 'chr9_KI270717v1_random', 'chr9_KI270720v1_random', 'chr11_KI270721v1_random', 'chr14_KI270722v1_random', 'chr14_KI270724v1_random', 'chr17_KI270730v1_random', 'chr22_KI270731v1_random', 'chr22_KI270735v1_random', 'chr22_KI270739v1_random', 'chrY_KI270740v1_random', 'chrUn_KI270302v1', 'chrUn_KI270304v1', 'chrUn_KI270303v1', 'chrUn_KI270305v1', 'chrUn_KI270322v1', 'chrUn_KI270320v1', 'chrUn_KI270310v1', 'chrUn_KI270316v1', 'chrUn_KI270315v1', 'chrUn_KI270312v1', 'chrUn_KI270311v1', 'chrUn_KI270317v1', 'chrUn_KI270412v1', 'chrUn_KI270411v1', 'chrUn_KI270414v1', 'chrUn_KI270419v1', 'chrUn_KI270418v1', 'chrUn_KI270420v1', 'chrUn_KI270424v1', 'chrUn_KI270417v1', 'chrUn_KI270422v1', 'chrUn_KI270423v1', 'chrUn_KI270425v1', 'chrUn_KI270429v1', 'chrUn_KI270465v1', 'chrUn_KI270468v1', 'chrUn_KI270510v1', 'chrUn_KI270509v1', 'chrUn_KI270518v1', 'chrUn_KI270508v1', 'chrUn_KI270516v1', 'chrUn_KI270512v1', 'chrUn_KI270522v1', 'chrUn_KI270511v1', 'chrUn_KI270515v1', 'chrUn_KI270507v1', 'chrUn_KI270517v1', 'chrUn_KI270529v1', 'chrUn_KI270528v1', 'chrUn_KI270530v1', 'chrUn_KI270539v1', 'chrUn_KI270538v1', 'chrUn_KI270544v1', 'chrUn_KI270548v1', 'chrUn_KI270583v1', 'chrUn_KI270587v1', 'chrUn_KI270580v1', 'chrUn_KI270581v1', 'chrUn_KI270590v1', 'chrUn_KI270584v1', 'chrUn_KI270582v1', 'chrUn_KI270588v1', 'chrUn_KI270591v1', 'chrUn_KI270330v1', 'chrUn_KI270329v1', 'chrUn_KI270334v1', 'chrUn_KI270335v1', 'chrUn_KI270338v1', 'chrUn_KI270340v1', 'chrUn_KI270336v1', 'chrUn_KI270363v1', 'chrUn_KI270364v1', 'chrUn_KI270378v1', 'chrUn_KI270379v1', 'chrUn_KI270389v1', 'chrUn_KI270390v1', 'chrUn_KI270387v1', 'chrUn_KI270395v1', 'chrUn_KI270396v1', 'chrUn_KI270388v1', 'chrUn_KI270394v1', 'chrUn_KI270386v1', 'chrUn_KI270391v1', 'chrUn_KI270383v1', 'chrUn_KI270393v1', 'chrUn_KI270384v1', 'chrUn_KI270392v1', 'chrUn_KI270381v1', 'chrUn_KI270385v1', 'chrUn_KI270382v1', 'chrUn_KI270376v1', 'chrUn_KI270374v1', 'chrUn_KI270372v1', 'chrUn_KI270373v1', 'chrUn_KI270375v1', 'chrUn_KI270371v1', 'chrUn_KI270448v1', 'chrUn_GL000226v1', 'chrUn_GL000213v1', 'chrUn_KI270745v1', 'chrUn_KI270748v1', 'chrUn_KI270754v1', 'chrUn_KI270755v1', 'chrUn_KI270757v1'] Keeping 86 chunk(s) of reads: ['chr1', 'chr2', 'chr3', 'chr4', 'chr5', 'chr6', 'chr7', 'chr8', 'chr9', 'chr10', 'chr11', 'chr12', 'chr13', 'chr14', 'chr15', 'chr16', 'chr17', 'chr18', 'chr19', 'chr20', 'chr21', 'chr22', 'chrX', 'chrY', 'chr1_KI270706v1_random', 'chr1_KI270708v1_random', 'chr1_KI270711v1_random', 'chr1_KI270712v1_random', 'chr1_KI270713v1_random', 'chr1_KI270714v1_random', 'chr2_KI270716v1_random', 'chr3_GL000221v1_random', 'chr4_GL000008v2_random', 'chr5_GL000208v1_random', 'chr9_KI270718v1_random', 'chr9_KI270719v1_random', 'chr14_GL000009v2_random', 'chr14_GL000225v1_random', 'chr14_GL000194v1_random', 'chr14_KI270723v1_random', 'chr14_KI270725v1_random', 'chr14_KI270726v1_random', 'chr15_KI270727v1_random', 'chr16_KI270728v1_random', 'chr17_GL000205v2_random', 'chr17_KI270729v1_random', 'chr22_KI270732v1_random', 'chr22_KI270733v1_random', 'chr22_KI270734v1_random', 'chr22_KI270736v1_random', 'chr22_KI270737v1_random', 'chr22_KI270738v1_random', 'chrUn_KI270442v1', 'chrUn_KI270466v1', 'chrUn_KI270467v1', 'chrUn_KI270435v1', 'chrUn_KI270438v1', 'chrUn_KI270519v1', 'chrUn_KI270579v1', 'chrUn_KI270589v1', 'chrUn_KI270593v1', 'chrUn_KI270333v1', 'chrUn_KI270337v1', 'chrUn_KI270362v1', 'chrUn_KI270366v1', 'chrUn_KI270521v1', 'chrUn_GL000195v1', 'chrUn_GL000219v1', 'chrUn_GL000220v1', 'chrUn_GL000224v1', 'chrUn_KI270741v1', 'chrUn_KI270743v1', 'chrUn_KI270744v1', 'chrUn_KI270746v1', 'chrUn_KI270747v1', 'chrUn_KI270749v1', 'chrUn_KI270750v1', 'chrUn_KI270751v1', 'chrUn_KI270752v1', 'chrUn_KI270753v1', 'chrUn_KI270756v1', 'chrUn_GL000214v1', 'chrUn_KI270742v1', 'chrUn_GL000216v2', 'chrUn_GL000218v1', 'chrEBV'] Reduce step (merge files)... Merging 86 files into output file: '/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/signal_hg38/H9_PRO-seq_1_minus_exact_body_0-mer.bw' Merging 86 files into output file: '/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_1/signal_hg38/H9_PRO-seq_1_minus_smooth_body_0-mer.bw'Command completed. Elapsed time: 0:07:12. Running peak memory: 6.178GB.
PID: 35283; Command: /scratch/jps3dp/tools/databio//peppro/tools/bamSitesToWig.py; Return code: 0; Memory used: 2.763GB
Pipeline completed. Epilogue
- Elapsed time (this run): 2:50:51
- Total elapsed time (all runs): 5:20:58
- Peak memory (this run): 6.178 GB
- Pipeline completed time: 2020-06-15 15:50:01