### Pipeline run code and environment:
* Command: `/scratch/jps3dp/tools/databio//peppro/pipelines/peppro.py --sample-name K562_RNA-seq_50 --genome hg38 --input /project/shefflab/data//guertin/fastq/K562_50pctRNA.fastq.gz --single-or-paired SINGLE -O /project/shefflab/processed//peppro/paper/6.11.2020/results_pipeline -P 12 -M 12000 --protocol PRO --umi-len 0 --scale --prealignments human_rDNA --dirty`
* Compute host: udc-ba26-10
* Working dir: /sfs/lustre/bahamut/scratch/jps3dp/tools/databio/ppqc
* Outfolder: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/
* Pipeline started at: (06-11 17:13:57) elapsed: 7.0 _TIME_
### Version log:
* Python version: 3.6.6
* Pypiper dir: `/sfs/qumulo/qhome/jps3dp/.local/lib/python3.6/site-packages/pypiper`
* Pypiper version: 0.12.1
* Pipeline dir: `/sfs/lustre/bahamut/scratch/jps3dp/tools/databio/peppro/pipelines`
* Pipeline version: 0.9.8
* Pipeline hash: 887a9a85408962dc3ca070dc954e59a5d4d73a10
* Pipeline branch: * master
* Pipeline date: 2020-06-09 11:36:49 -0400
* Pipeline diff: 40 files changed, 16373 insertions(+), 3522 deletions(-)
### Arguments passed to pipeline:
* `TSS_name`: `None`
* `adapter`: `cutadapt`
* `anno_name`: `None`
* `complexity`: `False`
* `config_file`: `peppro.yaml`
* `cores`: `12`
* `coverage`: `False`
* `dedup`: `seqkit`
* `dirty`: `True`
* `ensembl_gene_body`: `None`
* `ensembl_tss`: `None`
* `exon_name`: `None`
* `force_follow`: `False`
* `genome_assembly`: `hg38`
* `input`: `['/project/shefflab/data//guertin/fastq/K562_50pctRNA.fastq.gz']`
* `input2`: `None`
* `intron_name`: `None`
* `keep`: `False`
* `logdev`: `False`
* `max_len`: `-1`
* `mem`: `12000`
* `new_start`: `False`
* `no_fifo`: `False`
* `output_parent`: `/project/shefflab/processed//peppro/paper/6.11.2020/results_pipeline`
* `paired_end`: `False`
* `pre_name`: `None`
* `prealignments`: `['human_rDNA']`
* `prioritize`: `False`
* `protocol`: `PRO`
* `recover`: `False`
* `sample_name`: `K562_RNA-seq_50`
* `scale`: `True`
* `search_file`: `None`
* `silent`: `False`
* `single_or_paired`: `SINGLE`
* `sob`: `False`
* `testmode`: `False`
* `trimmer`: `seqtk`
* `umi_len`: `0`
* `verbosity`: `None`
----------------------------------------
Local input file: /project/shefflab/data//guertin/fastq/K562_50pctRNA.fastq.gz
> `File_mb` 5536.54 PEPPRO _RES_
> `Read_type` SINGLE PEPPRO _RES_
> `Genome` hg38 PEPPRO _RES_
Detected PRO input
### Merge/link and fastq conversion: (06-11 17:13:58) elapsed: 1.0 _TIME_
Number of input file sets: 1
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/raw/K562_RNA-seq_50.fastq.gz`
> `ln -sf /project/shefflab/data//guertin/fastq/K562_50pctRNA.fastq.gz /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/raw/K562_RNA-seq_50.fastq.gz` (167230)
Command completed. Elapsed time: 0:00:00. Running peak memory: 0GB.
PID: 167230; Command: ln; Return code: 0; Memory used: 0.0GB
Local input file: '/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/raw/K562_RNA-seq_50.fastq.gz'
Found .fastq.gz file
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/fastq/K562_RNA-seq_50_R1.fastq`
> `pigz -f -p 12 -d -c /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/raw/K562_RNA-seq_50.fastq.gz > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/fastq/K562_RNA-seq_50_R1.fastq` (167231)
Command completed. Elapsed time: 0:02:58. Running peak memory: 0.002GB.
PID: 167231; Command: pigz; Return code: 0; Memory used: 0.002GB
> `Raw_reads` 70000000 PEPPRO _RES_
> `Fastq_reads` 70000000 PEPPRO _RES_
['/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/raw/K562_RNA-seq_50.fastq.gz']
### FASTQ processing: (06-11 17:18:58) elapsed: 300.0 _TIME_
> `cutadapt --version`
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/fastq/K562_RNA-seq_50_R1_processed.fastq`
> `(cutadapt -j 12 -m 2 -O 1 -a TGGAATTCTCGGGTGCCAAGG /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/fastq/K562_RNA-seq_50_R1.fastq -o /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/fastq/K562_RNA-seq_50_R1_noadap.fastq --too-short-output /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/fastq/K562_RNA-seq_50_R1_short.fastq ) > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/cutadapt/K562_RNA-seq_50_R1_cutadapt.txt` (168232)
Command completed. Elapsed time: 0:03:10. Running peak memory: 4.783GB.
PID: 168232; Command: cutadapt; Return code: 0; Memory used: 4.783GB
> `seqtk trimfq -b 0 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/fastq/K562_RNA-seq_50_R1_noadap.fastq | seqtk seq -L 2 -r - > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/fastq/K562_RNA-seq_50_R1_processed.fastq` (168703,168704)
Command completed. Elapsed time: 0:02:18. Running peak memory: 4.783GB.
PID: 168704; Command: seqtk; Return code: 0; Memory used: 0.002GB
PID: 168703; Command: seqtk; Return code: 0; Memory used: 0.001GB
> `grep 'Reads with adapters:' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/cutadapt/K562_RNA-seq_50_R1_cutadapt.txt | awk '{print $(NF-1)}'`
> `Reads_with_adapter` 42246395.0 PEPPRO _RES_
> `grep 'Total basepairs processed:' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/cutadapt/K562_RNA-seq_50_R1_cutadapt.txt | awk '{print $(NF-1)}'`
> `awk '{sum+=$1*$2} END {printf "%.0f", sum}' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/cutadapt/K562_RNA-seq_50_R1_cutadapt.txt`
> `wc -l /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/fastq/K562_RNA-seq_50_R1_short.fastq | awk '{print $1}'`
> `Uninformative_adapter_reads` 875215.0 PEPPRO _RES_
> `Pct_uninformative_adapter_reads` 1.2503 PEPPRO _RES_
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/fastqc/K562_RNA-seq_50_R1_processed_fastqc.html`
> `echo '### Calculate the number of trimmed reads'` (169377)
### Calculate the number of trimmed reads
Command completed. Elapsed time: 0:00:00. Running peak memory: 4.783GB.
PID: 169377; Command: echo; Return code: 0; Memory used: 0.0GB
Evaluating read trimming
> `Trimmed_reads` 69124785 PEPPRO _RES_
> `Trim_loss_rate` 1.25 PEPPRO _RES_
Targetless command, running...
> `fastqc --noextract --outdir /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/fastqc /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/fastq/K562_RNA-seq_50_R1_processed.fastq` (169588)
Started analysis of K562_RNA-seq_50_R1_processed.fastq
Approx 5% complete for K562_RNA-seq_50_R1_processed.fastq
Approx 10% complete for K562_RNA-seq_50_R1_processed.fastq
Approx 15% complete for K562_RNA-seq_50_R1_processed.fastq
Approx 20% complete for K562_RNA-seq_50_R1_processed.fastq
Approx 25% complete for K562_RNA-seq_50_R1_processed.fastq
Approx 30% complete for K562_RNA-seq_50_R1_processed.fastq
Approx 35% complete for K562_RNA-seq_50_R1_processed.fastq
Approx 40% complete for K562_RNA-seq_50_R1_processed.fastq
Approx 45% complete for K562_RNA-seq_50_R1_processed.fastq
Approx 50% complete for K562_RNA-seq_50_R1_processed.fastq
Approx 55% complete for K562_RNA-seq_50_R1_processed.fastq
Approx 60% complete for K562_RNA-seq_50_R1_processed.fastq
Approx 65% complete for K562_RNA-seq_50_R1_processed.fastq
Approx 70% complete for K562_RNA-seq_50_R1_processed.fastq
Approx 75% complete for K562_RNA-seq_50_R1_processed.fastq
Approx 80% complete for K562_RNA-seq_50_R1_processed.fastq
Approx 85% complete for K562_RNA-seq_50_R1_processed.fastq
Approx 90% complete for K562_RNA-seq_50_R1_processed.fastq
Approx 95% complete for K562_RNA-seq_50_R1_processed.fastq
Analysis complete for K562_RNA-seq_50_R1_processed.fastq
Command completed. Elapsed time: 0:04:36. Running peak memory: 4.783GB.
PID: 169588; Command: fastqc; Return code: 0; Memory used: 0.244GB
> `FastQC report r1` fastqc/K562_RNA-seq_50_R1_processed_fastqc.html FastQC report r1 None PEPPRO _OBJ_
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/fastq/processed_R1.flag`
> `touch /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/fastq/processed_R1.flag` (172751)
Command completed. Elapsed time: 0:00:00. Running peak memory: 4.783GB.
PID: 172751; Command: touch; Return code: 0; Memory used: 0.002GB
### Plot adapter insertion distribution (06-11 17:30:44) elapsed: 706.0 _TIME_
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/cutadapt/K562_RNA-seq_50_R1_adapter_insertion_distribution.pdf`
> `Rscript /scratch/jps3dp/tools/databio//peppro/tools/PEPPRO.R cutadapt -i /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/cutadapt/K562_RNA-seq_50_R1_cutadapt.txt -o /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/cutadapt` (172754)
Adapter insertion distribution plot completed!
Command completed. Elapsed time: 0:00:06. Running peak memory: 4.783GB.
PID: 172754; Command: Rscript; Return code: 0; Memory used: 0.205GB
> `Adapter insertion distribution` cutadapt/K562_RNA-seq_50_R1_adapter_insertion_distribution.pdf Adapter insertion distribution cutadapt/K562_RNA-seq_50_R1_adapter_insertion_distribution.png PEPPRO _OBJ_
Missing stat 'Peak_adapter_insertion_size'
> `awk '/count/,0' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/cutadapt/K562_RNA-seq_50_R1_cutadapt.txt | awk 'NR>2 {print prev} {prev=$0}' | awk '{if ($3/$2 < 0.01) print $1, $2}' | awk 'BEGIN{max= 0; max_len=0; len=0}{if ($2>0+max) {max=$2; len=$1}; max_len=$1} END{print max_len-len}'`
> `Peak_adapter_insertion_size` 34 PEPPRO _RES_
Missing stat 'Degradation_ratio'
### Calculating degradation ratio (06-11 17:30:50) elapsed: 6.0 _TIME_
> `awk 'NR>2 {print prev} {prev=$0}' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/cutadapt/K562_RNA-seq_50_R1_cutadapt.txt | awk '{ if ($1 == 10) {status = 1}} END {if (status) {print status} else {print 0}}'`
> `awk 'NR>2 {print prev} {prev=$0}' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/cutadapt/K562_RNA-seq_50_R1_cutadapt.txt | awk '{ if ($1 == 20) {status = 1}} END {if (status) {print status} else {print 0}}'`
> `awk 'NR>2 {print prev} {prev=$0}' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/cutadapt/K562_RNA-seq_50_R1_cutadapt.txt | awk '{ if ($1 == 30) {status = 1}} END {if (status) {print status} else {print 0}}'`
> `awk 'NR>2 {print prev} {prev=$0}' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/cutadapt/K562_RNA-seq_50_R1_cutadapt.txt | awk '{ if ($1 == 40) {status = 1}} END {if (status) {print status} else {print 0}}'`
> `awk '/count/,0' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/cutadapt/K562_RNA-seq_50_R1_cutadapt.txt | awk 'NR>2 {print prev} {prev=$0}' | awk '{if ($3/$2 < 0.01) print $1, $2}' | awk '{a[NR]=$1; b[NR]=$2; max_len=$1}{if ($1 > max_len) {max_len=$1}} END{ for (i in a) print 1+max_len-a[i], b[i]}' | sort -nk1 | awk '($1 <= 20 && $1 >= 10){degradedSum += $2}; ($1 >= 30 && $1 <= 40){intactSum += $2} END {if (intactSum < 1) {intactSum = 1} print degradedSum/intactSum}'`
> `Degradation_ratio` 0.231 PEPPRO _RES_
### Prealignments (06-11 17:30:50) elapsed: 0.0 _TIME_
Prealignment assemblies: ['human_rDNA']
### Map to human_rDNA (06-11 17:30:50) elapsed: 0.0 _TIME_
> `(bowtie2 -p 12 -k 1 -D 20 -R 3 -N 1 -L 20 -i S,1,0.50 -x /project/shefflab/genomes/human_rDNA/bowtie2_index/default/human_rDNA --rg-id K562_RNA-seq_50 -U /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/fastq/K562_RNA-seq_50_R1_processed.fastq --un /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/prealignments/K562_RNA-seq_50_human_rDNA_unmap.fq 2>&1 > /dev/null)`
Missing stat 'Aligned_reads_human_rDNA'
69124785 reads; of these:
69124785 (100.00%) were unpaired; of these:
65165732 (94.27%) aligned 0 times
3959053 (5.73%) aligned exactly 1 time
0 (0.00%) aligned >1 times
5.73% overall alignment rate
> `Aligned_reads_human_rDNA` 3959053.0 PEPPRO _RES_
> `Alignment_rate_human_rDNA` 5.73 PEPPRO _RES_
### Map to genome (06-11 17:38:29) elapsed: 459.0 _TIME_
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_sort.bam`
> `bowtie2 -p 12 --very-sensitive --rg-id K562_RNA-seq_50 -x /project/shefflab/genomes/hg38/bowtie2_index/default/hg38 -U /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/prealignments/K562_RNA-seq_50_human_rDNA_unmap.fq | samtools view -bS - -@ 1 | samtools sort - -@ 1 -T /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/tmpusv9ec2r -o /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_temp.bam` (177638,177647,177648)
65165732 reads; of these:
65165732 (100.00%) were unpaired; of these:
6142637 (9.43%) aligned 0 times
39932788 (61.28%) aligned exactly 1 time
19090307 (29.30%) aligned >1 times
90.57% overall alignment rate
[bam_sort_core] merging from 21 files and 1 in-memory blocks...
Command completed. Elapsed time: 0:41:02. Running peak memory: 4.783GB.
PID: 177647; Command: samtools; Return code: 0; Memory used: 0.004GB
PID: 177638; Command: bowtie2; Return code: 0; Memory used: 3.715GB
PID: 177648; Command: samtools; Return code: 0; Memory used: 0.959GB
> `samtools view -q 10 -b -@ 12 -U /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_fail_qc.bam /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_temp.bam > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_sort.bam` (11542)
Command completed. Elapsed time: 0:02:53. Running peak memory: 4.783GB.
PID: 11542; Command: samtools; Return code: 0; Memory used: 0.019GB
> `samtools depth -b /project/shefflab/genomes/hg38/refgene_anno/default/hg38_pre-mRNA.bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_sort.bam | awk '{counter++;sum+=$3}END{print sum/counter}'`
> `Mapped_reads` 59023095 PEPPRO _RES_
> `QC_filtered_reads` 9634021 PEPPRO _RES_
> `Aligned_reads` 49389074 PEPPRO _RES_
> `Alignment_rate` 71.45 PEPPRO _RES_
> `Total_efficiency` 70.56 PEPPRO _RES_
> `Read_depth` 6.57 PEPPRO _RES_
### Compress all unmapped read files (06-11 18:44:33) elapsed: 3965.0 _TIME_
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/prealignments/K562_RNA-seq_50_human_rDNA_unmap.fq.gz`
> `pigz -f -p 12 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/prealignments/K562_RNA-seq_50_human_rDNA_unmap.fq` (26942)
Command completed. Elapsed time: 0:02:35. Running peak memory: 4.783GB.
PID: 26942; Command: pigz; Return code: 0; Memory used: 0.011GB
Missing stat 'Mitochondrial_reads'
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_temp.bam.bai`
> `samtools index /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_temp.bam` (28473)
Command completed. Elapsed time: 0:01:13. Running peak memory: 4.783GB.
PID: 28473; Command: samtools; Return code: 0; Memory used: 0.015GB
> `samtools idxstats /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_temp.bam | grep -we 'chrM' -we 'chrMT' -we 'M' -we 'MT' -we 'rCRSd' -we 'rCRSd_3k'| cut -f 3`
> `Mitochondrial_reads` 2741453 PEPPRO _RES_
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_noMT.bam`
> `samtools index /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_sort.bam` (29054)
Command completed. Elapsed time: 0:00:56. Running peak memory: 4.783GB.
PID: 29054; Command: samtools; Return code: 0; Memory used: 0.015GB
> `samtools idxstats /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_sort.bam | cut -f 1-2 | awk '{print $1, 0, $2}' | grep -vwe 'chrM' -vwe 'chrMT' -vwe 'M' -vwe 'MT' -vwe 'rCRSd' -vwe 'rCRSd_3k' > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/chr_sizes.bed` (29495,29496,29498,29499)
Command completed. Elapsed time: 0:00:00. Running peak memory: 4.783GB.
PID: 29496; Command: cut; Return code: 0; Memory used: 0.0GB
PID: 29499; Command: grep; Return code: 0; Memory used: 0.0GB
PID: 29495; Command: samtools; Return code: 0; Memory used: 0.009GB
PID: 29498; Command: awk; Return code: 0; Memory used: 0.0GB
> `samtools view -L /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/chr_sizes.bed -b -@ 12 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_sort.bam > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_noMT.bam` (29501)
Command completed. Elapsed time: 0:01:05. Running peak memory: 4.783GB.
PID: 29501; Command: samtools; Return code: 0; Memory used: 0.019GB
> `mv /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_noMT.bam /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_sort.bam` (30351)
Command completed. Elapsed time: 0:00:00. Running peak memory: 4.783GB.
PID: 30351; Command: mv; Return code: 0; Memory used: 0.002GB
> `samtools index /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_sort.bam` (30353)
Command completed. Elapsed time: 0:00:51. Running peak memory: 4.783GB.
PID: 30353; Command: samtools; Return code: 0; Memory used: 0.015GB
Missing stat 'Maximum_read_length'
> `samtools stats /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_sort.bam | grep '^SN' | cut -f 2- | grep 'maximum length:' | cut -f 2-`
> `Maximum_read_length` 100 PEPPRO _RES_
Missing stat 'Genome_size'
> `awk '{sum+=$2} END {printf "%.0f", sum}' /project/shefflab/genomes/hg38/fasta/default/hg38.chrom.sizes`
> `Genome_size` 3099922541 PEPPRO _RES_
### Calculate NRF, PBC1, and PBC2 (06-11 18:54:00) elapsed: 567.0 _TIME_
Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_sort.bam.bai`
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_bamQC.tsv`
> `/scratch/jps3dp/tools/databio//peppro/tools/bamQC.py --silent -i /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_sort.bam -c 12 -o /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_bamQC.tsv` (32039)
Configured logger 'root' using pararead v0.6
Registering input file: '/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_sort.bam'
Temporary files will be stored in: '/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/tmp_K562_RNA-seq_50_sort__1aozd3o'
Processing with 12 cores...
Discarding 97 chunk(s) of reads: ['chrM', 'chr1_KI270709v1_random', 'chr1_KI270710v1_random', 'chr2_KI270715v1_random', 'chr2_KI270716v1_random', 'chr9_KI270717v1_random', 'chr22_KI270739v1_random', 'chrY_KI270740v1_random', 'chrUn_KI270302v1', 'chrUn_KI270304v1', 'chrUn_KI270303v1', 'chrUn_KI270305v1', 'chrUn_KI270322v1', 'chrUn_KI270320v1', 'chrUn_KI270310v1', 'chrUn_KI270316v1', 'chrUn_KI270315v1', 'chrUn_KI270312v1', 'chrUn_KI270311v1', 'chrUn_KI270317v1', 'chrUn_KI270412v1', 'chrUn_KI270411v1', 'chrUn_KI270414v1', 'chrUn_KI270419v1', 'chrUn_KI270418v1', 'chrUn_KI270420v1', 'chrUn_KI270424v1', 'chrUn_KI270417v1', 'chrUn_KI270422v1', 'chrUn_KI270423v1', 'chrUn_KI270425v1', 'chrUn_KI270429v1', 'chrUn_KI270465v1', 'chrUn_KI270468v1', 'chrUn_KI270510v1', 'chrUn_KI270509v1', 'chrUn_KI270518v1', 'chrUn_KI270508v1', 'chrUn_KI270516v1', 'chrUn_KI270512v1', 'chrUn_KI270522v1', 'chrUn_KI270511v1', 'chrUn_KI270507v1', 'chrUn_KI270517v1', 'chrUn_KI270529v1', 'chrUn_KI270528v1', 'chrUn_KI270530v1', 'chrUn_KI270539v1', 'chrUn_KI270544v1', 'chrUn_KI270548v1', 'chrUn_KI270587v1', 'chrUn_KI270581v1', 'chrUn_KI270579v1', 'chrUn_KI270588v1', 'chrUn_KI270593v1', 'chrUn_KI270329v1', 'chrUn_KI270334v1', 'chrUn_KI270333v1', 'chrUn_KI270335v1', 'chrUn_KI270338v1', 'chrUn_KI270340v1', 'chrUn_KI270336v1', 'chrUn_KI270337v1', 'chrUn_KI270363v1', 'chrUn_KI270364v1', 'chrUn_KI270362v1', 'chrUn_KI270366v1', 'chrUn_KI270378v1', 'chrUn_KI270379v1', 'chrUn_KI270389v1', 'chrUn_KI270390v1', 'chrUn_KI270387v1', 'chrUn_KI270395v1', 'chrUn_KI270396v1', 'chrUn_KI270388v1', 'chrUn_KI270394v1', 'chrUn_KI270386v1', 'chrUn_KI270391v1', 'chrUn_KI270383v1', 'chrUn_KI270393v1', 'chrUn_KI270384v1', 'chrUn_KI270392v1', 'chrUn_KI270381v1', 'chrUn_KI270385v1', 'chrUn_KI270382v1', 'chrUn_KI270376v1', 'chrUn_KI270374v1', 'chrUn_KI270372v1', 'chrUn_KI270373v1', 'chrUn_KI270375v1', 'chrUn_KI270371v1', 'chrUn_GL000226v1', 'chrUn_KI270747v1', 'chrUn_KI270752v1', 'chrUn_KI270753v1', 'chrUn_KI270755v1', 'chrUn_KI270756v1']
Keeping 98 chunk(s) of reads: ['chr1', 'chr2', 'chr3', 'chr4', 'chr5', 'chr6', 'chr7', 'chr8', 'chr9', 'chr10', 'chr11', 'chr12', 'chr13', 'chr14', 'chr15', 'chr16', 'chr17', 'chr18', 'chr19', 'chr20', 'chr21', 'chr22', 'chrX', 'chrY', 'chr1_KI270706v1_random', 'chr1_KI270707v1_random', 'chr1_KI270708v1_random', 'chr1_KI270711v1_random', 'chr1_KI270712v1_random', 'chr1_KI270713v1_random', 'chr1_KI270714v1_random', 'chr3_GL000221v1_random', 'chr4_GL000008v2_random', 'chr5_GL000208v1_random', 'chr9_KI270718v1_random', 'chr9_KI270719v1_random', 'chr9_KI270720v1_random', 'chr11_KI270721v1_random', 'chr14_GL000009v2_random', 'chr14_GL000225v1_random', 'chr14_KI270722v1_random', 'chr14_GL000194v1_random', 'chr14_KI270723v1_random', 'chr14_KI270724v1_random', 'chr14_KI270725v1_random', 'chr14_KI270726v1_random', 'chr15_KI270727v1_random', 'chr16_KI270728v1_random', 'chr17_GL000205v2_random', 'chr17_KI270729v1_random', 'chr17_KI270730v1_random', 'chr22_KI270731v1_random', 'chr22_KI270732v1_random', 'chr22_KI270733v1_random', 'chr22_KI270734v1_random', 'chr22_KI270735v1_random', 'chr22_KI270736v1_random', 'chr22_KI270737v1_random', 'chr22_KI270738v1_random', 'chrUn_KI270442v1', 'chrUn_KI270466v1', 'chrUn_KI270467v1', 'chrUn_KI270435v1', 'chrUn_KI270438v1', 'chrUn_KI270519v1', 'chrUn_KI270515v1', 'chrUn_KI270538v1', 'chrUn_KI270583v1', 'chrUn_KI270580v1', 'chrUn_KI270589v1', 'chrUn_KI270590v1', 'chrUn_KI270584v1', 'chrUn_KI270582v1', 'chrUn_KI270591v1', 'chrUn_KI270330v1', 'chrUn_KI270448v1', 'chrUn_KI270521v1', 'chrUn_GL000195v1', 'chrUn_GL000219v1', 'chrUn_GL000220v1', 'chrUn_GL000224v1', 'chrUn_KI270741v1', 'chrUn_GL000213v1', 'chrUn_KI270743v1', 'chrUn_KI270744v1', 'chrUn_KI270745v1', 'chrUn_KI270746v1', 'chrUn_KI270748v1', 'chrUn_KI270749v1', 'chrUn_KI270750v1', 'chrUn_KI270751v1', 'chrUn_KI270754v1', 'chrUn_KI270757v1', 'chrUn_GL000214v1', 'chrUn_KI270742v1', 'chrUn_GL000216v2', 'chrUn_GL000218v1', 'chrEBV']
Command completed. Elapsed time: 0:01:14. Running peak memory: 4.783GB.
PID: 32039; Command: /scratch/jps3dp/tools/databio//peppro/tools/bamQC.py; Return code: 0; Memory used: 2.006GB
> `awk '{ for (i=1; i<=NF; ++i) { if ($i ~ "NRF") c=i } getline; print $c }' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_bamQC.tsv`
Got SIGTERM. Failing gracefully... (06-11 18:55:14) elapsed: 74.0 _TIME_
Setting dynamic recover file: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/recover.lock.QC_hg38__K562_RNA-seq_50_bamQC.tsv
### Pipeline failed at: (06-11 18:55:14) elapsed: 0.0 _TIME_
Total time: 1:41:25
Failure reason: SIGTERM
Pipeline aborted.
### Pipeline run code and environment:
* Command: `/scratch/jps3dp/tools/databio//peppro/pipelines/peppro.py --sample-name K562_RNA-seq_50 --genome hg38 --input /project/shefflab/data//guertin/fastq/K562_50pctRNA.fastq.gz --single-or-paired SINGLE -O /project/shefflab/processed//peppro/paper/6.11.2020/results_pipeline -P 12 -M 12000 --protocol PRO --umi-len 0 --scale --prealignments human_rDNA --dirty -R`
* Compute host: udc-ba25-32c1
* Working dir: /sfs/lustre/bahamut/scratch/jps3dp/tools/databio/ppqc
* Outfolder: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/
* Pipeline started at: (06-11 19:08:42) elapsed: 1.0 _TIME_
### Version log:
* Python version: 3.6.6
* Pypiper dir: `/sfs/qumulo/qhome/jps3dp/.local/lib/python3.6/site-packages/pypiper`
* Pypiper version: 0.12.1
* Pipeline dir: `/sfs/lustre/bahamut/scratch/jps3dp/tools/databio/peppro/pipelines`
* Pipeline version: 0.9.8
* Pipeline hash: 887a9a85408962dc3ca070dc954e59a5d4d73a10
* Pipeline branch: * master
* Pipeline date: 2020-06-09 11:36:49 -0400
* Pipeline diff: 40 files changed, 16373 insertions(+), 3522 deletions(-)
### Arguments passed to pipeline:
* `TSS_name`: `None`
* `adapter`: `cutadapt`
* `anno_name`: `None`
* `complexity`: `False`
* `config_file`: `peppro.yaml`
* `cores`: `12`
* `coverage`: `False`
* `dedup`: `seqkit`
* `dirty`: `True`
* `ensembl_gene_body`: `None`
* `ensembl_tss`: `None`
* `exon_name`: `None`
* `force_follow`: `False`
* `genome_assembly`: `hg38`
* `input`: `['/project/shefflab/data//guertin/fastq/K562_50pctRNA.fastq.gz']`
* `input2`: `None`
* `intron_name`: `None`
* `keep`: `False`
* `logdev`: `False`
* `max_len`: `-1`
* `mem`: `12000`
* `new_start`: `False`
* `no_fifo`: `False`
* `output_parent`: `/project/shefflab/processed//peppro/paper/6.11.2020/results_pipeline`
* `paired_end`: `False`
* `pre_name`: `None`
* `prealignments`: `['human_rDNA']`
* `prioritize`: `False`
* `protocol`: `PRO`
* `recover`: `True`
* `sample_name`: `K562_RNA-seq_50`
* `scale`: `True`
* `search_file`: `None`
* `silent`: `False`
* `single_or_paired`: `SINGLE`
* `sob`: `False`
* `testmode`: `False`
* `trimmer`: `seqtk`
* `umi_len`: `0`
* `verbosity`: `None`
----------------------------------------
Removed existing flag: '/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/PEPPRO_failed.flag'
Local input file: /project/shefflab/data//guertin/fastq/K562_50pctRNA.fastq.gz
> `File_mb` 5536.54 PEPPRO _RES_
> `Read_type` SINGLE PEPPRO _RES_
> `Genome` hg38 PEPPRO _RES_
Detected PRO input
### Merge/link and fastq conversion: (06-11 19:08:43) elapsed: 1.0 _TIME_
Number of input file sets: 1
Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/raw/K562_RNA-seq_50.fastq.gz`
Local input file: '/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/raw/K562_RNA-seq_50.fastq.gz'
Found .fastq.gz file
Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/fastq/K562_RNA-seq_50_R1.fastq`
['/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/raw/K562_RNA-seq_50.fastq.gz']
### FASTQ processing: (06-11 19:08:43) elapsed: 0.0 _TIME_
Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/fastq/processed_R1.flag`
### Plot adapter insertion distribution (06-11 19:08:43) elapsed: 0.0 _TIME_
Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/cutadapt/K562_RNA-seq_50_R1_adapter_insertion_distribution.pdf`
> `Adapter insertion distribution` cutadapt/K562_RNA-seq_50_R1_adapter_insertion_distribution.pdf Adapter insertion distribution cutadapt/K562_RNA-seq_50_R1_adapter_insertion_distribution.png PEPPRO _OBJ_
### Prealignments (06-11 19:08:43) elapsed: 0.0 _TIME_
Prealignment assemblies: ['human_rDNA']
### Map to human_rDNA (06-11 19:08:43) elapsed: 0.0 _TIME_
No files match cleanup pattern: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/prealignments/K562_RNA-seq_50_human_rDNA_unmap.fq
### Map to genome (06-11 19:08:43) elapsed: 0.0 _TIME_
Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_sort.bam`
Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_sort.bam`
### Compress all unmapped read files (06-11 19:08:44) elapsed: 0.0 _TIME_
Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/prealignments/K562_RNA-seq_50_human_rDNA_unmap.fq.gz`
### Calculate NRF, PBC1, and PBC2 (06-11 19:08:44) elapsed: 0.0 _TIME_
Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_sort.bam.bai`
Found lock file: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/lock.QC_hg38__K562_RNA-seq_50_bamQC.tsv
Overwriting target...
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_bamQC.tsv`
> `/scratch/jps3dp/tools/databio//peppro/tools/bamQC.py --silent -i /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_sort.bam -c 12 -o /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_bamQC.tsv` (98040)
Configured logger 'root' using pararead v0.6
Output file already exists and will be overwritten: '/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_bamQC.tsv'
Registering input file: '/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_sort.bam'
Temporary files will be stored in: '/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/tmp_K562_RNA-seq_50_sort_euhivhzr'
Processing with 12 cores...
Discarding 97 chunk(s) of reads: ['chrM', 'chr1_KI270709v1_random', 'chr1_KI270710v1_random', 'chr2_KI270715v1_random', 'chr2_KI270716v1_random', 'chr9_KI270717v1_random', 'chr22_KI270739v1_random', 'chrY_KI270740v1_random', 'chrUn_KI270302v1', 'chrUn_KI270304v1', 'chrUn_KI270303v1', 'chrUn_KI270305v1', 'chrUn_KI270322v1', 'chrUn_KI270320v1', 'chrUn_KI270310v1', 'chrUn_KI270316v1', 'chrUn_KI270315v1', 'chrUn_KI270312v1', 'chrUn_KI270311v1', 'chrUn_KI270317v1', 'chrUn_KI270412v1', 'chrUn_KI270411v1', 'chrUn_KI270414v1', 'chrUn_KI270419v1', 'chrUn_KI270418v1', 'chrUn_KI270420v1', 'chrUn_KI270424v1', 'chrUn_KI270417v1', 'chrUn_KI270422v1', 'chrUn_KI270423v1', 'chrUn_KI270425v1', 'chrUn_KI270429v1', 'chrUn_KI270465v1', 'chrUn_KI270468v1', 'chrUn_KI270510v1', 'chrUn_KI270509v1', 'chrUn_KI270518v1', 'chrUn_KI270508v1', 'chrUn_KI270516v1', 'chrUn_KI270512v1', 'chrUn_KI270522v1', 'chrUn_KI270511v1', 'chrUn_KI270507v1', 'chrUn_KI270517v1', 'chrUn_KI270529v1', 'chrUn_KI270528v1', 'chrUn_KI270530v1', 'chrUn_KI270539v1', 'chrUn_KI270544v1', 'chrUn_KI270548v1', 'chrUn_KI270587v1', 'chrUn_KI270581v1', 'chrUn_KI270579v1', 'chrUn_KI270588v1', 'chrUn_KI270593v1', 'chrUn_KI270329v1', 'chrUn_KI270334v1', 'chrUn_KI270333v1', 'chrUn_KI270335v1', 'chrUn_KI270338v1', 'chrUn_KI270340v1', 'chrUn_KI270336v1', 'chrUn_KI270337v1', 'chrUn_KI270363v1', 'chrUn_KI270364v1', 'chrUn_KI270362v1', 'chrUn_KI270366v1', 'chrUn_KI270378v1', 'chrUn_KI270379v1', 'chrUn_KI270389v1', 'chrUn_KI270390v1', 'chrUn_KI270387v1', 'chrUn_KI270395v1', 'chrUn_KI270396v1', 'chrUn_KI270388v1', 'chrUn_KI270394v1', 'chrUn_KI270386v1', 'chrUn_KI270391v1', 'chrUn_KI270383v1', 'chrUn_KI270393v1', 'chrUn_KI270384v1', 'chrUn_KI270392v1', 'chrUn_KI270381v1', 'chrUn_KI270385v1', 'chrUn_KI270382v1', 'chrUn_KI270376v1', 'chrUn_KI270374v1', 'chrUn_KI270372v1', 'chrUn_KI270373v1', 'chrUn_KI270375v1', 'chrUn_KI270371v1', 'chrUn_GL000226v1', 'chrUn_KI270747v1', 'chrUn_KI270752v1', 'chrUn_KI270753v1', 'chrUn_KI270755v1', 'chrUn_KI270756v1']
Keeping 98 chunk(s) of reads: ['chr1', 'chr2', 'chr3', 'chr4', 'chr5', 'chr6', 'chr7', 'chr8', 'chr9', 'chr10', 'chr11', 'chr12', 'chr13', 'chr14', 'chr15', 'chr16', 'chr17', 'chr18', 'chr19', 'chr20', 'chr21', 'chr22', 'chrX', 'chrY', 'chr1_KI270706v1_random', 'chr1_KI270707v1_random', 'chr1_KI270708v1_random', 'chr1_KI270711v1_random', 'chr1_KI270712v1_random', 'chr1_KI270713v1_random', 'chr1_KI270714v1_random', 'chr3_GL000221v1_random', 'chr4_GL000008v2_random', 'chr5_GL000208v1_random', 'chr9_KI270718v1_random', 'chr9_KI270719v1_random', 'chr9_KI270720v1_random', 'chr11_KI270721v1_random', 'chr14_GL000009v2_random', 'chr14_GL000225v1_random', 'chr14_KI270722v1_random', 'chr14_GL000194v1_random', 'chr14_KI270723v1_random', 'chr14_KI270724v1_random', 'chr14_KI270725v1_random', 'chr14_KI270726v1_random', 'chr15_KI270727v1_random', 'chr16_KI270728v1_random', 'chr17_GL000205v2_random', 'chr17_KI270729v1_random', 'chr17_KI270730v1_random', 'chr22_KI270731v1_random', 'chr22_KI270732v1_random', 'chr22_KI270733v1_random', 'chr22_KI270734v1_random', 'chr22_KI270735v1_random', 'chr22_KI270736v1_random', 'chr22_KI270737v1_random', 'chr22_KI270738v1_random', 'chrUn_KI270442v1', 'chrUn_KI270466v1', 'chrUn_KI270467v1', 'chrUn_KI270435v1', 'chrUn_KI270438v1', 'chrUn_KI270519v1', 'chrUn_KI270515v1', 'chrUn_KI270538v1', 'chrUn_KI270583v1', 'chrUn_KI270580v1', 'chrUn_KI270589v1', 'chrUn_KI270590v1', 'chrUn_KI270584v1', 'chrUn_KI270582v1', 'chrUn_KI270591v1', 'chrUn_KI270330v1', 'chrUn_KI270448v1', 'chrUn_KI270521v1', 'chrUn_GL000195v1', 'chrUn_GL000219v1', 'chrUn_GL000220v1', 'chrUn_GL000224v1', 'chrUn_KI270741v1', 'chrUn_GL000213v1', 'chrUn_KI270743v1', 'chrUn_KI270744v1', 'chrUn_KI270745v1', 'chrUn_KI270746v1', 'chrUn_KI270748v1', 'chrUn_KI270749v1', 'chrUn_KI270750v1', 'chrUn_KI270751v1', 'chrUn_KI270754v1', 'chrUn_KI270757v1', 'chrUn_GL000214v1', 'chrUn_KI270742v1', 'chrUn_GL000216v2', 'chrUn_GL000218v1', 'chrEBV']
Command completed. Elapsed time: 0:01:12. Running peak memory: 1.497GB.
PID: 98040; Command: /scratch/jps3dp/tools/databio//peppro/tools/bamQC.py; Return code: 0; Memory used: 1.497GB
> `awk '{ for (i=1; i<=NF; ++i) { if ($i ~ "NRF") c=i } getline; print $c }' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_bamQC.tsv`
> `awk '{ for (i=1; i<=NF; ++i) { if ($i ~ "PBC1") c=i } getline; print $c }' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_bamQC.tsv`
> `awk '{ for (i=1; i<=NF; ++i) { if ($i ~ "PBC2") c=i } getline; print $c }' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_bamQC.tsv`
> `NRF` 0.73 PEPPRO _RES_
> `PBC1` 0.89 PEPPRO _RES_
> `PBC2` 14.29 PEPPRO _RES_
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_unmap.bam`
> `samtools view -b -@ 12 -f 4 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_temp.bam > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_unmap.bam` (98138)
Command completed. Elapsed time: 0:00:22. Running peak memory: 1.497GB.
PID: 98138; Command: samtools; Return code: 0; Memory used: 0.017GB
> `samtools view -c -f 4 -@ 12 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_temp.bam`
> `Unmapped_reads` 6142637 PEPPRO _RES_
### Split BAM by strand (06-11 19:10:28) elapsed: 104.0 _TIME_
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_plus.bam`,`/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_minus.bam`
> `samtools view -bh -F 20 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_sort.bam > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_plus.bam` (98510)
Command completed. Elapsed time: 0:03:45. Running peak memory: 1.497GB.
PID: 98510; Command: samtools; Return code: 0; Memory used: 0.006GB
> `samtools view -bh -f 16 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_sort.bam > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_minus.bam` (99399)
Command completed. Elapsed time: 0:03:42. Running peak memory: 1.497GB.
PID: 99399; Command: samtools; Return code: 0; Memory used: 0.006GB
### Calculate TSS enrichment (06-11 19:17:55) elapsed: 447.0 _TIME_
Missing stat 'TSS_non-coding_score'
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/plus_TSS.tsv`,`/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/minus_TSS.tsv`
> `sed -n -e '/[[:space:]]+/w /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/plus_TSS.tsv' -e '/[[:space:]]-/w /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/minus_TSS.tsv' /project/shefflab/genomes/hg38/refgene_anno/default/hg38_TSS.bed` (100038)
Command completed. Elapsed time: 0:00:00. Running peak memory: 1.497GB.
PID: 100038; Command: sed; Return code: 0; Memory used: 0.002GB
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_plus_TssEnrichment.txt`
> `/scratch/jps3dp/tools/databio//peppro/tools/pyTssEnrichment.py -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_sort.bam -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/plus_TSS.tsv -p ends -c 12 -z -v -s 6 -o /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_plus_TssEnrichment.txt` (100040)
Command completed. Elapsed time: 0:00:10. Running peak memory: 1.497GB.
PID: 100040; Command: /scratch/jps3dp/tools/databio//peppro/tools/pyTssEnrichment.py; Return code: 0; Memory used: 0.53GB
> `TSS_coding_score` 17.0 PEPPRO _RES_
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_minus_TssEnrichment.txt`
> `/scratch/jps3dp/tools/databio//peppro/tools/pyTssEnrichment.py -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_sort.bam -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/minus_TSS.tsv -p ends -c 12 -z -v -s 6 -o /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_minus_TssEnrichment.txt` (100086)
Command completed. Elapsed time: 0:00:07. Running peak memory: 1.497GB.
PID: 100086; Command: /scratch/jps3dp/tools/databio//peppro/tools/pyTssEnrichment.py; Return code: 0; Memory used: 0.541GB
> `TSS_non-coding_score` 4.6 PEPPRO _RES_
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_TSSenrichment.pdf`
> `Rscript /scratch/jps3dp/tools/databio//peppro/tools/PEPPRO.R tss -i /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_plus_TssEnrichment.txt /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_minus_TssEnrichment.txt` (100130)
Generating TSS plot with /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_plus_TssEnrichment.txt and /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_minus_TssEnrichment.txt
`geom_smooth()` using formula 'y ~ x'
`geom_smooth()` using formula 'y ~ x'
`geom_smooth()` using formula 'y ~ x'
`geom_smooth()` using formula 'y ~ x'
TSS enrichment plot completed!
Command completed. Elapsed time: 0:00:06. Running peak memory: 1.497GB.
PID: 100130; Command: Rscript; Return code: 0; Memory used: 0.316GB
> `TSS enrichment` QC_hg38/K562_RNA-seq_50_TSSenrichment.pdf TSS enrichment QC_hg38/K562_RNA-seq_50_TSSenrichment.png PEPPRO _OBJ_
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/chr_order.txt`
> `samtools view -H /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_sort.bam | grep 'SN:' | awk -F':' '{print $2,$3}' | awk -F' ' -v OFS=' ' '{print $1,$3}' > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/chr_order.txt` (100167,100168,100169,100170)
Command completed. Elapsed time: 0:00:00. Running peak memory: 1.497GB.
PID: 100167; Command: samtools; Return code: 0; Memory used: 0.0GB
PID: 100169; Command: awk; Return code: 0; Memory used: 0.0GB
PID: 100168; Command: grep; Return code: 0; Memory used: 0.0GB
PID: 100170; Command: awk; Return code: 0; Memory used: 0.0GB
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/chr_keep.txt`
> `cut -f 1 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/chr_keep.txt` (100172)
Command completed. Elapsed time: 0:00:00. Running peak memory: 1.497GB.
PID: 100172; Command: cut; Return code: 0; Memory used: 0.0GB
### Calculate Pause Index (PI) (06-11 19:18:18) elapsed: 23.0 _TIME_
Missing stat 'Pause_index'
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/hg38_ensembl_tss.bed`,`/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/hg38_ensembl_gene_body.bed`
> `grep -wf /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/chr_keep.txt /project/shefflab/genomes/hg38/ensembl_gtf/default/hg38_ensembl_TSS.bed | bedtools sort -i stdin -faidx /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/hg38_ensembl_tss.bed` (100174,100175)
Command completed. Elapsed time: 0:00:02. Running peak memory: 1.497GB.
PID: 100174; Command: grep; Return code: 0; Memory used: 0.004GB
PID: 100175; Command: bedtools; Return code: 0; Memory used: 0.093GB
> `grep -wf /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/chr_keep.txt /project/shefflab/genomes/hg38/ensembl_gtf/default/hg38_ensembl_gene_body.bed | bedtools sort -i stdin -faidx /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/hg38_ensembl_gene_body.bed` (100178,100179)
Command completed. Elapsed time: 0:00:00. Running peak memory: 1.497GB.
PID: 100178; Command: grep; Return code: 0; Memory used: 0.003GB
PID: 100179; Command: bedtools; Return code: 0; Memory used: 0.005GB
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_TSS_density.bed`
> `bedtools coverage -sorted -counts -s -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/hg38_ensembl_tss.bed -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_sort.bam -g /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/chr_order.txt | awk '$7>0' | sort -k4,4 -k7,7nr | sort -k4,4 -u > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_TSS_density.bed` (100182,100183,100184,100185)
Command completed. Elapsed time: 0:01:13. Running peak memory: 1.497GB.
PID: 100183; Command: awk; Return code: 0; Memory used: 0.001GB
PID: 100185; Command: sort; Return code: 0; Memory used: 0.002GB
PID: 100182; Command: bedtools; Return code: 0; Memory used: 0.039GB
PID: 100184; Command: sort; Return code: 0; Memory used: 0.006GB
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_gene_body_density.bed`
> `bedtools coverage -sorted -counts -s -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/hg38_ensembl_gene_body.bed -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_sort.bam -g /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/chr_order.txt | awk '$7>0' | sort -k4 > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_gene_body_density.bed` (100327,100328,100329)
Command completed. Elapsed time: 0:01:44. Running peak memory: 1.497GB.
PID: 100327; Command: bedtools; Return code: 0; Memory used: 0.2GB
PID: 100329; Command: sort; Return code: 0; Memory used: 0.005GB
PID: 100328; Command: awk; Return code: 0; Memory used: 0.001GB
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_pause_index.bed`
> `join --nocheck-order -j4 -o 1.1 1.2 1.3 1.4 1.6 1.7 2.2 2.3 2.7 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_TSS_density.bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_gene_body_density.bed | awk -v OFS=' ' '{ if ($5 == "+"){print $1, $2, $8, $4, sqrt((($6+$9)/sqrt(($8-$2)^2))^2), ($6/sqrt(($3-$2)^2))/($9/sqrt(($8-$7)^2)), $5} else {print $1, $2, $8, $4, sqrt((($6+$9)/sqrt(($3-$7)^2))^2),($6/sqrt(($3-$2)^2))/($9/sqrt(($8-$7)^2)), $5}}' | env LC_COLLATE=C sort -k1,1 -k2,2n > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/tmpw4q3o71f` (100810,100811,100812)
Command completed. Elapsed time: 0:00:00. Running peak memory: 1.497GB.
PID: 100810; Command: join; Return code: 0; Memory used: 0.001GB
PID: 100812; Command: env; Return code: 0; Memory used: 0.004GB
PID: 100811; Command: awk; Return code: 0; Memory used: 0.001GB
> `awk '{print $5}' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/tmpw4q3o71f | sort -n | awk 'BEGIN{i=0} {s[i]=$1; i++;} END{print s[int(NR*0.5-0.5)]}`
/bin/sh: -c: line 0: unexpected EOF while looking for matching `''
/bin/sh: -c: line 1: syntax error: unexpected end of file
### Pipeline failed at: (06-11 19:21:17) elapsed: 179.0 _TIME_
Total time: 0:12:37
Failure reason: Command 'awk '{print $5}' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/tmpw4q3o71f | sort -n | awk 'BEGIN{i=0} {s[i]=$1; i++;} END{print s[int(NR*0.5-0.5)]}' returned non-zero exit status 1.
### Pipeline run code and environment:
* Command: `/scratch/jps3dp/tools/databio//peppro/pipelines/peppro.py --sample-name K562_RNA-seq_50 --genome hg38 --input /project/shefflab/data//guertin/fastq/K562_50pctRNA.fastq.gz --single-or-paired SINGLE -O /project/shefflab/processed//peppro/paper/6.11.2020/results_pipeline -P 12 -M 12000 --protocol PRO --umi-len 0 --scale --prealignments human_rDNA --dirty -R`
* Compute host: udc-ba26-29c1
* Working dir: /sfs/lustre/bahamut/scratch/jps3dp/tools/databio/ppqc
* Outfolder: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/
* Pipeline started at: (06-14 21:11:55) elapsed: 7.0 _TIME_
### Version log:
* Python version: 3.6.6
* Pypiper dir: `/sfs/qumulo/qhome/jps3dp/.local/lib/python3.6/site-packages/pypiper`
* Pypiper version: 0.12.1
* Pipeline dir: `/sfs/lustre/bahamut/scratch/jps3dp/tools/databio/peppro/pipelines`
* Pipeline version: 0.9.8
* Pipeline hash: 887a9a85408962dc3ca070dc954e59a5d4d73a10
* Pipeline branch: * master
* Pipeline date: 2020-06-09 11:36:49 -0400
* Pipeline diff: 40 files changed, 16373 insertions(+), 3522 deletions(-)
### Arguments passed to pipeline:
* `TSS_name`: `None`
* `adapter`: `cutadapt`
* `anno_name`: `None`
* `complexity`: `False`
* `config_file`: `peppro.yaml`
* `cores`: `12`
* `coverage`: `False`
* `dedup`: `seqkit`
* `dirty`: `True`
* `ensembl_gene_body`: `None`
* `ensembl_tss`: `None`
* `exon_name`: `None`
* `force_follow`: `False`
* `genome_assembly`: `hg38`
* `input`: `['/project/shefflab/data//guertin/fastq/K562_50pctRNA.fastq.gz']`
* `input2`: `None`
* `intron_name`: `None`
* `keep`: `False`
* `logdev`: `False`
* `max_len`: `-1`
* `mem`: `12000`
* `new_start`: `False`
* `no_fifo`: `False`
* `output_parent`: `/project/shefflab/processed//peppro/paper/6.11.2020/results_pipeline`
* `paired_end`: `False`
* `pre_name`: `None`
* `prealignments`: `['human_rDNA']`
* `prioritize`: `False`
* `protocol`: `PRO`
* `recover`: `True`
* `sample_name`: `K562_RNA-seq_50`
* `scale`: `True`
* `search_file`: `None`
* `silent`: `False`
* `single_or_paired`: `SINGLE`
* `sob`: `False`
* `testmode`: `False`
* `trimmer`: `seqtk`
* `umi_len`: `0`
* `verbosity`: `None`
----------------------------------------
Removed existing flag: '/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/PEPPRO_failed.flag'
Local input file: /project/shefflab/data//guertin/fastq/K562_50pctRNA.fastq.gz
> `File_mb` 5536.54 PEPPRO _RES_
> `Read_type` SINGLE PEPPRO _RES_
> `Genome` hg38 PEPPRO _RES_
Detected PRO input
### Merge/link and fastq conversion: (06-14 21:11:55) elapsed: 1.0 _TIME_
Number of input file sets: 1
Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/raw/K562_RNA-seq_50.fastq.gz`
Local input file: '/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/raw/K562_RNA-seq_50.fastq.gz'
Found .fastq.gz file
Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/fastq/K562_RNA-seq_50_R1.fastq`
['/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/raw/K562_RNA-seq_50.fastq.gz']
### FASTQ processing: (06-14 21:11:55) elapsed: 0.0 _TIME_
Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/fastq/processed_R1.flag`
### Plot adapter insertion distribution (06-14 21:11:55) elapsed: 0.0 _TIME_
Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/cutadapt/K562_RNA-seq_50_R1_adapter_insertion_distribution.pdf`
> `Adapter insertion distribution` cutadapt/K562_RNA-seq_50_R1_adapter_insertion_distribution.pdf Adapter insertion distribution cutadapt/K562_RNA-seq_50_R1_adapter_insertion_distribution.png PEPPRO _OBJ_
### Prealignments (06-14 21:11:55) elapsed: 0.0 _TIME_
Prealignment assemblies: ['human_rDNA']
### Map to human_rDNA (06-14 21:11:55) elapsed: 0.0 _TIME_
No files match cleanup pattern: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/prealignments/K562_RNA-seq_50_human_rDNA_unmap.fq
### Map to genome (06-14 21:11:55) elapsed: 0.0 _TIME_
Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_sort.bam`
Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_sort.bam`
### Compress all unmapped read files (06-14 21:11:55) elapsed: 0.0 _TIME_
Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/prealignments/K562_RNA-seq_50_human_rDNA_unmap.fq.gz`
### Calculate NRF, PBC1, and PBC2 (06-14 21:11:55) elapsed: 0.0 _TIME_
Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_sort.bam.bai`
Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_bamQC.tsv`
Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_unmap.bam`
### Split BAM by strand (06-14 21:11:55) elapsed: 0.0 _TIME_
Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_plus.bam`
Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_minus.bam`
### Calculate TSS enrichment (06-14 21:11:55) elapsed: 0.0 _TIME_
Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_TSSenrichment.pdf`
> `TSS enrichment` QC_hg38/K562_RNA-seq_50_TSSenrichment.pdf TSS enrichment QC_hg38/K562_RNA-seq_50_TSSenrichment.png PEPPRO _OBJ_
### Calculate Pause Index (PI) (06-14 21:11:55) elapsed: 0.0 _TIME_
Missing stat 'Pause_index'
Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/hg38_ensembl_tss.bed`
Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/hg38_ensembl_gene_body.bed`
Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_TSS_density.bed`
Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_gene_body_density.bed`
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_pause_index.bed`
> `join --nocheck-order -j4 -o 1.1 1.2 1.3 1.4 1.6 1.7 2.2 2.3 2.7 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_TSS_density.bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_gene_body_density.bed | awk -v OFS=' ' '{ if ($5 == "+"){print $1, $2, $8, $4, sqrt((($6+$9)/sqrt(($8-$2)^2))^2), ($6/sqrt(($3-$2)^2))/($9/sqrt(($8-$7)^2)), $5} else {print $1, $2, $8, $4, sqrt((($6+$9)/sqrt(($3-$7)^2))^2),($6/sqrt(($3-$2)^2))/($9/sqrt(($8-$7)^2)), $5}}' | env LC_COLLATE=C sort -k1,1 -k2,2n > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/tmpb66p4_vg` (168920,168921,168922)
Command completed. Elapsed time: 0:00:00. Running peak memory: 0.004GB.
PID: 168920; Command: join; Return code: 0; Memory used: 0.001GB
PID: 168922; Command: env; Return code: 0; Memory used: 0.004GB
PID: 168921; Command: awk; Return code: 0; Memory used: 0.001GB
> `awk '{print $5}' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/tmpb66p4_vg | sort -n | awk 'BEGIN{i=0} {s[i]=$1; i++;} END{print s[int(NR*0.5-0.5)]}'`
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_pause_index.bed`
> `awk -v OFS=' ' '{ if ($5 > 0.0403063) {print $1, $2, $3, $4, $6, $7}}' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/tmpb66p4_vg > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_pause_index.bed` (168930)
Command completed. Elapsed time: 0:00:00. Running peak memory: 0.004GB.
PID: 168930; Command: awk; Return code: 0; Memory used: 0.002GB
> `sort -k5,5n /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_pause_index.bed | awk ' { a[i++]=$5; } END { x=int((i+1)/2); if (x < (i+1)/2) print (a[x-1]+a[x])/2; else print a[x-1]; }'`
> `Pause_index` 8.38 PEPPRO _RES_
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_pause_index.pdf`
> `Rscript /scratch/jps3dp/tools/databio//peppro/tools/PEPPRO.R pi --annotate -i /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_pause_index.bed` (168935)
Pause index plot completed!
Command completed. Elapsed time: 0:00:06. Running peak memory: 0.281GB.
PID: 168935; Command: Rscript; Return code: 0; Memory used: 0.281GB
> `Pause index` QC_hg38/K562_RNA-seq_50_pause_index.pdf Pause index QC_hg38/K562_RNA-seq_50_pause_index.png PEPPRO _OBJ_
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_pause_index.bed.gz`
> `pigz -f -p 12 -f /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_pause_index.bed` (168956)
Command completed. Elapsed time: 0:00:00. Running peak memory: 0.281GB.
PID: 168956; Command: pigz; Return code: 0; Memory used: 0.002GB
### Calculate Fraction of Reads in pre-mature mRNA (06-14 21:12:02) elapsed: 6.0 _TIME_
Missing stat 'Plus_FRiP'
> `samtools view -@ 4 -c -L /project/shefflab/genomes/hg38/refgene_anno/default/hg38_pre-mRNA.bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_plus.bam`
49389074 18570746
> `Plus_FRiP` 0.38 PEPPRO _RES_
Missing stat 'Minus_FRiP'
> `samtools view -@ 4 -c -L /project/shefflab/genomes/hg38/refgene_anno/default/hg38_pre-mRNA.bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_minus.bam`
49389074 18261694
> `Minus_FRiP` 0.37 PEPPRO _RES_
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/signal_hg38/K562_RNA-seq_50_gene_coverage.bed`
> `grep -wf /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/chr_keep.txt /project/shefflab/genomes/hg38/refgene_anno/default/hg38_pre-mRNA.bed | bedtools sort -i stdin -faidx /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/hg38_gene_sort.bed` (169056,169057)
Command completed. Elapsed time: 0:00:01. Running peak memory: 0.281GB.
PID: 169056; Command: grep; Return code: 0; Memory used: 0.004GB
PID: 169057; Command: bedtools; Return code: 0; Memory used: 0.006GB
> `bedtools coverage -sorted -counts -s -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/hg38_gene_sort.bed -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_sort.bam -g /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/signal_hg38/K562_RNA-seq_50_gene_coverage.bed` (169060)
Command completed. Elapsed time: 0:01:53. Running peak memory: 0.281GB.
PID: 169060; Command: bedtools; Return code: 0; Memory used: 0.151GB
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/raw/hg38_annotations.bed`
> `ln -sf /project/shefflab/genomes/hg38/feat_annotation/default/hg38_annotations.bed.gz /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/raw/hg38_annotations.bed.gz` (169357)
Command completed. Elapsed time: 0:00:00. Running peak memory: 0.281GB.
PID: 169357; Command: ln; Return code: 0; Memory used: 0.002GB
> `pigz -f -p 12 -d -c /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/raw/hg38_annotations.bed.gz > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/raw/hg38_annotations.bed` (169358)
Command completed. Elapsed time: 0:00:00. Running peak memory: 0.281GB.
PID: 169358; Command: pigz; Return code: 0; Memory used: 0.002GB
### Calculate cumulative and terminal fraction of reads in features (cFRiF/FRiF) (06-14 21:15:25) elapsed: 203.0 _TIME_
> `cut -f 4 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/raw/hg38_annotations.bed | sort -u`
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/Enhancer`
> `awk -F' ' '{print>"/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/"$4}' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/raw/hg38_annotations.bed` (169366)
Command completed. Elapsed time: 0:00:01. Running peak memory: 0.281GB.
PID: 169366; Command: awk; Return code: 0; Memory used: 0.002GB
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/Enhancer_sort.bed`
> `cut -f 1 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/chr_order.txt | grep -wf - /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/Enhancer | cut -f 1-3 | bedtools sort -i stdin -faidx /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/Enhancer_sort.bed` (169368,169369,169370,169371)
Command completed. Elapsed time: 0:00:01. Running peak memory: 0.281GB.
PID: 169368; Command: cut; Return code: 0; Memory used: 0.0GB
PID: 169369; Command: grep; Return code: 0; Memory used: 0.002GB
PID: 169371; Command: bedtools; Return code: 0; Memory used: 0.01GB
PID: 169370; Command: cut; Return code: 0; Memory used: 0.001GB
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_Enhancer_plus_coverage.bed`
> `bedtools coverage -sorted -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/Enhancer_sort.bed -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_plus.bam -g /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_Enhancer_plus_coverage.bed` (169374)
Command completed. Elapsed time: 0:00:37. Running peak memory: 0.281GB.
PID: 169374; Command: bedtools; Return code: 0; Memory used: 0.023GB
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_Enhancer_minus_coverage.bed`
> `bedtools coverage -sorted -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/Enhancer_sort.bed -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_minus.bam -g /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_Enhancer_minus_coverage.bed` (169406)
Command completed. Elapsed time: 0:00:38. Running peak memory: 0.281GB.
PID: 169406; Command: bedtools; Return code: 0; Memory used: 0.028GB
Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/Promoter`
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/Promoter_sort.bed`
> `cut -f 1 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/chr_order.txt | grep -wf - /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/Promoter | cut -f 1-3 | bedtools sort -i stdin -faidx /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/Promoter_sort.bed` (169442,169443,169444,169445)
Command completed. Elapsed time: 0:00:00. Running peak memory: 0.281GB.
PID: 169442; Command: cut; Return code: 0; Memory used: 0.0GB
PID: 169443; Command: grep; Return code: 0; Memory used: 0.003GB
PID: 169445; Command: bedtools; Return code: 0; Memory used: 0.009GB
PID: 169444; Command: cut; Return code: 0; Memory used: 0.001GB
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_Promoter_plus_coverage.bed`
> `bedtools coverage -sorted -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/Promoter_sort.bed -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_plus.bam -g /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_Promoter_plus_coverage.bed` (169448)
Command completed. Elapsed time: 0:00:41. Running peak memory: 0.281GB.
PID: 169448; Command: bedtools; Return code: 0; Memory used: 0.079GB
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_Promoter_minus_coverage.bed`
> `bedtools coverage -sorted -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/Promoter_sort.bed -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_minus.bam -g /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_Promoter_minus_coverage.bed` (169485)
Command completed. Elapsed time: 0:00:43. Running peak memory: 0.281GB.
PID: 169485; Command: bedtools; Return code: 0; Memory used: 0.062GB
Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/Promoter Flanking Region`
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/Promoter_Flanking_Region`
> `mv "/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/Promoter Flanking Region" "/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/Promoter_Flanking_Region"` (169523)
Command completed. Elapsed time: 0:00:00. Running peak memory: 0.281GB.
PID: 169523; Command: mv; Return code: 0; Memory used: 0.002GB
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/Promoter_Flanking_Region_sort.bed`
> `cut -f 1 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/chr_order.txt | grep -wf - /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/Promoter_Flanking_Region | cut -f 1-3 | bedtools sort -i stdin -faidx /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/Promoter_Flanking_Region_sort.bed` (169524,169525,169526,169528)
Command completed. Elapsed time: 0:00:01. Running peak memory: 0.281GB.
PID: 169524; Command: cut; Return code: 0; Memory used: 0.0GB
PID: 169526; Command: cut; Return code: 0; Memory used: 0.001GB
PID: 169525; Command: grep; Return code: 0; Memory used: 0.002GB
PID: 169528; Command: bedtools; Return code: 0; Memory used: 0.048GB
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_Promoter_Flanking_Region_plus_coverage.bed`
> `bedtools coverage -sorted -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/Promoter_Flanking_Region_sort.bed -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_plus.bam -g /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_Promoter_Flanking_Region_plus_coverage.bed` (169530)
Command completed. Elapsed time: 0:00:39. Running peak memory: 0.281GB.
PID: 169530; Command: bedtools; Return code: 0; Memory used: 0.018GB
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_Promoter_Flanking_Region_minus_coverage.bed`
> `bedtools coverage -sorted -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/Promoter_Flanking_Region_sort.bed -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_minus.bam -g /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_Promoter_Flanking_Region_minus_coverage.bed` (169567)
Command completed. Elapsed time: 0:00:38. Running peak memory: 0.281GB.
PID: 169567; Command: bedtools; Return code: 0; Memory used: 0.039GB
Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/5' UTR`
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/5_UTR`
> `mv "/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/5' UTR" "/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/5_UTR"` (169603)
Command completed. Elapsed time: 0:00:00. Running peak memory: 0.281GB.
PID: 169603; Command: mv; Return code: 0; Memory used: 0.002GB
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/5_UTR_sort.bed`
> `cut -f 1 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/chr_order.txt | grep -wf - /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/5_UTR | cut -f 1-3 | bedtools sort -i stdin -faidx /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/5_UTR_sort.bed` (169604,169605,169606,169607)
Command completed. Elapsed time: 0:00:01. Running peak memory: 0.281GB.
PID: 169604; Command: cut; Return code: 0; Memory used: 0.0GB
PID: 169605; Command: grep; Return code: 0; Memory used: 0.003GB
PID: 169607; Command: bedtools; Return code: 0; Memory used: 0.01GB
PID: 169606; Command: cut; Return code: 0; Memory used: 0.001GB
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_5_UTR_plus_coverage.bed`
> `bedtools coverage -sorted -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/5_UTR_sort.bed -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_plus.bam -g /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_5_UTR_plus_coverage.bed` (169610)
Command completed. Elapsed time: 0:00:38. Running peak memory: 0.281GB.
PID: 169610; Command: bedtools; Return code: 0; Memory used: 0.018GB
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_5_UTR_minus_coverage.bed`
> `bedtools coverage -sorted -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/5_UTR_sort.bed -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_minus.bam -g /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_5_UTR_minus_coverage.bed` (169863)
Command completed. Elapsed time: 0:00:37. Running peak memory: 0.281GB.
PID: 169863; Command: bedtools; Return code: 0; Memory used: 0.022GB
Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/3' UTR`
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/3_UTR`
> `mv "/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/3' UTR" "/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/3_UTR"` (169919)
Command completed. Elapsed time: 0:00:00. Running peak memory: 0.281GB.
PID: 169919; Command: mv; Return code: 0; Memory used: 0.002GB
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/3_UTR_sort.bed`
> `cut -f 1 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/chr_order.txt | grep -wf - /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/3_UTR | cut -f 1-3 | bedtools sort -i stdin -faidx /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/3_UTR_sort.bed` (169920,169921,169922,169923)
Command completed. Elapsed time: 0:00:01. Running peak memory: 0.281GB.
PID: 169920; Command: cut; Return code: 0; Memory used: 0.0GB
PID: 169921; Command: grep; Return code: 0; Memory used: 0.003GB
PID: 169923; Command: bedtools; Return code: 0; Memory used: 0.011GB
PID: 169922; Command: cut; Return code: 0; Memory used: 0.001GB
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_3_UTR_plus_coverage.bed`
> `bedtools coverage -sorted -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/3_UTR_sort.bed -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_plus.bam -g /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_3_UTR_plus_coverage.bed` (169926)
Command completed. Elapsed time: 0:00:40. Running peak memory: 0.281GB.
PID: 169926; Command: bedtools; Return code: 0; Memory used: 0.034GB
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_3_UTR_minus_coverage.bed`
> `bedtools coverage -sorted -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/3_UTR_sort.bed -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_minus.bam -g /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_3_UTR_minus_coverage.bed` (169963)
Command completed. Elapsed time: 0:00:39. Running peak memory: 0.281GB.
PID: 169963; Command: bedtools; Return code: 0; Memory used: 0.044GB
Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/Exon`
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/Exon_sort.bed`
> `cut -f 1 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/chr_order.txt | grep -wf - /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/Exon | cut -f 1-3 | bedtools sort -i stdin -faidx /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/Exon_sort.bed` (169997,169998,169999,170000)
Command completed. Elapsed time: 0:00:03. Running peak memory: 0.281GB.
PID: 169997; Command: cut; Return code: 0; Memory used: 0.0GB
PID: 169998; Command: grep; Return code: 0; Memory used: 0.003GB
PID: 170000; Command: bedtools; Return code: 0; Memory used: 0.158GB
PID: 169999; Command: cut; Return code: 0; Memory used: 0.001GB
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_Exon_plus_coverage.bed`
> `bedtools coverage -sorted -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/Exon_sort.bed -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_plus.bam -g /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_Exon_plus_coverage.bed` (170004)
Command completed. Elapsed time: 0:00:50. Running peak memory: 0.281GB.
PID: 170004; Command: bedtools; Return code: 0; Memory used: 0.131GB
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_Exon_minus_coverage.bed`
> `bedtools coverage -sorted -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/Exon_sort.bed -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_minus.bam -g /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_Exon_minus_coverage.bed` (170050)
Command completed. Elapsed time: 0:00:47. Running peak memory: 0.281GB.
PID: 170050; Command: bedtools; Return code: 0; Memory used: 0.072GB
Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/Intron`
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/Intron_sort.bed`
> `cut -f 1 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/chr_order.txt | grep -wf - /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/Intron | cut -f 1-3 | bedtools sort -i stdin -faidx /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/Intron_sort.bed` (170093,170094,170095,170096)
Command completed. Elapsed time: 0:00:01. Running peak memory: 0.281GB.
PID: 170093; Command: cut; Return code: 0; Memory used: 0.0GB
PID: 170095; Command: cut; Return code: 0; Memory used: 0.001GB
PID: 170094; Command: grep; Return code: 0; Memory used: 0.002GB
PID: 170096; Command: bedtools; Return code: 0; Memory used: 0.077GB
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_Intron_plus_coverage.bed`
> `bedtools coverage -sorted -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/Intron_sort.bed -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_plus.bam -g /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_Intron_plus_coverage.bed` (170099)
Command completed. Elapsed time: 0:00:45. Running peak memory: 0.281GB.
PID: 170099; Command: bedtools; Return code: 0; Memory used: 0.112GB
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_Intron_minus_coverage.bed`
> `bedtools coverage -sorted -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/Intron_sort.bed -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_minus.bam -g /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_Intron_minus_coverage.bed` (170141)
Command completed. Elapsed time: 0:00:47. Running peak memory: 0.281GB.
PID: 170141; Command: bedtools; Return code: 0; Memory used: 0.055GB
### Plot cFRiF/FRiF (06-14 21:25:12) elapsed: 588.0 _TIME_
> `samtools view -@ 12 -q 10 -c -F4 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_plus.bam`
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_cFRiF.pdf`
> `Rscript /scratch/jps3dp/tools/databio//peppro/tools/PEPPRO.R frif -s K562_RNA-seq_50 -z 3099922541 -n 23733539 -y cfrif --reads -o /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_cFRiF.pdf --bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_Enhancer_plus_coverage.bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_Promoter_plus_coverage.bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_Promoter_Flanking_Region_plus_coverage.bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_5_UTR_plus_coverage.bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_3_UTR_plus_coverage.bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_Exon_plus_coverage.bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_Intron_plus_coverage.bed` (170394)
Cumulative cfrif plot completed!
Command completed. Elapsed time: 0:00:41. Running peak memory: 0.446GB.
PID: 170394; Command: Rscript; Return code: 0; Memory used: 0.446GB
> `cFRiF` QC_hg38/K562_RNA-seq_50_cFRiF.pdf cFRiF QC_hg38/K562_RNA-seq_50_cFRiF.png PEPPRO _OBJ_
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_FRiF.pdf`
> `Rscript /scratch/jps3dp/tools/databio//peppro/tools/PEPPRO.R frif -s K562_RNA-seq_50 -z 3099922541 -n 23733539 -y frif --reads -o /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_FRiF.pdf --bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_Enhancer_plus_coverage.bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_Promoter_plus_coverage.bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_Promoter_Flanking_Region_plus_coverage.bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_5_UTR_plus_coverage.bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_3_UTR_plus_coverage.bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_Exon_plus_coverage.bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_Intron_plus_coverage.bed` (170442)
Cumulative frif plot completed!
Command completed. Elapsed time: 0:00:29. Running peak memory: 0.446GB.
PID: 170442; Command: Rscript; Return code: 0; Memory used: 0.446GB
> `FRiF` QC_hg38/K562_RNA-seq_50_FRiF.pdf FRiF QC_hg38/K562_RNA-seq_50_FRiF.png PEPPRO _OBJ_
### Calculate mRNA contamination (06-14 21:26:27) elapsed: 74.0 _TIME_
Missing stat 'mRNA_contamination'
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/hg38_exons_sort.bed`,`/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/hg38_introns_sort.bed`
> `grep -wf /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/chr_keep.txt /project/shefflab/genomes/hg38/refgene_anno/default/hg38_exons.bed | bedtools sort -i stdin -faidx /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/hg38_exons_sort.bed` (170481,170482)
Command completed. Elapsed time: 0:00:05. Running peak memory: 0.446GB.
PID: 170482; Command: bedtools; Return code: 0; Memory used: 0.086GB
PID: 170481; Command: grep; Return code: 0; Memory used: 0.004GB
> `grep -wf /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/chr_keep.txt /project/shefflab/genomes/hg38/refgene_anno/default/hg38_introns.bed | bedtools sort -i stdin -faidx /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/chr_order.txt | bedtools sort -i stdin -faidx /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/hg38_introns_sort.bed` (170488,170489,170490)
Command completed. Elapsed time: 0:00:05. Running peak memory: 0.446GB.
PID: 170488; Command: grep; Return code: 0; Memory used: 0.005GB
PID: 170490; Command: bedtools; Return code: 0; Memory used: 0.007GB
PID: 170489; Command: bedtools; Return code: 0; Memory used: 0.033GB
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_exons_coverage.bed`,`/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_introns_coverage.bed`
> `bedtools coverage -sorted -counts -s -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/hg38_exons_sort.bed -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_sort.bam -g /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_exons_coverage.bed` (170497)
Command completed. Elapsed time: 0:01:35. Running peak memory: 0.446GB.
PID: 170497; Command: bedtools; Return code: 0; Memory used: 0.116GB
> `bedtools coverage -sorted -counts -s -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/hg38_introns_sort.bed -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_sort.bam -g /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_introns_coverage.bed` (170580)
Command completed. Elapsed time: 0:01:39. Running peak memory: 0.446GB.
PID: 170580; Command: bedtools; Return code: 0; Memory used: 0.112GB
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_exons_rpkm.bed`
> `awk -v OFS=' ' '{chrom[$4] = $1; if($4!=prev4) {chromStart[$4] = $2} strand[$4] = $6; readCount[$4] += $7; exonCount[$4] += 1; geneSizeKB[$4] += (sqrt(($3-$2+0.00000001)^2)/1000); gene[$4] = $4; chromEnd[$4]=$3; prev4=$4} END { for (a in readCount) { print chrom[a], chromStart[a], chromEnd[a], gene[a], (readCount[a]/49.389074)/geneSizeKB[a], strand[a]}}' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_exons_coverage.bed | awk '$5>0' | sort -k4 > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_exons_rpkm.bed` (170669,170670,170671)
Command completed. Elapsed time: 0:00:01. Running peak memory: 0.446GB.
PID: 170669; Command: awk; Return code: 0; Memory used: 0.007GB
PID: 170671; Command: sort; Return code: 0; Memory used: 0.003GB
PID: 170670; Command: awk; Return code: 0; Memory used: 0.001GB
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_introns_rpkm.bed`
> `awk -v OFS=' ' '{chrom[$4] = $1; if($4!=prev4) {chromStart[$4] = $2} strand[$4] = $6; readCount[$4] += $7; exonCount[$4] += 1; geneSizeKB[$4] += (sqrt(($3-$2+0.00000001)^2)/1000); gene[$4] = $4; chromEnd[$4]=$3; prev4=$4} END { for (a in readCount) { print chrom[a], chromStart[a], chromEnd[a], gene[a], (readCount[a]/49.389074)/geneSizeKB[a], strand[a]}}' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_introns_coverage.bed | awk '$5>0' | sort -k4 > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_introns_rpkm.bed` (170674,170675,170676)
Command completed. Elapsed time: 0:00:01. Running peak memory: 0.446GB.
PID: 170674; Command: awk; Return code: 0; Memory used: 0.008GB
PID: 170676; Command: sort; Return code: 0; Memory used: 0.004GB
PID: 170675; Command: awk; Return code: 0; Memory used: 0.001GB
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_exon_intron_ratios.bed`
> `join --nocheck-order -a1 -a2 -j4 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_introns_rpkm.bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_exons_rpkm.bed | awk -v OFS=' ' 'NF==11 {print $7, $8, $9, $1, ($10/$5), $11}' | sort -k1,1 -k2,2n > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_exon_intron_ratios.bed` (170679,170680,170681)
Command completed. Elapsed time: 0:00:00. Running peak memory: 0.446GB.
PID: 170679; Command: join; Return code: 0; Memory used: 0.001GB
PID: 170681; Command: sort; Return code: 0; Memory used: 0.004GB
PID: 170680; Command: awk; Return code: 0; Memory used: 0.001GB
> `awk '{print $5}' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_exon_intron_ratios.bed | sort -n | awk ' { a[i++]=$1; } END { x=int((i+1)/2); if (x < (i+1)/2) print (a[x-1]+a[x])/2; else print a[x-1]; }'`
> `mRNA_contamination` 4.15 PEPPRO _RES_
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_mRNA_contamination.pdf`
> `Rscript /scratch/jps3dp/tools/databio//peppro/tools/PEPPRO.R mrna -i /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_exon_intron_ratios.bed --annotate` (170687)
mRNA contamination plot completed!
Command completed. Elapsed time: 0:00:05. Running peak memory: 0.446GB.
PID: 170687; Command: Rscript; Return code: 0; Memory used: 0.316GB
> `mRNA contamination` QC_hg38/K562_RNA-seq_50_mRNA_contamination.pdf mRNA contamination QC_hg38/K562_RNA-seq_50_mRNA_contamination.png PEPPRO _OBJ_
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_exon_intron_ratios.bed.gz`
> `pigz -f -p 12 -f /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/QC_hg38/K562_RNA-seq_50_exon_intron_ratios.bed` (170708)
Command completed. Elapsed time: 0:00:00. Running peak memory: 0.446GB.
PID: 170708; Command: pigz; Return code: 0; Memory used: 0.003GB
### Produce bigWig files (06-14 21:29:59) elapsed: 212.0 _TIME_
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/signal_hg38/K562_RNA-seq_50_plus_exact_body_0-mer.bw`,`/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/signal_hg38/K562_RNA-seq_50_plus_smooth_body_0-mer.bw`
> `samtools index /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_plus.bam` (170717)
Command completed. Elapsed time: 0:00:26. Running peak memory: 0.446GB.
PID: 170717; Command: samtools; Return code: 0; Memory used: 0.009GB
> `/scratch/jps3dp/tools/databio//peppro/tools/bamSitesToWig.py -i /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_plus.bam -c /project/shefflab/genomes/hg38/fasta/default/hg38.chrom.sizes -o /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/signal_hg38/K562_RNA-seq_50_plus_exact_body_0-mer.bw -w /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/signal_hg38/K562_RNA-seq_50_plus_smooth_body_0-mer.bw -p 8 --variable-step --tail-edge --scale 49389074.0` (171000)
Cutting parallel chroms in half to accommodate two tracks.
Registering input file: '/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_plus.bam'
Temporary files will be stored in: 'tmp_K562_RNA-seq_50_plus_cuttrace_1gg30hxf'
Processing with 4 cores...
stdin is empty of data
stdin is empty of data
Discarding 108 chunk(s) of reads: ['chrM', 'chr1_KI270709v1_random', 'chr1_KI270710v1_random', 'chr2_KI270715v1_random', 'chr2_KI270716v1_random', 'chr9_KI270717v1_random', 'chr14_KI270724v1_random', 'chr14_KI270725v1_random', 'chr17_KI270730v1_random', 'chr22_KI270732v1_random', 'chr22_KI270738v1_random', 'chr22_KI270739v1_random', 'chrY_KI270740v1_random', 'chrUn_KI270302v1', 'chrUn_KI270304v1', 'chrUn_KI270303v1', 'chrUn_KI270305v1', 'chrUn_KI270322v1', 'chrUn_KI270320v1', 'chrUn_KI270310v1', 'chrUn_KI270316v1', 'chrUn_KI270315v1', 'chrUn_KI270312v1', 'chrUn_KI270311v1', 'chrUn_KI270317v1', 'chrUn_KI270412v1', 'chrUn_KI270411v1', 'chrUn_KI270414v1', 'chrUn_KI270419v1', 'chrUn_KI270418v1', 'chrUn_KI270420v1', 'chrUn_KI270424v1', 'chrUn_KI270417v1', 'chrUn_KI270422v1', 'chrUn_KI270423v1', 'chrUn_KI270425v1', 'chrUn_KI270429v1', 'chrUn_KI270465v1', 'chrUn_KI270468v1', 'chrUn_KI270510v1', 'chrUn_KI270509v1', 'chrUn_KI270518v1', 'chrUn_KI270508v1', 'chrUn_KI270516v1', 'chrUn_KI270512v1', 'chrUn_KI270519v1', 'chrUn_KI270522v1', 'chrUn_KI270511v1', 'chrUn_KI270515v1', 'chrUn_KI270507v1', 'chrUn_KI270517v1', 'chrUn_KI270529v1', 'chrUn_KI270528v1', 'chrUn_KI270530v1', 'chrUn_KI270539v1', 'chrUn_KI270538v1', 'chrUn_KI270544v1', 'chrUn_KI270548v1', 'chrUn_KI270583v1', 'chrUn_KI270587v1', 'chrUn_KI270581v1', 'chrUn_KI270579v1', 'chrUn_KI270588v1', 'chrUn_KI270593v1', 'chrUn_KI270329v1', 'chrUn_KI270334v1', 'chrUn_KI270333v1', 'chrUn_KI270335v1', 'chrUn_KI270338v1', 'chrUn_KI270340v1', 'chrUn_KI270336v1', 'chrUn_KI270337v1', 'chrUn_KI270363v1', 'chrUn_KI270364v1', 'chrUn_KI270362v1', 'chrUn_KI270366v1', 'chrUn_KI270378v1', 'chrUn_KI270379v1', 'chrUn_KI270389v1', 'chrUn_KI270390v1', 'chrUn_KI270387v1', 'chrUn_KI270395v1', 'chrUn_KI270396v1', 'chrUn_KI270388v1', 'chrUn_KI270394v1', 'chrUn_KI270386v1', 'chrUn_KI270391v1', 'chrUn_KI270383v1', 'chrUn_KI270393v1', 'chrUn_KI270384v1', 'chrUn_KI270392v1', 'chrUn_KI270381v1', 'chrUn_KI270385v1', 'chrUn_KI270382v1', 'chrUn_KI270376v1', 'chrUn_KI270374v1', 'chrUn_KI270372v1', 'chrUn_KI270373v1', 'chrUn_KI270375v1', 'chrUn_KI270371v1', 'chrUn_KI270448v1', 'chrUn_GL000226v1', 'chrUn_KI270746v1', 'chrUn_KI270747v1', 'chrUn_KI270752v1', 'chrUn_KI270753v1', 'chrUn_KI270755v1', 'chrUn_KI270756v1']
Keeping 87 chunk(s) of reads: ['chr1', 'chr2', 'chr3', 'chr4', 'chr5', 'chr6', 'chr7', 'chr8', 'chr9', 'chr10', 'chr11', 'chr12', 'chr13', 'chr14', 'chr15', 'chr16', 'chr17', 'chr18', 'chr19', 'chr20', 'chr21', 'chr22', 'chrX', 'chrY', 'chr1_KI270706v1_random', 'chr1_KI270707v1_random', 'chr1_KI270708v1_random', 'chr1_KI270711v1_random', 'chr1_KI270712v1_random', 'chr1_KI270713v1_random', 'chr1_KI270714v1_random', 'chr3_GL000221v1_random', 'chr4_GL000008v2_random', 'chr5_GL000208v1_random', 'chr9_KI270718v1_random', 'chr9_KI270719v1_random', 'chr9_KI270720v1_random', 'chr11_KI270721v1_random', 'chr14_GL000009v2_random', 'chr14_GL000225v1_random', 'chr14_KI270722v1_random', 'chr14_GL000194v1_random', 'chr14_KI270723v1_random', 'chr14_KI270726v1_random', 'chr15_KI270727v1_random', 'chr16_KI270728v1_random', 'chr17_GL000205v2_random', 'chr17_KI270729v1_random', 'chr22_KI270731v1_random', 'chr22_KI270733v1_random', 'chr22_KI270734v1_random', 'chr22_KI270735v1_random', 'chr22_KI270736v1_random', 'chr22_KI270737v1_random', 'chrUn_KI270442v1', 'chrUn_KI270466v1', 'chrUn_KI270467v1', 'chrUn_KI270435v1', 'chrUn_KI270438v1', 'chrUn_KI270580v1', 'chrUn_KI270589v1', 'chrUn_KI270590v1', 'chrUn_KI270584v1', 'chrUn_KI270582v1', 'chrUn_KI270591v1', 'chrUn_KI270330v1', 'chrUn_KI270521v1', 'chrUn_GL000195v1', 'chrUn_GL000219v1', 'chrUn_GL000220v1', 'chrUn_GL000224v1', 'chrUn_KI270741v1', 'chrUn_GL000213v1', 'chrUn_KI270743v1', 'chrUn_KI270744v1', 'chrUn_KI270745v1', 'chrUn_KI270748v1', 'chrUn_KI270749v1', 'chrUn_KI270750v1', 'chrUn_KI270751v1', 'chrUn_KI270754v1', 'chrUn_KI270757v1', 'chrUn_GL000214v1', 'chrUn_KI270742v1', 'chrUn_GL000216v2', 'chrUn_GL000218v1', 'chrEBV']
Reduce step (merge files)...
Merging 87 files into output file: '/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/signal_hg38/K562_RNA-seq_50_plus_exact_body_0-mer.bw'
Merging 87 files into output file: '/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/signal_hg38/K562_RNA-seq_50_plus_smooth_body_0-mer.bw'
Command completed. Elapsed time: 0:08:31. Running peak memory: 3.651GB.
PID: 171000; Command: /scratch/jps3dp/tools/databio//peppro/tools/bamSitesToWig.py; Return code: 0; Memory used: 3.651GB
Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/signal_hg38/K562_RNA-seq_50_minus_exact_body_0-mer.bw`,`/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/signal_hg38/K562_RNA-seq_50_minus_smooth_body_0-mer.bw`
> `samtools index /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_minus.bam` (172486)
Command completed. Elapsed time: 0:00:24. Running peak memory: 3.651GB.
PID: 172486; Command: samtools; Return code: 0; Memory used: 0.01GB
> `/scratch/jps3dp/tools/databio//peppro/tools/bamSitesToWig.py -i /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_minus.bam -c /project/shefflab/genomes/hg38/fasta/default/hg38.chrom.sizes -o /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/signal_hg38/K562_RNA-seq_50_minus_exact_body_0-mer.bw -w /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/signal_hg38/K562_RNA-seq_50_minus_smooth_body_0-mer.bw -p 8 --variable-step --tail-edge --scale 49389074.0` (172509)
Cutting parallel chroms in half to accommodate two tracks.
Registering input file: '/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/aligned_hg38/K562_RNA-seq_50_minus.bam'
Temporary files will be stored in: 'tmp_K562_RNA-seq_50_minus_cuttrace_4d299_gk'
Processing with 4 cores...
stdin is empty of data
Discarding 110 chunk(s) of reads: ['chrM', 'chr1_KI270709v1_random', 'chr1_KI270710v1_random', 'chr2_KI270715v1_random', 'chr2_KI270716v1_random', 'chr9_KI270717v1_random', 'chr9_KI270718v1_random', 'chr14_KI270723v1_random', 'chr22_KI270737v1_random', 'chr22_KI270739v1_random', 'chrY_KI270740v1_random', 'chrUn_KI270302v1', 'chrUn_KI270304v1', 'chrUn_KI270303v1', 'chrUn_KI270305v1', 'chrUn_KI270322v1', 'chrUn_KI270320v1', 'chrUn_KI270310v1', 'chrUn_KI270316v1', 'chrUn_KI270315v1', 'chrUn_KI270312v1', 'chrUn_KI270311v1', 'chrUn_KI270317v1', 'chrUn_KI270412v1', 'chrUn_KI270411v1', 'chrUn_KI270414v1', 'chrUn_KI270419v1', 'chrUn_KI270418v1', 'chrUn_KI270420v1', 'chrUn_KI270424v1', 'chrUn_KI270417v1', 'chrUn_KI270422v1', 'chrUn_KI270423v1', 'chrUn_KI270425v1', 'chrUn_KI270429v1', 'chrUn_KI270465v1', 'chrUn_KI270435v1', 'chrUn_KI270468v1', 'chrUn_KI270510v1', 'chrUn_KI270509v1', 'chrUn_KI270518v1', 'chrUn_KI270508v1', 'chrUn_KI270516v1', 'chrUn_KI270512v1', 'chrUn_KI270522v1', 'chrUn_KI270511v1', 'chrUn_KI270507v1', 'chrUn_KI270517v1', 'chrUn_KI270529v1', 'chrUn_KI270528v1', 'chrUn_KI270530v1', 'chrUn_KI270539v1', 'chrUn_KI270544v1', 'chrUn_KI270548v1', 'chrUn_KI270587v1', 'chrUn_KI270580v1', 'chrUn_KI270581v1', 'chrUn_KI270579v1', 'chrUn_KI270590v1', 'chrUn_KI270584v1', 'chrUn_KI270582v1', 'chrUn_KI270588v1', 'chrUn_KI270593v1', 'chrUn_KI270329v1', 'chrUn_KI270334v1', 'chrUn_KI270333v1', 'chrUn_KI270335v1', 'chrUn_KI270338v1', 'chrUn_KI270340v1', 'chrUn_KI270336v1', 'chrUn_KI270337v1', 'chrUn_KI270363v1', 'chrUn_KI270364v1', 'chrUn_KI270362v1', 'chrUn_KI270366v1', 'chrUn_KI270378v1', 'chrUn_KI270379v1', 'chrUn_KI270389v1', 'chrUn_KI270390v1', 'chrUn_KI270387v1', 'chrUn_KI270395v1', 'chrUn_KI270396v1', 'chrUn_KI270388v1', 'chrUn_KI270394v1', 'chrUn_KI270386v1', 'chrUn_KI270391v1', 'chrUn_KI270383v1', 'chrUn_KI270393v1', 'chrUn_KI270384v1', 'chrUn_KI270392v1', 'chrUn_KI270381v1', 'chrUn_KI270385v1', 'chrUn_KI270382v1', 'chrUn_KI270376v1', 'chrUn_KI270374v1', 'chrUn_KI270372v1', 'chrUn_KI270373v1', 'chrUn_KI270375v1', 'chrUn_KI270371v1', 'chrUn_KI270521v1', 'chrUn_KI270741v1', 'chrUn_GL000226v1', 'chrUn_GL000213v1', 'chrUn_KI270747v1', 'chrUn_KI270748v1', 'chrUn_KI270752v1', 'chrUn_KI270753v1', 'chrUn_KI270755v1', 'chrUn_KI270756v1', 'chrUn_KI270757v1']
Keeping 85 chunk(s) of reads: ['chr1', 'chr2', 'chr3', 'chr4', 'chr5', 'chr6', 'chr7', 'chr8', 'chr9', 'chr10', 'chr11', 'chr12', 'chr13', 'chr14', 'chr15', 'chr16', 'chr17', 'chr18', 'chr19', 'chr20', 'chr21', 'chr22', 'chrX', 'chrY', 'chr1_KI270706v1_random', 'chr1_KI270707v1_random', 'chr1_KI270708v1_random', 'chr1_KI270711v1_random', 'chr1_KI270712v1_random', 'chr1_KI270713v1_random', 'chr1_KI270714v1_random', 'chr3_GL000221v1_random', 'chr4_GL000008v2_random', 'chr5_GL000208v1_random', 'chr9_KI270719v1_random', 'chr9_KI270720v1_random', 'chr11_KI270721v1_random', 'chr14_GL000009v2_random', 'chr14_GL000225v1_random', 'chr14_KI270722v1_random', 'chr14_GL000194v1_random', 'chr14_KI270724v1_random', 'chr14_KI270725v1_random', 'chr14_KI270726v1_random', 'chr15_KI270727v1_random', 'chr16_KI270728v1_random', 'chr17_GL000205v2_random', 'chr17_KI270729v1_random', 'chr17_KI270730v1_random', 'chr22_KI270731v1_random', 'chr22_KI270732v1_random', 'chr22_KI270733v1_random', 'chr22_KI270734v1_random', 'chr22_KI270735v1_random', 'chr22_KI270736v1_random', 'chr22_KI270738v1_random', 'chrUn_KI270442v1', 'chrUn_KI270466v1', 'chrUn_KI270467v1', 'chrUn_KI270438v1', 'chrUn_KI270519v1', 'chrUn_KI270515v1', 'chrUn_KI270538v1', 'chrUn_KI270583v1', 'chrUn_KI270589v1', 'chrUn_KI270591v1', 'chrUn_KI270330v1', 'chrUn_KI270448v1', 'chrUn_GL000195v1', 'chrUn_GL000219v1', 'chrUn_GL000220v1', 'chrUn_GL000224v1', 'chrUn_KI270743v1', 'chrUn_KI270744v1', 'chrUn_KI270745v1', 'chrUn_KI270746v1', 'chrUn_KI270749v1', 'chrUn_KI270750v1', 'chrUn_KI270751v1', 'chrUn_KI270754v1', 'chrUn_GL000214v1', 'chrUn_KI270742v1', 'chrUn_GL000216v2', 'chrUn_GL000218v1', 'chrEBV']
Reduce step (merge files)...
Merging 85 files into output file: '/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/signal_hg38/K562_RNA-seq_50_minus_exact_body_0-mer.bw'
Merging 85 files into output file: '/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/K562_RNA-seq_50/signal_hg38/K562_RNA-seq_50_minus_smooth_body_0-mer.bw'
Command completed. Elapsed time: 0:08:21. Running peak memory: 3.651GB.
PID: 172509; Command: /scratch/jps3dp/tools/databio//peppro/tools/bamSitesToWig.py; Return code: 0; Memory used: 3.282GB
### Pipeline completed. Epilogue
* Elapsed time (this run): 0:35:54
* Total elapsed time (all runs): 2:43:14
* Peak memory (this run): 3.6507 GB
* Pipeline completed time: 2020-06-14 21:47:41