### Pipeline run code and environment: * Command: `/scratch/jps3dp/tools/databio//peppro/pipelines/peppro.py --sample-name H9_PRO-seq_20 --genome hg38 --input /project/shefflab/data//guertin/fastq/H9_PRO-seq_20pct_PE1.fastq.gz --single-or-paired PAIRED -O /project/shefflab/processed//peppro/paper/6.11.2020/results_pipeline -P 4 -M 8000 --input2 /project/shefflab/data//guertin/fastq/H9_PRO-seq_20pct_PE2.fastq.gz --protocol PRO --umi-len 8 --scale --prealignments human_rDNA --dirty` * Compute host: udc-aw29-25a * Working dir: /sfs/lustre/bahamut/scratch/jps3dp/tools/databio/ppqc * Outfolder: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/ * Pipeline started at: (06-11 17:54:22) elapsed: 5.0 _TIME_ ### Version log: * Python version: 3.6.6 * Pypiper dir: `/sfs/qumulo/qhome/jps3dp/.local/lib/python3.6/site-packages/pypiper` * Pypiper version: 0.12.1 * Pipeline dir: `/sfs/lustre/bahamut/scratch/jps3dp/tools/databio/peppro/pipelines` * Pipeline version: 0.9.8 * Pipeline hash: 887a9a85408962dc3ca070dc954e59a5d4d73a10 * Pipeline branch: * master * Pipeline date: 2020-06-09 11:36:49 -0400 * Pipeline diff: 40 files changed, 16373 insertions(+), 3522 deletions(-) ### Arguments passed to pipeline: * `TSS_name`: `None` * `adapter`: `cutadapt` * `anno_name`: `None` * `complexity`: `False` * `config_file`: `peppro.yaml` * `cores`: `4` * `coverage`: `False` * `dedup`: `seqkit` * `dirty`: `True` * `ensembl_gene_body`: `None` * `ensembl_tss`: `None` * `exon_name`: `None` * `force_follow`: `False` * `genome_assembly`: `hg38` * `input`: `['/project/shefflab/data//guertin/fastq/H9_PRO-seq_20pct_PE1.fastq.gz']` * `input2`: `['/project/shefflab/data//guertin/fastq/H9_PRO-seq_20pct_PE2.fastq.gz']` * `intron_name`: `None` * `keep`: `False` * `logdev`: `False` * `max_len`: `-1` * `mem`: `8000` * `new_start`: `False` * `no_fifo`: `False` * `output_parent`: `/project/shefflab/processed//peppro/paper/6.11.2020/results_pipeline` * `paired_end`: `True` * `pre_name`: `None` * `prealignments`: `['human_rDNA']` * `prioritize`: `False` * `protocol`: `PRO` * `recover`: `False` * `sample_name`: `H9_PRO-seq_20` * `scale`: `True` * `search_file`: `None` * `silent`: `False` * `single_or_paired`: `PAIRED` * `sob`: `False` * `testmode`: `False` * `trimmer`: `seqtk` * `umi_len`: `8` * `verbosity`: `None` ---------------------------------------- Local input file: /project/shefflab/data//guertin/fastq/H9_PRO-seq_20pct_PE1.fastq.gz Local input file: /project/shefflab/data//guertin/fastq/H9_PRO-seq_20pct_PE2.fastq.gz > `File_mb` 492.8 PEPPRO _RES_ > `Read_type` PAIRED PEPPRO _RES_ > `Genome` hg38 PEPPRO _RES_ Detected PRO input ### Merge/link and fastq conversion: (06-11 17:54:24) elapsed: 1.0 _TIME_ Number of input file sets: 2 Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/raw/H9_PRO-seq_20_R1.fastq.gz` > `ln -sf /project/shefflab/data//guertin/fastq/H9_PRO-seq_20pct_PE1.fastq.gz /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/raw/H9_PRO-seq_20_R1.fastq.gz` (51553)
Command completed. Elapsed time: 0:00:00. Running peak memory: 0GB. PID: 51553; Command: ln; Return code: 0; Memory used: 0.0GB Local input file: '/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/raw/H9_PRO-seq_20_R1.fastq.gz' Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/raw/H9_PRO-seq_20_R2.fastq.gz` > `ln -sf /project/shefflab/data//guertin/fastq/H9_PRO-seq_20pct_PE2.fastq.gz /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/raw/H9_PRO-seq_20_R2.fastq.gz` (51555)
Command completed. Elapsed time: 0:00:00. Running peak memory: 0GB. PID: 51555; Command: ln; Return code: 0; Memory used: 0.0GB Local input file: '/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/raw/H9_PRO-seq_20_R2.fastq.gz' Found .fastq.gz file Found .fastq.gz file Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R1.fastq`,`/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R2.fastq` > `pigz -f -p 4 -d -c /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/raw/H9_PRO-seq_20_R1.fastq.gz > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R1.fastq` (51556)
Command completed. Elapsed time: 0:00:22. Running peak memory: 0.002GB. PID: 51556; Command: pigz; Return code: 0; Memory used: 0.002GB > `pigz -f -p 4 -d -c /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/raw/H9_PRO-seq_20_R2.fastq.gz > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R2.fastq` (52032)
Command completed. Elapsed time: 0:00:20. Running peak memory: 0.002GB. PID: 52032; Command: pigz; Return code: 0; Memory used: 0.002GB > `Raw_reads` 23139336 PEPPRO _RES_ > `Fastq_reads` 23139336 PEPPRO _RES_ ['/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/raw/H9_PRO-seq_20_R1.fastq.gz', '/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/raw/H9_PRO-seq_20_R2.fastq.gz'] ### FASTQ processing: (06-11 17:55:14) elapsed: 51.0 _TIME_ > `cutadapt --version` Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R1_processed.fastq` > `(cutadapt -j 4 -m 10 -O 1 -a TGGAATTCTCGGGTGCCAAGG /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R1.fastq -o /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R1_noadap.fastq --too-short-output /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R1_short.fastq ) > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/cutadapt/H9_PRO-seq_20_R1_cutadapt.txt` (52485)
Command completed. Elapsed time: 0:00:31. Running peak memory: 0.923GB. PID: 52485; Command: cutadapt; Return code: 0; Memory used: 0.923GB > `seqtk trimfq -b 8 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R1_noadap.fastq | seqtk seq -L 10 -r - > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R1_processed.fastq` (52688,52690)
Command completed. Elapsed time: 0:00:13. Running peak memory: 0.923GB. PID: 52688; Command: seqtk; Return code: 0; Memory used: 0.001GB PID: 52690; Command: seqtk; Return code: 0; Memory used: 0.002GB Evaluating read trimming > `Trimmed_reads` 5665407 PEPPRO _RES_ > `Trim_loss_rate` 75.52 PEPPRO _RES_ Targetless command, running... > `fastqc --noextract --outdir /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastqc /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R1_processed.fastq` (52772)
Started analysis of H9_PRO-seq_20_R1_processed.fastq Approx 5% complete for H9_PRO-seq_20_R1_processed.fastq Approx 10% complete for H9_PRO-seq_20_R1_processed.fastq Approx 15% complete for H9_PRO-seq_20_R1_processed.fastq Approx 20% complete for H9_PRO-seq_20_R1_processed.fastq Approx 25% complete for H9_PRO-seq_20_R1_processed.fastq Approx 30% complete for H9_PRO-seq_20_R1_processed.fastq Approx 35% complete for H9_PRO-seq_20_R1_processed.fastq Approx 40% complete for H9_PRO-seq_20_R1_processed.fastq Approx 45% complete for H9_PRO-seq_20_R1_processed.fastq Approx 50% complete for H9_PRO-seq_20_R1_processed.fastq Approx 55% complete for H9_PRO-seq_20_R1_processed.fastq Approx 60% complete for H9_PRO-seq_20_R1_processed.fastq Approx 65% complete for H9_PRO-seq_20_R1_processed.fastq Approx 70% complete for H9_PRO-seq_20_R1_processed.fastq Approx 75% complete for H9_PRO-seq_20_R1_processed.fastq Approx 80% complete for H9_PRO-seq_20_R1_processed.fastq Approx 85% complete for H9_PRO-seq_20_R1_processed.fastq Approx 90% complete for H9_PRO-seq_20_R1_processed.fastq Approx 95% complete for H9_PRO-seq_20_R1_processed.fastq Analysis complete for H9_PRO-seq_20_R1_processed.fastqCommand completed. Elapsed time: 0:00:15. Running peak memory: 0.923GB. PID: 52772; Command: fastqc; Return code: 0; Memory used: 0.179GB > `FastQC report r1` fastqc/H9_PRO-seq_20_R1_processed_fastqc.html FastQC report r1 None PEPPRO _OBJ_ Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R1_trimmed.fastq` > `seqkit rmdup --threads 4 --by-seq --ignore-case -o /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R1_dedup.fastq /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R1_noadap.fastq` (52955)
[INFO][0m 233822 duplicated records removedCommand completed. Elapsed time: 0:00:17. Running peak memory: 0.923GB. PID: 52955; Command: seqkit; Return code: 0; Memory used: 0.499GB > `seqtk trimfq -b 8 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R1_dedup.fastq | seqtk seq -L 10 -r - > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R1_trimmed.fastq` (53108,53109)
Command completed. Elapsed time: 0:00:07. Running peak memory: 0.923GB. PID: 53108; Command: seqtk; Return code: 0; Memory used: 0.001GB PID: 53109; Command: seqtk; Return code: 0; Memory used: 0.002GB > `grep 'Reads with adapters:' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/cutadapt/H9_PRO-seq_20_R1_cutadapt.txt | awk '{print $(NF-1)}'` > `Reads_with_adapter` 9167377.0 PEPPRO _RES_ > `grep 'Total basepairs processed:' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/cutadapt/H9_PRO-seq_20_R1_cutadapt.txt | awk '{print $(NF-1)}'` > `awk '{sum+=$1*$2} END {printf "%.0f", sum}' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/cutadapt/H9_PRO-seq_20_R1_cutadapt.txt` > `wc -l /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R1_short.fastq | awk '{print $1}'` > `Uninformative_adapter_reads` 5720857.0 PEPPRO _RES_ > `Duplicate_reads` 233822.0 PEPPRO _RES_ > `Pct_uninformative_adapter_reads` 49.447 PEPPRO _RES_ Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/processed_R1.flag` > `touch /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/processed_R1.flag` (53193)
Command completed. Elapsed time: 0:00:00. Running peak memory: 0.923GB. PID: 53193; Command: touch; Return code: 0; Memory used: 0.0GB > `cutadapt --version` Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R2_trimmed.fastq` > `(cutadapt -j 4 -m 10 -O 1 -a GATCGTCGGACTGTAGAACTCTGAAC /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R2.fastq -o /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R2_noadap.fastq --too-short-output /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R2_short.fastq ) > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/cutadapt/H9_PRO-seq_20_R2_cutadapt.txt` (53198)
Command completed. Elapsed time: 0:00:36. Running peak memory: 0.923GB. PID: 53198; Command: cutadapt; Return code: 0; Memory used: 0.889GB > `seqtk trimfq -b 8 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R2_noadap.fastq | seqtk seq -L 10 -r - > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R2_trimmed.fastq` (53413,53414)
Command completed. Elapsed time: 0:00:13. Running peak memory: 0.923GB. PID: 53413; Command: seqtk; Return code: 0; Memory used: 0.001GB PID: 53414; Command: seqtk; Return code: 0; Memory used: 0.002GB Evaluating read trimming > `Trimmed_reads` 11330814 PEPPRO _RES_ > `Trim_loss_rate` 51.03 PEPPRO _RES_ Targetless command, running... > `fastqc --noextract --outdir /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastqc /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R1_processed.fastq` (53502)
Started analysis of H9_PRO-seq_20_R1_processed.fastq Approx 5% complete for H9_PRO-seq_20_R1_processed.fastq Approx 10% complete for H9_PRO-seq_20_R1_processed.fastq Approx 15% complete for H9_PRO-seq_20_R1_processed.fastq Approx 20% complete for H9_PRO-seq_20_R1_processed.fastq Approx 25% complete for H9_PRO-seq_20_R1_processed.fastq Approx 30% complete for H9_PRO-seq_20_R1_processed.fastq Approx 35% complete for H9_PRO-seq_20_R1_processed.fastq Approx 40% complete for H9_PRO-seq_20_R1_processed.fastq Approx 45% complete for H9_PRO-seq_20_R1_processed.fastq Approx 50% complete for H9_PRO-seq_20_R1_processed.fastq Approx 55% complete for H9_PRO-seq_20_R1_processed.fastq Approx 60% complete for H9_PRO-seq_20_R1_processed.fastq Approx 65% complete for H9_PRO-seq_20_R1_processed.fastq Approx 70% complete for H9_PRO-seq_20_R1_processed.fastq Approx 75% complete for H9_PRO-seq_20_R1_processed.fastq Approx 80% complete for H9_PRO-seq_20_R1_processed.fastq Approx 85% complete for H9_PRO-seq_20_R1_processed.fastq Approx 90% complete for H9_PRO-seq_20_R1_processed.fastq Approx 95% complete for H9_PRO-seq_20_R1_processed.fastq Analysis complete for H9_PRO-seq_20_R1_processed.fastqCommand completed. Elapsed time: 0:00:17. Running peak memory: 0.923GB. PID: 53502; Command: fastqc; Return code: 0; Memory used: 0.175GB > `FastQC report r1` fastqc/H9_PRO-seq_20_R1_processed_fastqc.html FastQC report r1 None PEPPRO _OBJ_ Targetless command, running... > `fastqc --noextract --outdir /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastqc /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R2_trimmed.fastq` (53620)
Started analysis of H9_PRO-seq_20_R2_trimmed.fastq Approx 5% complete for H9_PRO-seq_20_R2_trimmed.fastq Approx 10% complete for H9_PRO-seq_20_R2_trimmed.fastq Approx 15% complete for H9_PRO-seq_20_R2_trimmed.fastq Approx 20% complete for H9_PRO-seq_20_R2_trimmed.fastq Approx 25% complete for H9_PRO-seq_20_R2_trimmed.fastq Approx 30% complete for H9_PRO-seq_20_R2_trimmed.fastq Approx 35% complete for H9_PRO-seq_20_R2_trimmed.fastq Approx 40% complete for H9_PRO-seq_20_R2_trimmed.fastq Approx 45% complete for H9_PRO-seq_20_R2_trimmed.fastq Approx 50% complete for H9_PRO-seq_20_R2_trimmed.fastq Approx 55% complete for H9_PRO-seq_20_R2_trimmed.fastq Approx 60% complete for H9_PRO-seq_20_R2_trimmed.fastq Approx 65% complete for H9_PRO-seq_20_R2_trimmed.fastq Approx 70% complete for H9_PRO-seq_20_R2_trimmed.fastq Approx 75% complete for H9_PRO-seq_20_R2_trimmed.fastq Approx 80% complete for H9_PRO-seq_20_R2_trimmed.fastq Approx 85% complete for H9_PRO-seq_20_R2_trimmed.fastq Approx 90% complete for H9_PRO-seq_20_R2_trimmed.fastq Approx 95% complete for H9_PRO-seq_20_R2_trimmed.fastq Analysis complete for H9_PRO-seq_20_R2_trimmed.fastqCommand completed. Elapsed time: 0:00:15. Running peak memory: 0.923GB. PID: 53620; Command: fastqc; Return code: 0; Memory used: 0.182GB > `FastQC report r2` fastqc/H9_PRO-seq_20_R2_trimmed_fastqc.html FastQC report r2 None PEPPRO _OBJ_ Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/cutadapt/H9_PRO-seq_20.hist`,`/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/cutadapt/H9_PRO-seq_20.histogram` > `fastq_pair -t 20825402 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R1_noadap.fastq /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R2_noadap.fastq` (53727)
Left paired: 5808026 Right paired: 5808026 Left single: 40785 Right single: 535780 Writing the paired reads to /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R1_noadap.fastq.paired.fq and /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R2_noadap.fastq.paired.fq. Writing the single reads to /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R1_noadap.fastq.single.fq and /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R2_noadap.fastq.single.fqCommand completed. Elapsed time: 0:00:45. Running peak memory: 1.008GB. PID: 53727; Command: fastq_pair; Return code: 0; Memory used: 1.008GB > `flash -q -t 4 --compress-prog=pigz --suffix=gz /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R1_noadap.fastq.paired.fq /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R2_noadap.fastq.paired.fq -o H9_PRO-seq_20 -d /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/cutadapt` (53994)
Command completed. Elapsed time: 0:00:31. Running peak memory: 1.008GB. PID: 53994; Command: flash; Return code: 0; Memory used: 0.058GB ### Plot adapter insertion distribution (06-11 17:59:20) elapsed: 245.0 _TIME_ Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/cutadapt/H9_PRO-seq_20_adapter_insertion_distribution.pdf` > `Rscript /scratch/jps3dp/tools/databio//peppro/tools/PEPPRO.R adapt -i /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/cutadapt/H9_PRO-seq_20.hist -o /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/cutadapt -u 8` (54314)
Adapter insertion distribution plot completed!Command completed. Elapsed time: 0:00:07. Running peak memory: 1.008GB. PID: 54314; Command: Rscript; Return code: 0; Memory used: 0.202GB > `Adapter insertion distribution` cutadapt/H9_PRO-seq_20_adapter_insertion_distribution.pdf Adapter insertion distribution cutadapt/H9_PRO-seq_20_adapter_insertion_distribution.png PEPPRO _OBJ_ Missing stat 'Peak_adapter_insertion_size' > `awk 'NR>2 {print prev} {prev=$0}' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/cutadapt/H9_PRO-seq_20.hist | awk 'BEGIN{max= 0; max_len=0; len=0}{if ($2>0+max) {max=$2; len=$1}; max_len=$1} END{print len-8}'` > `Peak_adapter_insertion_size` 20 PEPPRO _RES_ Missing stat 'Degradation_ratio' ### Calculating degradation ratio (06-11 17:59:27) elapsed: 7.0 _TIME_ > `awk '{ if (($1-8) == 10) {status = 1}} END {if (status) {print status} else {print 0}}' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/cutadapt/H9_PRO-seq_20.hist` > `awk '{ if (($1-8) == 20) {status = 1}} END {if (status) {print status} else {print 0}}' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/cutadapt/H9_PRO-seq_20.hist` > `awk '{ if (($1-8) == 30) {status = 1}} END {if (status) {print status} else {print 0}}' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/cutadapt/H9_PRO-seq_20.hist` > `awk '{ if (($1-8) == 40) {status = 1}} END {if (status) {print status} else {print 0}}' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/cutadapt/H9_PRO-seq_20.hist` > `awk '(($1-8 ) <= 20 && ($1-8 ) >= 10){degradedSum += $2}; (($1-8 ) >= 30 && ($1-8) <= 40){intactSum += $2} END {if (intactSum < 1) {intactSum = 1} print degradedSum/intactSum}' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/cutadapt/H9_PRO-seq_20.hist` > `Degradation_ratio` 1.0309 PEPPRO _RES_ Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R2_trimmed_dups.fastq` > `cp /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R2_trimmed.fastq /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R2_trimmed_dups.fastq` (54377)
Command completed. Elapsed time: 0:00:06. Running peak memory: 1.008GB. PID: 54377; Command: cp; Return code: 0; Memory used: 0.002GB Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/processed_R2.flag` > `touch /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/processed_R2.flag` (54414)
Command completed. Elapsed time: 0:00:00. Running peak memory: 1.008GB. PID: 54414; Command: touch; Return code: 0; Memory used: 0.0GB Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/repaired.flag` > `fastq_pair -t 20825402 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R1_processed.fastq /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R2_trimmed.fastq` (54416)
Left paired: 5620420 Right paired: 5620420 Left single: 44987 Right single: 550060 Writing the paired reads to /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R1_processed.fastq.paired.fq and /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R2_trimmed.fastq.paired.fq. Writing the single reads to /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R1_processed.fastq.single.fq and /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R2_trimmed.fastq.single.fqCommand completed. Elapsed time: 0:00:42. Running peak memory: 1.056GB. PID: 54416; Command: fastq_pair; Return code: 0; Memory used: 1.056GB > `mv /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R1_processed.fastq.paired.fq /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R1_processed.fastq` (55018)
Command completed. Elapsed time: 0:00:00. Running peak memory: 1.056GB. PID: 55018; Command: mv; Return code: 0; Memory used: 0.002GB > `mv /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R2_trimmed.fastq.paired.fq /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R2_trimmed.fastq` (55020)
Command completed. Elapsed time: 0:00:00. Running peak memory: 1.056GB. PID: 55020; Command: mv; Return code: 0; Memory used: 0.001GB > `touch /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/repaired.flag` (55024)
Command completed. Elapsed time: 0:00:00. Running peak memory: 1.056GB. PID: 55024; Command: touch; Return code: 0; Memory used: 0.0GB Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/dups_repaired.flag` > `fastq_pair -t 20825402 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R1_trimmed.fastq /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R2_trimmed_dups.fastq` (55027)
Left paired: 5407657 Right paired: 5407657 Left single: 35362 Right single: 762823 Writing the paired reads to /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R1_trimmed.fastq.paired.fq and /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R2_trimmed_dups.fastq.paired.fq. Writing the single reads to /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R1_trimmed.fastq.single.fq and /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R2_trimmed_dups.fastq.single.fqCommand completed. Elapsed time: 0:00:36. Running peak memory: 1.056GB. PID: 55027; Command: fastq_pair; Return code: 0; Memory used: 1.046GB > `mv /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R1_trimmed.fastq.paired.fq /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R1_trimmed.fastq` (55246)
Command completed. Elapsed time: 0:00:00. Running peak memory: 1.056GB. PID: 55246; Command: mv; Return code: 0; Memory used: 0.001GB > `mv /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R2_trimmed_dups.fastq.paired.fq /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R2_trimmed_dups.fastq` (55248)
Command completed. Elapsed time: 0:00:00. Running peak memory: 1.056GB. PID: 55248; Command: mv; Return code: 0; Memory used: 0.001GB > `touch /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/dups_repaired.flag` (55250)
Command completed. Elapsed time: 0:00:00. Running peak memory: 1.056GB. PID: 55250; Command: touch; Return code: 0; Memory used: 0.0GB ### Prealignments (06-11 18:00:54) elapsed: 86.0 _TIME_ Prealignment assemblies: ['human_rDNA'] ### Map to human_rDNA (06-11 18:00:54) elapsed: 0.0 _TIME_ Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/prealignments/human_rDNA_bt2` > `mkfifo /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/prealignments/human_rDNA_bt2` (55251)
Command completed. Elapsed time: 0:00:00. Running peak memory: 1.056GB. PID: 55251; Command: mkfifo; Return code: 0; Memory used: 0.001GB File not added to cleanup: prealignments/human_rDNA_bt2 Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/prealignments/H9_PRO-seq_20_human_rDNA_unmap_R2.fq.gz` > `perl /scratch/jps3dp/tools/databio//peppro/tools/filter_paired_fq.pl /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/prealignments/human_rDNA_bt2 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R1_processed.fastq /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R2_trimmed.fastq /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/prealignments/H9_PRO-seq_20_human_rDNA_unmap_R1.fq /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/prealignments/H9_PRO-seq_20_human_rDNA_unmap_R2.fq` (55252)
> `(bowtie2 -p 4 -k 1 -D 20 -R 3 -N 1 -L 20 -i S,1,0.50 -x /project/shefflab/genomes/human_rDNA/bowtie2_index/default/human_rDNA --rg-id H9_PRO-seq_20 -U /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R1_processed.fastq --un /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/prealignments/human_rDNA_bt2 2>&1 > /dev/null)` not gzipping output File not added to cleanup: prealignments/human_rDNA_bt2 Missing stat 'Aligned_reads_human_rDNA' 5620420 reads; of these: 5620420 (100.00%) were unpaired; of these: 5037944 (89.64%) aligned 0 times 582476 (10.36%) aligned exactly 1 time 0 (0.00%) aligned >1 times 10.36% overall alignment rate > `Aligned_reads_human_rDNA` 1164952.0 PEPPRO _RES_ > `Alignment_rate_human_rDNA` 10.28 PEPPRO _RES_ ### Map to human_rDNA (06-11 18:01:39) elapsed: 45.0 _TIME_ Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/prealignments/human_rDNA_dups_bt2` > `mkfifo /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/prealignments/human_rDNA_dups_bt2` (55654)
Command completed. Elapsed time: 0:00:00. Running peak memory: 1.056GB. PID: 55654; Command: mkfifo; Return code: 0; Memory used: 0.002GB File not added to cleanup: prealignments/human_rDNA_dups_bt2 Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/prealignments/H9_PRO-seq_20_human_rDNA_unmap_R2.fq.gz` > `perl /scratch/jps3dp/tools/databio//peppro/tools/filter_paired_fq.pl /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/prealignments/human_rDNA_dups_bt2 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R1_trimmed.fastq /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R2_trimmed_dups.fastq /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/prealignments/H9_PRO-seq_20_human_rDNA_unmap_dups_R1.fq /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/prealignments/H9_PRO-seq_20_human_rDNA_unmap_dups_R2.fq` (55657)
> `(bowtie2 -p 4 -k 1 -D 20 -R 3 -N 1 -L 20 -i S,1,0.50 -x /project/shefflab/genomes/human_rDNA/bowtie2_index/default/human_rDNA --rg-id H9_PRO-seq_20 -U /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R1_trimmed.fastq --un /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/prealignments/human_rDNA_dups_bt2 2>&1 > /dev/null)` not gzipping output 582476 reads skipped 0 reads lost File not added to cleanup: prealignments/human_rDNA_dups_bt2 ### Map to genome (06-11 18:02:35) elapsed: 56.0 _TIME_ No files match cleanup pattern: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_fail_qc_dups.bam Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_sort.bam` > `bowtie2 -p 4 --very-sensitive -X 2000 --rg-id H9_PRO-seq_20 -x /project/shefflab/genomes/hg38/bowtie2_index/default/hg38 --rf -1 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/prealignments/H9_PRO-seq_20_human_rDNA_unmap_R1.fq -2 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/prealignments/H9_PRO-seq_20_human_rDNA_unmap_R2.fq | samtools view -bS - -@ 1 | samtools sort - -@ 1 -T /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/tmpz5b6rmel -o /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_temp.bam` (56007,56009,56010)
515625 reads skipped 0 reads lost 5037944 reads; of these: 5037944 (100.00%) were paired; of these: 2087736 (41.44%) aligned concordantly 0 times 2420178 (48.04%) aligned concordantly exactly 1 time 530030 (10.52%) aligned concordantly >1 times ---- 2087736 pairs aligned concordantly 0 times; of these: 464022 (22.23%) aligned discordantly 1 time ---- 1623714 pairs aligned 0 times concordantly or discordantly; of these: 3247428 mates make up the pairs; of these: 1333631 (41.07%) aligned 0 times 699598 (21.54%) aligned exactly 1 time 1214199 (37.39%) aligned >1 times 86.76% overall alignment rate [bam_sort_core] merging from 2 files and 1 in-memory blocks...Command completed. Elapsed time: 0:22:54. Running peak memory: 3.556GB. PID: 56007; Command: bowtie2; Return code: 0; Memory used: 3.556GB PID: 56009; Command: samtools; Return code: 0; Memory used: 0.004GB PID: 56010; Command: samtools; Return code: 0; Memory used: 0.896GB > `samtools view -q 10 -b -@ 4 -U /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_fail_qc.bam /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_temp.bam > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_sort.bam` (63957)
Command completed. Elapsed time: 0:00:31. Running peak memory: 3.556GB. PID: 63957; Command: samtools; Return code: 0; Memory used: 0.011GB > `samtools depth -b /project/shefflab/genomes/hg38/refgene_anno/default/hg38_pre-mRNA.bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_sort.bam | awk '{counter++;sum+=$3}END{print sum/counter}'` > `Mapped_reads` 8742257 PEPPRO _RES_ > `QC_filtered_reads` 5108893 PEPPRO _RES_ > `Aligned_reads` 3633363.5 PEPPRO _RES_ > `Alignment_rate` 32.07 PEPPRO _RES_ > `Total_efficiency` 15.7 PEPPRO _RES_ > `Read_depth` 1.77 PEPPRO _RES_ Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_sort_dups.bam` > `bowtie2 -p 4 --very-sensitive -X 2000 --rg-id H9_PRO-seq_20 -x /project/shefflab/genomes/hg38/bowtie2_index/default/hg38 --rf -1 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/prealignments/H9_PRO-seq_20_human_rDNA_unmap_dups_R1.fq -2 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/prealignments/H9_PRO-seq_20_human_rDNA_unmap_dups_R2.fq | samtools view -bS - -@ 1 | samtools sort - -@ 1 -T /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/tmpz5b6rmel -o /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_temp_dups.bam` (65706,65707,65708)
4892032 reads; of these: 4892032 (100.00%) were paired; of these: 1990905 (40.70%) aligned concordantly 0 times 2383527 (48.72%) aligned concordantly exactly 1 time 517600 (10.58%) aligned concordantly >1 times ---- 1990905 pairs aligned concordantly 0 times; of these: 457156 (22.96%) aligned discordantly 1 time ---- 1533749 pairs aligned 0 times concordantly or discordantly; of these: 3067498 mates make up the pairs; of these: 1262056 (41.14%) aligned 0 times 688075 (22.43%) aligned exactly 1 time 1117367 (36.43%) aligned >1 times 87.10% overall alignment rate [bam_sort_core] merging from 2 files and 1 in-memory blocks...Command completed. Elapsed time: 0:21:26. Running peak memory: 3.556GB. PID: 65706; Command: bowtie2; Return code: 0; Memory used: 3.544GB PID: 65707; Command: samtools; Return code: 0; Memory used: 0.004GB PID: 65708; Command: samtools; Return code: 0; Memory used: 0.896GB > `samtools view -q 10 -b -@ 4 -U /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_fail_qc_dups.bam /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_temp_dups.bam > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_sort_dups.bam` (70827)
Command completed. Elapsed time: 0:00:29. Running peak memory: 3.556GB. PID: 70827; Command: samtools; Return code: 0; Memory used: 0.011GB ### Compress all unmapped read files (06-11 18:52:19) elapsed: 2984.0 _TIME_ Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/prealignments/H9_PRO-seq_20_human_rDNA_unmap_R2.fq.gz` > `pigz -f -p 4 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/prealignments/H9_PRO-seq_20_human_rDNA_unmap_R2.fq` (70864)
Command completed. Elapsed time: 0:00:11. Running peak memory: 3.556GB. PID: 70864; Command: pigz; Return code: 0; Memory used: 0.004GB Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/prealignments/H9_PRO-seq_20_human_rDNA_unmap_R1.fq.gz` > `pigz -f -p 4 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/prealignments/H9_PRO-seq_20_human_rDNA_unmap_R1.fq` (70880)
Command completed. Elapsed time: 0:00:13. Running peak memory: 3.556GB. PID: 70880; Command: pigz; Return code: 0; Memory used: 0.004GB Missing stat 'Mitochondrial_reads' Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_temp.bam.bai` > `samtools index /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_temp.bam` (70946)
Command completed. Elapsed time: 0:00:08. Running peak memory: 3.556GB. PID: 70946; Command: samtools; Return code: 0; Memory used: 0.012GB > `samtools idxstats /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_temp.bam | grep -we 'chrM' -we 'chrMT' -we 'M' -we 'MT' -we 'rCRSd' -we 'rCRSd_3k'| cut -f 3` > `Mitochondrial_reads` 168231 PEPPRO _RES_ Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_noMT.bam` > `samtools index /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_sort.bam` (70961)
Command completed. Elapsed time: 0:00:06. Running peak memory: 3.556GB. PID: 70961; Command: samtools; Return code: 0; Memory used: 0.012GB > `samtools idxstats /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_sort.bam | cut -f 1-2 | awk '{print $1, 0, $2}' | grep -vwe 'chrM' -vwe 'chrMT' -vwe 'M' -vwe 'MT' -vwe 'rCRSd' -vwe 'rCRSd_3k' > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/chr_sizes.bed` (70975,70976,70977,70978)
Command completed. Elapsed time: 0:00:00. Running peak memory: 3.556GB. PID: 70976; Command: cut; Return code: 0; Memory used: 0.0GB PID: 70978; Command: grep; Return code: 0; Memory used: 0.0GB PID: 70975; Command: samtools; Return code: 0; Memory used: 0.008GB PID: 70977; Command: awk; Return code: 0; Memory used: 0.0GB > `samtools view -L /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/chr_sizes.bed -b -@ 4 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_sort.bam > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_noMT.bam` (70981)
Command completed. Elapsed time: 0:00:15. Running peak memory: 3.556GB. PID: 70981; Command: samtools; Return code: 0; Memory used: 0.012GB > `mv /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_noMT.bam /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_sort.bam` (71004)
Command completed. Elapsed time: 0:00:00. Running peak memory: 3.556GB. PID: 71004; Command: mv; Return code: 0; Memory used: 0.001GB > `samtools index /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_sort.bam` (71005)
Command completed. Elapsed time: 0:00:06. Running peak memory: 3.556GB. PID: 71005; Command: samtools; Return code: 0; Memory used: 0.012GB ### Split BAM file (06-11 18:53:21) elapsed: 62.0 _TIME_ Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_PE1.bam`,`/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_PE2.bam` > `samtools view -b -f 64 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_sort.bam | samtools sort - -@ 4 > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_PE1.bam` (71011,71012)
[bam_sort_core] merging from 0 files and 4 in-memory blocks...Command completed. Elapsed time: 0:00:36. Running peak memory: 3.556GB. PID: 71011; Command: samtools; Return code: 0; Memory used: 0.004GB PID: 71012; Command: samtools; Return code: 0; Memory used: 1.043GB > `samtools view -b -f 128 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_sort.bam | samtools sort - -@ 4 > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_PE2.bam` (71157,71158)
[bam_sort_core] merging from 0 files and 4 in-memory blocks...Command completed. Elapsed time: 0:00:30. Running peak memory: 3.556GB. PID: 71157; Command: samtools; Return code: 0; Memory used: 0.004GB PID: 71158; Command: samtools; Return code: 0; Memory used: 0.838GB Missing stat 'Maximum_read_length' > `samtools stats /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_PE1.bam | grep '^SN' | cut -f 2- | grep 'maximum length:' | cut -f 2-` > `Maximum_read_length` 38 PEPPRO _RES_ Missing stat 'Genome_size' > `awk '{sum+=$2} END {printf "%.0f", sum}' /project/shefflab/genomes/hg38/fasta/default/hg38.chrom.sizes` > `Genome_size` 3099922541 PEPPRO _RES_ Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_temp_dups.bam.bai` > `samtools index /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_temp_dups.bam` (71219)
Command completed. Elapsed time: 0:00:07. Running peak memory: 3.556GB. PID: 71219; Command: samtools; Return code: 0; Memory used: 0.013GB > `samtools idxstats /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_temp_dups.bam | grep -we 'chrM' -we 'chrMT' -we 'M' -we 'MT' -we 'rCRSd' -we 'rCRSd_3k'| cut -f 3` Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_sort_dups.bam` No files match cleanup pattern: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_sort_dups.bam.bai Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_dups_PE1.bam`,`/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_dups_PE2.bam` > `samtools view -b -f 64 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_sort_dups.bam | samtools sort - -@ 4 > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_dups_PE1.bam` (71276,71277)
Got SIGTERM. Failing gracefully... (06-11 18:55:15) elapsed: 114.0 _TIME_ Child process 55252 (perl) was already terminated. Child process 55657 (perl) was already terminated. [W::bgzf_read_block] EOF marker is absent. The input is probably truncated Child process 71276 (samtools) terminated after 0 sec. Child process 71277 (samtools) terminated after 0 sec. Setting dynamic recover file: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/recover.lock.aligned_hg38__H9_PRO-seq_20_dups_PE1.bam Setting dynamic recover file: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/recover.lock.aligned_hg38__H9_PRO-seq_20_dups_PE2.bam ### Pipeline failed at: (06-11 18:55:15) elapsed: 0.0 _TIME_ Total time: 1:00:58 Failure reason: SIGTERM Pipeline aborted. ### Pipeline run code and environment: * Command: `/scratch/jps3dp/tools/databio//peppro/pipelines/peppro.py --sample-name H9_PRO-seq_20 --genome hg38 --input /project/shefflab/data//guertin/fastq/H9_PRO-seq_20pct_PE1.fastq.gz --single-or-paired PAIRED -O /project/shefflab/processed//peppro/paper/6.11.2020/results_pipeline -P 4 -M 8000 --input2 /project/shefflab/data//guertin/fastq/H9_PRO-seq_20pct_PE2.fastq.gz --protocol PRO --umi-len 8 --scale --prealignments human_rDNA --dirty -R` * Compute host: udc-ba26-35c0 * Working dir: /sfs/lustre/bahamut/scratch/jps3dp/tools/databio/ppqc * Outfolder: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/ * Pipeline started at: (06-11 19:13:13) elapsed: 6.0 _TIME_ ### Version log: * Python version: 3.6.6 * Pypiper dir: `/sfs/qumulo/qhome/jps3dp/.local/lib/python3.6/site-packages/pypiper` * Pypiper version: 0.12.1 * Pipeline dir: `/sfs/lustre/bahamut/scratch/jps3dp/tools/databio/peppro/pipelines` * Pipeline version: 0.9.8 * Pipeline hash: 887a9a85408962dc3ca070dc954e59a5d4d73a10 * Pipeline branch: * master * Pipeline date: 2020-06-09 11:36:49 -0400 * Pipeline diff: 40 files changed, 16373 insertions(+), 3522 deletions(-) ### Arguments passed to pipeline: * `TSS_name`: `None` * `adapter`: `cutadapt` * `anno_name`: `None` * `complexity`: `False` * `config_file`: `peppro.yaml` * `cores`: `4` * `coverage`: `False` * `dedup`: `seqkit` * `dirty`: `True` * `ensembl_gene_body`: `None` * `ensembl_tss`: `None` * `exon_name`: `None` * `force_follow`: `False` * `genome_assembly`: `hg38` * `input`: `['/project/shefflab/data//guertin/fastq/H9_PRO-seq_20pct_PE1.fastq.gz']` * `input2`: `['/project/shefflab/data//guertin/fastq/H9_PRO-seq_20pct_PE2.fastq.gz']` * `intron_name`: `None` * `keep`: `False` * `logdev`: `False` * `max_len`: `-1` * `mem`: `8000` * `new_start`: `False` * `no_fifo`: `False` * `output_parent`: `/project/shefflab/processed//peppro/paper/6.11.2020/results_pipeline` * `paired_end`: `True` * `pre_name`: `None` * `prealignments`: `['human_rDNA']` * `prioritize`: `False` * `protocol`: `PRO` * `recover`: `True` * `sample_name`: `H9_PRO-seq_20` * `scale`: `True` * `search_file`: `None` * `silent`: `False` * `single_or_paired`: `PAIRED` * `sob`: `False` * `testmode`: `False` * `trimmer`: `seqtk` * `umi_len`: `8` * `verbosity`: `None` ---------------------------------------- Removed existing flag: '/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/PEPPRO_failed.flag' Local input file: /project/shefflab/data//guertin/fastq/H9_PRO-seq_20pct_PE1.fastq.gz Local input file: /project/shefflab/data//guertin/fastq/H9_PRO-seq_20pct_PE2.fastq.gz > `File_mb` 492.8 PEPPRO _RES_ > `Read_type` PAIRED PEPPRO _RES_ > `Genome` hg38 PEPPRO _RES_ Detected PRO input ### Merge/link and fastq conversion: (06-11 19:13:14) elapsed: 1.0 _TIME_ Number of input file sets: 2 Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/raw/H9_PRO-seq_20_R1.fastq.gz` Local input file: '/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/raw/H9_PRO-seq_20_R1.fastq.gz' Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/raw/H9_PRO-seq_20_R2.fastq.gz` Local input file: '/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/raw/H9_PRO-seq_20_R2.fastq.gz' Found .fastq.gz file Found .fastq.gz file Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R1.fastq` Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R2.fastq` ['/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/raw/H9_PRO-seq_20_R1.fastq.gz', '/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/raw/H9_PRO-seq_20_R2.fastq.gz'] ### FASTQ processing: (06-11 19:13:14) elapsed: 0.0 _TIME_ Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/processed_R1.flag` Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/processed_R2.flag` Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/repaired.flag` Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/dups_repaired.flag` ### Prealignments (06-11 19:13:14) elapsed: 0.0 _TIME_ Prealignment assemblies: ['human_rDNA'] ### Map to human_rDNA (06-11 19:13:14) elapsed: 0.0 _TIME_ Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/prealignments/human_rDNA_bt2` > `mkfifo /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/prealignments/human_rDNA_bt2` (21214)
Command completed. Elapsed time: 0:00:00. Running peak memory: 0.001GB. PID: 21214; Command: mkfifo; Return code: 0; Memory used: 0.001GB File not added to cleanup: prealignments/human_rDNA_bt2 Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/prealignments/H9_PRO-seq_20_human_rDNA_unmap_R2.fq.gz` File not added to cleanup: prealignments/human_rDNA_bt2 ### Map to human_rDNA (06-11 19:13:14) elapsed: 0.0 _TIME_ Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/prealignments/human_rDNA_dups_bt2` > `mkfifo /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/prealignments/human_rDNA_dups_bt2` (21215)
Command completed. Elapsed time: 0:00:00. Running peak memory: 0.001GB. PID: 21215; Command: mkfifo; Return code: 0; Memory used: 0.0GB File not added to cleanup: prealignments/human_rDNA_dups_bt2 Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/prealignments/H9_PRO-seq_20_human_rDNA_unmap_R2.fq.gz` File not added to cleanup: prealignments/human_rDNA_dups_bt2 ### Map to genome (06-11 19:13:14) elapsed: 0.0 _TIME_ Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_sort.bam` Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_sort.bam` Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_sort.bam` Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_sort_dups.bam` ### Compress all unmapped read files (06-11 19:13:14) elapsed: 0.0 _TIME_ Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/prealignments/H9_PRO-seq_20_human_rDNA_unmap_R1.fq.gz` Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/prealignments/H9_PRO-seq_20_human_rDNA_unmap_R2.fq.gz` ### Split BAM file (06-11 19:13:14) elapsed: 0.0 _TIME_ Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_PE1.bam` Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_PE2.bam` > `samtools idxstats /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_temp_dups.bam | grep -we 'chrM' -we 'chrMT' -we 'M' -we 'MT' -we 'rCRSd' -we 'rCRSd_3k'| cut -f 3` Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_sort_dups.bam` No files match cleanup pattern: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_sort_dups.bam.bai Found lock file: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/lock.aligned_hg38__H9_PRO-seq_20_dups_PE1.bam Overwriting target... Found lock file: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/lock.aligned_hg38__H9_PRO-seq_20_dups_PE2.bam Overwriting target... Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_dups_PE1.bam`,`/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_dups_PE2.bam` > `samtools view -b -f 64 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_sort_dups.bam | samtools sort - -@ 4 > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_dups_PE1.bam` (21221,21222)
[bam_sort_core] merging from 0 files and 4 in-memory blocks...Command completed. Elapsed time: 0:00:39. Running peak memory: 1.05GB. PID: 21221; Command: samtools; Return code: 0; Memory used: 0.004GB PID: 21222; Command: samtools; Return code: 0; Memory used: 1.05GB > `samtools view -b -f 128 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_sort_dups.bam | samtools sort - -@ 4 > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_dups_PE2.bam` (21271,21272)
[bam_sort_core] merging from 0 files and 4 in-memory blocks...Command completed. Elapsed time: 0:00:32. Running peak memory: 1.05GB. PID: 21271; Command: samtools; Return code: 0; Memory used: 0.004GB PID: 21272; Command: samtools; Return code: 0; Memory used: 0.846GB ### Calculate library complexity (06-11 19:14:25) elapsed: 71.0 _TIME_ Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_preseq_out.txt` > `preseq c_curve -v -o /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_preseq_out.txt -B /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_dups_PE1.bam` (21318)
BAM_INPUT TOTAL READS = 3908887 COUNTS_SUM = 3908887 DISTINCT READS = 3.55352e+06 DISTINCT COUNTS = 127 MAX COUNT = 7844 COUNTS OF 1 = 3.36495e+06 OBSERVED COUNTS (7845) 1 3364952 2 139399 3 26560 4 9493 5 4329 6 2470 7 1478 8 1000 9 680 10 535 11 410 12 334 13 255 14 181 15 166 16 150 17 108 18 101 19 88 20 79 21 68 22 55 23 48 24 47 25 41 26 44 27 36 28 15 29 30 30 19 31 19 32 24 33 17 34 15 35 16 36 11 37 10 38 11 39 13 40 10 41 9 42 7 43 11 44 7 45 10 46 9 47 12 48 4 49 10 50 6 51 2 52 3 53 2 54 2 55 5 56 4 57 2 58 4 59 4 60 5 61 4 62 4 63 2 64 1 65 1 67 2 68 2 70 3 71 2 72 3 73 3 74 3 78 1 79 2 80 1 81 1 82 1 83 1 86 2 89 1 91 1 93 1 94 1 95 1 96 1 98 1 100 1 101 1 104 1 107 1 108 1 109 1 112 1 114 1 115 2 119 1 120 1 125 1 135 1 141 1 143 1 148 1 154 1 155 1 156 1 173 1 192 1 195 1 241 1 251 1 296 1 298 1 303 1 351 1 406 1 414 1 445 1 521 1 529 1 578 1 773 1 1010 1 1984 1 2411 1 2958 1 6314 1 7844 1 sample size: 1000000 sample size: 2000000 sample size: 3000000Command completed. Elapsed time: 0:00:23. Running peak memory: 1.05GB. PID: 21318; Command: preseq; Return code: 0; Memory used: 0.005GB Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_preseq_yield.txt` > `preseq lc_extrap -v -o /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_preseq_yield.txt -B /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_dups_PE1.bam` (21337)
BAM_INPUT TOTAL READS = 3908887 DISTINCT READS = 3.55352e+06 DISTINCT COUNTS = 127 MAX COUNT = 7844 COUNTS OF 1 = 3.36495e+06 MAX TERMS = 64 OBSERVED COUNTS (7845) 1 3364952 2 139399 3 26560 4 9493 5 4329 6 2470 7 1478 8 1000 9 680 10 535 11 410 12 334 13 255 14 181 15 166 16 150 17 108 18 101 19 88 20 79 21 68 22 55 23 48 24 47 25 41 26 44 27 36 28 15 29 30 30 19 31 19 32 24 33 17 34 15 35 16 36 11 37 10 38 11 39 13 40 10 41 9 42 7 43 11 44 7 45 10 46 9 47 12 48 4 49 10 50 6 51 2 52 3 53 2 54 2 55 5 56 4 57 2 58 4 59 4 60 5 61 4 62 4 63 2 64 1 65 1 67 2 68 2 70 3 71 2 72 3 73 3 74 3 78 1 79 2 80 1 81 1 82 1 83 1 86 2 89 1 91 1 93 1 94 1 95 1 96 1 98 1 100 1 101 1 104 1 107 1 108 1 109 1 112 1 114 1 115 2 119 1 120 1 125 1 135 1 141 1 143 1 148 1 154 1 155 1 156 1 173 1 192 1 195 1 241 1 251 1 296 1 298 1 303 1 351 1 406 1 414 1 445 1 521 1 529 1 578 1 773 1 1010 1 1984 1 2411 1 2958 1 6314 1 7844 1 [ESTIMATING YIELD CURVE] [BOOTSTRAPPING HISTOGRAM] _................................_.................................................................... [COMPUTING CONFIDENCE INTERVALS] [WRITING OUTPUT]Command completed. Elapsed time: 0:00:26. Running peak memory: 1.05GB. PID: 21337; Command: preseq; Return code: 0; Memory used: 0.005GB Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_preseq_counts.txt` > `echo '/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_preseq_yield.txt '$(samtools view -c -F 4 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_dups_PE1.bam)' '$(samtools view -c -F 4 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_PE1.bam) > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_preseq_counts.txt` (21564)
Command completed. Elapsed time: 0:00:07. Running peak memory: 1.05GB. PID: 21564; Command: echo; Return code: 0; Memory used: 0.005GB Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_preseq_plot.pdf`,`/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_preseq_plot.png` > `Rscript /scratch/jps3dp/tools/databio//peppro/tools/PEPPRO.R preseq -i /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_preseq_yield.txt -r /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_preseq_counts.txt -o /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_preseq_plot` (21575)
Processing H9_PRO-seq_20 INFO: Found real counts for H9_PRO-seq_20 - Total (M): 3.896875 Unique (M): 3.908887 Library complexity plot completed!Command completed. Elapsed time: 0:00:08. Running peak memory: 1.05GB. PID: 21575; Command: Rscript; Return code: 0; Memory used: 0.129GB > `Library complexity` QC_hg38/H9_PRO-seq_20_preseq_plot.pdf Library complexity QC_hg38/H9_PRO-seq_20_preseq_plot.png PEPPRO _OBJ_ Missing stat 'Frac_exp_unique_at_10M' > `grep -w '10000000' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_preseq_yield.txt | awk '{print $2}'` > `Frac_exp_unique_at_10M` 0.8521 PEPPRO _RES_ ### Calculate NRF, PBC1, and PBC2 (06-11 19:15:30) elapsed: 64.0 _TIME_ Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_PE1.bam.bai` > `samtools index /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_PE1.bam` (21598)
Command completed. Elapsed time: 0:00:04. Running peak memory: 1.05GB. PID: 21598; Command: samtools; Return code: 0; Memory used: 0.006GB Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_bamQC.tsv` > `/scratch/jps3dp/tools/databio//peppro/tools/bamQC.py --silent -i /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_PE1.bam -c 4 -o /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_bamQC.tsv` (21602)
Configured logger 'root' using pararead v0.6 Registering input file: '/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_PE1.bam' Temporary files will be stored in: '/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/tmp_H9_PRO-seq_20_PE1_qmolbs_6' Processing with 4 cores... Discarding 114 chunk(s) of reads: ['chrM', 'chr1_KI270707v1_random', 'chr1_KI270708v1_random', 'chr1_KI270710v1_random', 'chr2_KI270715v1_random', 'chr2_KI270716v1_random', 'chr4_GL000008v2_random', 'chr9_KI270717v1_random', 'chr9_KI270720v1_random', 'chr14_GL000009v2_random', 'chr14_KI270725v1_random', 'chr22_KI270731v1_random', 'chr22_KI270732v1_random', 'chr22_KI270735v1_random', 'chr22_KI270738v1_random', 'chr22_KI270739v1_random', 'chrY_KI270740v1_random', 'chrUn_KI270302v1', 'chrUn_KI270304v1', 'chrUn_KI270303v1', 'chrUn_KI270305v1', 'chrUn_KI270322v1', 'chrUn_KI270320v1', 'chrUn_KI270310v1', 'chrUn_KI270316v1', 'chrUn_KI270315v1', 'chrUn_KI270312v1', 'chrUn_KI270311v1', 'chrUn_KI270317v1', 'chrUn_KI270412v1', 'chrUn_KI270411v1', 'chrUn_KI270414v1', 'chrUn_KI270419v1', 'chrUn_KI270418v1', 'chrUn_KI270420v1', 'chrUn_KI270424v1', 'chrUn_KI270417v1', 'chrUn_KI270422v1', 'chrUn_KI270423v1', 'chrUn_KI270425v1', 'chrUn_KI270429v1', 'chrUn_KI270466v1', 'chrUn_KI270465v1', 'chrUn_KI270467v1', 'chrUn_KI270468v1', 'chrUn_KI270510v1', 'chrUn_KI270509v1', 'chrUn_KI270518v1', 'chrUn_KI270508v1', 'chrUn_KI270516v1', 'chrUn_KI270512v1', 'chrUn_KI270522v1', 'chrUn_KI270511v1', 'chrUn_KI270515v1', 'chrUn_KI270507v1', 'chrUn_KI270517v1', 'chrUn_KI270529v1', 'chrUn_KI270528v1', 'chrUn_KI270530v1', 'chrUn_KI270538v1', 'chrUn_KI270544v1', 'chrUn_KI270548v1', 'chrUn_KI270583v1', 'chrUn_KI270587v1', 'chrUn_KI270580v1', 'chrUn_KI270581v1', 'chrUn_KI270579v1', 'chrUn_KI270590v1', 'chrUn_KI270584v1', 'chrUn_KI270582v1', 'chrUn_KI270588v1', 'chrUn_KI270591v1', 'chrUn_KI270330v1', 'chrUn_KI270329v1', 'chrUn_KI270334v1', 'chrUn_KI270335v1', 'chrUn_KI270338v1', 'chrUn_KI270340v1', 'chrUn_KI270336v1', 'chrUn_KI270363v1', 'chrUn_KI270364v1', 'chrUn_KI270362v1', 'chrUn_KI270366v1', 'chrUn_KI270378v1', 'chrUn_KI270379v1', 'chrUn_KI270389v1', 'chrUn_KI270390v1', 'chrUn_KI270387v1', 'chrUn_KI270395v1', 'chrUn_KI270396v1', 'chrUn_KI270388v1', 'chrUn_KI270394v1', 'chrUn_KI270386v1', 'chrUn_KI270391v1', 'chrUn_KI270383v1', 'chrUn_KI270393v1', 'chrUn_KI270384v1', 'chrUn_KI270392v1', 'chrUn_KI270381v1', 'chrUn_KI270385v1', 'chrUn_KI270382v1', 'chrUn_KI270376v1', 'chrUn_KI270374v1', 'chrUn_KI270372v1', 'chrUn_KI270373v1', 'chrUn_KI270375v1', 'chrUn_KI270371v1', 'chrUn_KI270448v1', 'chrUn_KI270521v1', 'chrUn_GL000226v1', 'chrUn_KI270744v1', 'chrUn_KI270745v1', 'chrUn_KI270754v1', 'chrUn_KI270755v1'] Keeping 81 chunk(s) of reads: ['chr1', 'chr2', 'chr3', 'chr4', 'chr5', 'chr6', 'chr7', 'chr8', 'chr9', 'chr10', 'chr11', 'chr12', 'chr13', 'chr14', 'chr15', 'chr16', 'chr17', 'chr18', 'chr19', 'chr20', 'chr21', 'chr22', 'chrX', 'chrY', 'chr1_KI270706v1_random', 'chr1_KI270709v1_random', 'chr1_KI270711v1_random', 'chr1_KI270712v1_random', 'chr1_KI270713v1_random', 'chr1_KI270714v1_random', 'chr3_GL000221v1_random', 'chr5_GL000208v1_random', 'chr9_KI270718v1_random', 'chr9_KI270719v1_random', 'chr11_KI270721v1_random', 'chr14_GL000225v1_random', 'chr14_KI270722v1_random', 'chr14_GL000194v1_random', 'chr14_KI270723v1_random', 'chr14_KI270724v1_random', 'chr14_KI270726v1_random', 'chr15_KI270727v1_random', 'chr16_KI270728v1_random', 'chr17_GL000205v2_random', 'chr17_KI270729v1_random', 'chr17_KI270730v1_random', 'chr22_KI270733v1_random', 'chr22_KI270734v1_random', 'chr22_KI270736v1_random', 'chr22_KI270737v1_random', 'chrUn_KI270442v1', 'chrUn_KI270435v1', 'chrUn_KI270438v1', 'chrUn_KI270519v1', 'chrUn_KI270539v1', 'chrUn_KI270589v1', 'chrUn_KI270593v1', 'chrUn_KI270333v1', 'chrUn_KI270337v1', 'chrUn_GL000195v1', 'chrUn_GL000219v1', 'chrUn_GL000220v1', 'chrUn_GL000224v1', 'chrUn_KI270741v1', 'chrUn_GL000213v1', 'chrUn_KI270743v1', 'chrUn_KI270746v1', 'chrUn_KI270747v1', 'chrUn_KI270748v1', 'chrUn_KI270749v1', 'chrUn_KI270750v1', 'chrUn_KI270751v1', 'chrUn_KI270752v1', 'chrUn_KI270753v1', 'chrUn_KI270756v1', 'chrUn_KI270757v1', 'chrUn_GL000214v1', 'chrUn_KI270742v1', 'chrUn_GL000216v2', 'chrUn_GL000218v1', 'chrEBV']Command completed. Elapsed time: 0:00:11. Running peak memory: 1.05GB. PID: 21602; Command: /scratch/jps3dp/tools/databio//peppro/tools/bamQC.py; Return code: 0; Memory used: 0.285GB > `awk '{ for (i=1; i<=NF; ++i) { if ($i ~ "NRF") c=i } getline; print $c }' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_bamQC.tsv` > `awk '{ for (i=1; i<=NF; ++i) { if ($i ~ "PBC1") c=i } getline; print $c }' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_bamQC.tsv` > `awk '{ for (i=1; i<=NF; ++i) { if ($i ~ "PBC2") c=i } getline; print $c }' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_bamQC.tsv` > `NRF` 1.0 PEPPRO _RES_ > `PBC1` 1948437.5 PEPPRO _RES_ > `PBC2` 1948437.5 PEPPRO _RES_ Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_unmap.bam` > `samtools view -b -@ 4 -f 12 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_temp.bam > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_unmap.bam` (21625)
Command completed. Elapsed time: 0:00:04. Running peak memory: 1.05GB. PID: 21625; Command: samtools; Return code: 0; Memory used: 0.006GB > `samtools view -c -f 4 -@ 4 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_temp.bam` > `Unmapped_reads` 1333631 PEPPRO _RES_ ### Split BAM by strand (06-11 19:15:51) elapsed: 21.0 _TIME_ Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_plus.bam`,`/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_minus.bam` > `samtools view -bh -F 20 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_PE1.bam > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_plus.bam` (21644)
Command completed. Elapsed time: 0:00:16. Running peak memory: 1.05GB. PID: 21644; Command: samtools; Return code: 0; Memory used: 0.006GB > `samtools view -bh -f 16 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_PE1.bam > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_minus.bam` (21662)
Command completed. Elapsed time: 0:00:16. Running peak memory: 1.05GB. PID: 21662; Command: samtools; Return code: 0; Memory used: 0.006GB ### Calculate TSS enrichment (06-11 19:16:24) elapsed: 32.0 _TIME_ Missing stat 'TSS_non-coding_score' Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/plus_TSS.tsv`,`/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/minus_TSS.tsv` > `sed -n -e '/[[:space:]]+/w /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/plus_TSS.tsv' -e '/[[:space:]]-/w /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/minus_TSS.tsv' /project/shefflab/genomes/hg38/refgene_anno/default/hg38_TSS.bed` (21677)
Command completed. Elapsed time: 0:00:00. Running peak memory: 1.05GB. PID: 21677; Command: sed; Return code: 0; Memory used: 0.002GB Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_plus_TssEnrichment.txt` > `/scratch/jps3dp/tools/databio//peppro/tools/pyTssEnrichment.py -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_PE1.bam -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/plus_TSS.tsv -p ends -c 4 -z -v -s 6 -o /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_plus_TssEnrichment.txt` (21678)
Command completed. Elapsed time: 0:00:05. Running peak memory: 1.05GB. PID: 21678; Command: /scratch/jps3dp/tools/databio//peppro/tools/pyTssEnrichment.py; Return code: 0; Memory used: 0.321GB > `TSS_coding_score` 33.4 PEPPRO _RES_ Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_minus_TssEnrichment.txt` > `/scratch/jps3dp/tools/databio//peppro/tools/pyTssEnrichment.py -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_PE1.bam -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/minus_TSS.tsv -p ends -c 4 -z -v -s 6 -o /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_minus_TssEnrichment.txt` (21694)
Command completed. Elapsed time: 0:00:05. Running peak memory: 1.05GB. PID: 21694; Command: /scratch/jps3dp/tools/databio//peppro/tools/pyTssEnrichment.py; Return code: 0; Memory used: 0.354GB > `TSS_non-coding_score` 12.1 PEPPRO _RES_ Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_TSSenrichment.pdf` > `Rscript /scratch/jps3dp/tools/databio//peppro/tools/PEPPRO.R tss -i /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_plus_TssEnrichment.txt /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_minus_TssEnrichment.txt` (21710)
Generating TSS plot with /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_plus_TssEnrichment.txt and /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_minus_TssEnrichment.txt `geom_smooth()` using formula 'y ~ x' `geom_smooth()` using formula 'y ~ x' `geom_smooth()` using formula 'y ~ x' `geom_smooth()` using formula 'y ~ x' TSS enrichment plot completed!Command completed. Elapsed time: 0:00:07. Running peak memory: 1.05GB. PID: 21710; Command: Rscript; Return code: 0; Memory used: 0.285GB > `TSS enrichment` QC_hg38/H9_PRO-seq_20_TSSenrichment.pdf TSS enrichment QC_hg38/H9_PRO-seq_20_TSSenrichment.png PEPPRO _OBJ_ Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/chr_order.txt` > `samtools view -H /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_PE1.bam | grep 'SN:' | awk -F':' '{print $2,$3}' | awk -F' ' -v OFS=' ' '{print $1,$3}' > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/chr_order.txt` (21728,21729,21730,21731)
Command completed. Elapsed time: 0:00:00. Running peak memory: 1.05GB. PID: 21728; Command: samtools; Return code: 0; Memory used: 0.0GB PID: 21730; Command: awk; Return code: 0; Memory used: 0.0GB PID: 21729; Command: grep; Return code: 0; Memory used: 0.0GB PID: 21731; Command: awk; Return code: 0; Memory used: 0.0GB Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/chr_keep.txt` > `cut -f 1 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/chr_keep.txt` (21733)
Command completed. Elapsed time: 0:00:00. Running peak memory: 1.05GB. PID: 21733; Command: cut; Return code: 0; Memory used: 0.0GB ### Calculate Pause Index (PI) (06-11 19:16:42) elapsed: 18.0 _TIME_ Missing stat 'Pause_index' Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/hg38_ensembl_tss.bed`,`/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/hg38_ensembl_gene_body.bed` > `grep -wf /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/chr_keep.txt /project/shefflab/genomes/hg38/ensembl_gtf/default/hg38_ensembl_TSS.bed | bedtools sort -i stdin -faidx /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/hg38_ensembl_tss.bed` (21735,21736)
Command completed. Elapsed time: 0:00:02. Running peak memory: 1.05GB. PID: 21735; Command: grep; Return code: 0; Memory used: 0.003GB PID: 21736; Command: bedtools; Return code: 0; Memory used: 0.099GB > `grep -wf /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/chr_keep.txt /project/shefflab/genomes/hg38/ensembl_gtf/default/hg38_ensembl_gene_body.bed | bedtools sort -i stdin -faidx /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/hg38_ensembl_gene_body.bed` (21740,21741)
Command completed. Elapsed time: 0:00:00. Running peak memory: 1.05GB. PID: 21740; Command: grep; Return code: 0; Memory used: 0.003GB PID: 21741; Command: bedtools; Return code: 0; Memory used: 0.004GB Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_TSS_density.bed` > `bedtools coverage -sorted -counts -s -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/hg38_ensembl_tss.bed -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_PE1.bam -g /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/chr_order.txt | awk '$7>0' | sort -k4,4 -k7,7nr | sort -k4,4 -u > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_TSS_density.bed` (21743,21744,21745,21746)
Command completed. Elapsed time: 0:00:07. Running peak memory: 1.05GB. PID: 21744; Command: awk; Return code: 0; Memory used: 0.001GB PID: 21746; Command: sort; Return code: 0; Memory used: 0.002GB PID: 21743; Command: bedtools; Return code: 0; Memory used: 0.006GB PID: 21745; Command: sort; Return code: 0; Memory used: 0.005GB Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_gene_body_density.bed` > `bedtools coverage -sorted -counts -s -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/hg38_ensembl_gene_body.bed -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_PE1.bam -g /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/chr_order.txt | awk '$7>0' | sort -k4 > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_gene_body_density.bed` (21754,21755,21756)
Command completed. Elapsed time: 0:00:08. Running peak memory: 1.05GB. PID: 21755; Command: awk; Return code: 0; Memory used: 0.001GB PID: 21754; Command: bedtools; Return code: 0; Memory used: 0.027GB PID: 21756; Command: sort; Return code: 0; Memory used: 0.006GB Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_pause_index.bed` > `join --nocheck-order -j4 -o 1.1 1.2 1.3 1.4 1.6 1.7 2.2 2.3 2.7 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_TSS_density.bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_gene_body_density.bed | awk -v OFS=' ' '{ if ($5 == "+"){print $1, $2, $8, $4, sqrt((($6+$9)/sqrt(($8-$2)^2))^2), ($6/sqrt(($3-$2)^2))/($9/sqrt(($8-$7)^2)), $5} else {print $1, $2, $8, $4, sqrt((($6+$9)/sqrt(($3-$7)^2))^2),($6/sqrt(($3-$2)^2))/($9/sqrt(($8-$7)^2)), $5}}' | env LC_COLLATE=C sort -k1,1 -k2,2n > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/tmpnoh0qgsn` (21768,21769,21770)
Command completed. Elapsed time: 0:00:00. Running peak memory: 1.05GB. PID: 21768; Command: join; Return code: 0; Memory used: 0.001GB PID: 21770; Command: env; Return code: 0; Memory used: 0.005GB PID: 21769; Command: awk; Return code: 0; Memory used: 0.001GB > `awk '{print $5}' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/tmpnoh0qgsn | sort -n | awk 'BEGIN{i=0} {s[i]=$1; i++;} END{print s[int(NR*0.5-0.5)]}` /bin/sh: -c: line 0: unexpected EOF while looking for matching `'' /bin/sh: -c: line 1: syntax error: unexpected end of file ### Pipeline failed at: (06-11 19:17:00) elapsed: 18.0 _TIME_ Total time: 0:03:53 Failure reason: Command 'awk '{print $5}' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/tmpnoh0qgsn | sort -n | awk 'BEGIN{i=0} {s[i]=$1; i++;} END{print s[int(NR*0.5-0.5)]}' returned non-zero exit status 1. ### Pipeline run code and environment: * Command: `/scratch/jps3dp/tools/databio//peppro/pipelines/peppro.py --sample-name H9_PRO-seq_20 --genome hg38 --input /project/shefflab/data//guertin/fastq/H9_PRO-seq_20pct_PE1.fastq.gz --single-or-paired PAIRED -O /project/shefflab/processed//peppro/paper/6.11.2020/results_pipeline -P 4 -M 8000 --input2 /project/shefflab/data//guertin/fastq/H9_PRO-seq_20pct_PE2.fastq.gz --protocol PRO --umi-len 8 --scale --prealignments human_rDNA --dirty -R` * Compute host: udc-aj37-17c1 * Working dir: /sfs/lustre/bahamut/scratch/jps3dp/tools/databio/ppqc * Outfolder: /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/ * Pipeline started at: (06-14 21:18:39) elapsed: 5.0 _TIME_ ### Version log: * Python version: 3.6.6 * Pypiper dir: `/sfs/qumulo/qhome/jps3dp/.local/lib/python3.6/site-packages/pypiper` * Pypiper version: 0.12.1 * Pipeline dir: `/sfs/lustre/bahamut/scratch/jps3dp/tools/databio/peppro/pipelines` * Pipeline version: 0.9.8 * Pipeline hash: 887a9a85408962dc3ca070dc954e59a5d4d73a10 * Pipeline branch: * master * Pipeline date: 2020-06-09 11:36:49 -0400 * Pipeline diff: 40 files changed, 16373 insertions(+), 3522 deletions(-) ### Arguments passed to pipeline: * `TSS_name`: `None` * `adapter`: `cutadapt` * `anno_name`: `None` * `complexity`: `False` * `config_file`: `peppro.yaml` * `cores`: `4` * `coverage`: `False` * `dedup`: `seqkit` * `dirty`: `True` * `ensembl_gene_body`: `None` * `ensembl_tss`: `None` * `exon_name`: `None` * `force_follow`: `False` * `genome_assembly`: `hg38` * `input`: `['/project/shefflab/data//guertin/fastq/H9_PRO-seq_20pct_PE1.fastq.gz']` * `input2`: `['/project/shefflab/data//guertin/fastq/H9_PRO-seq_20pct_PE2.fastq.gz']` * `intron_name`: `None` * `keep`: `False` * `logdev`: `False` * `max_len`: `-1` * `mem`: `8000` * `new_start`: `False` * `no_fifo`: `False` * `output_parent`: `/project/shefflab/processed//peppro/paper/6.11.2020/results_pipeline` * `paired_end`: `True` * `pre_name`: `None` * `prealignments`: `['human_rDNA']` * `prioritize`: `False` * `protocol`: `PRO` * `recover`: `True` * `sample_name`: `H9_PRO-seq_20` * `scale`: `True` * `search_file`: `None` * `silent`: `False` * `single_or_paired`: `PAIRED` * `sob`: `False` * `testmode`: `False` * `trimmer`: `seqtk` * `umi_len`: `8` * `verbosity`: `None` ---------------------------------------- Removed existing flag: '/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/PEPPRO_failed.flag' Local input file: /project/shefflab/data//guertin/fastq/H9_PRO-seq_20pct_PE1.fastq.gz Local input file: /project/shefflab/data//guertin/fastq/H9_PRO-seq_20pct_PE2.fastq.gz > `File_mb` 492.8 PEPPRO _RES_ > `Read_type` PAIRED PEPPRO _RES_ > `Genome` hg38 PEPPRO _RES_ Detected PRO input ### Merge/link and fastq conversion: (06-14 21:18:40) elapsed: 1.0 _TIME_ Number of input file sets: 2 Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/raw/H9_PRO-seq_20_R1.fastq.gz` Local input file: '/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/raw/H9_PRO-seq_20_R1.fastq.gz' Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/raw/H9_PRO-seq_20_R2.fastq.gz` Local input file: '/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/raw/H9_PRO-seq_20_R2.fastq.gz' Found .fastq.gz file Found .fastq.gz file Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R1.fastq` Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/H9_PRO-seq_20_R2.fastq` ['/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/raw/H9_PRO-seq_20_R1.fastq.gz', '/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/raw/H9_PRO-seq_20_R2.fastq.gz'] ### FASTQ processing: (06-14 21:18:40) elapsed: 0.0 _TIME_ Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/processed_R1.flag` Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/processed_R2.flag` Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/repaired.flag` Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/fastq/dups_repaired.flag` ### Prealignments (06-14 21:18:40) elapsed: 0.0 _TIME_ Prealignment assemblies: ['human_rDNA'] ### Map to human_rDNA (06-14 21:18:40) elapsed: 0.0 _TIME_ Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/prealignments/human_rDNA_bt2` > `mkfifo /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/prealignments/human_rDNA_bt2` (315045)
Command completed. Elapsed time: 0:00:00. Running peak memory: 0.0GB. PID: 315045; Command: mkfifo; Return code: 0; Memory used: 0.0GB File not added to cleanup: prealignments/human_rDNA_bt2 Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/prealignments/H9_PRO-seq_20_human_rDNA_unmap_R2.fq.gz` File not added to cleanup: prealignments/human_rDNA_bt2 ### Map to human_rDNA (06-14 21:18:40) elapsed: 0.0 _TIME_ Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/prealignments/human_rDNA_dups_bt2` > `mkfifo /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/prealignments/human_rDNA_dups_bt2` (315138)
Command completed. Elapsed time: 0:00:00. Running peak memory: 0.0GB. PID: 315138; Command: mkfifo; Return code: 0; Memory used: 0.0GB File not added to cleanup: prealignments/human_rDNA_dups_bt2 Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/prealignments/H9_PRO-seq_20_human_rDNA_unmap_R2.fq.gz` File not added to cleanup: prealignments/human_rDNA_dups_bt2 ### Map to genome (06-14 21:18:40) elapsed: 0.0 _TIME_ Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_sort.bam` Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_sort.bam` Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_sort.bam` ### Compress all unmapped read files (06-14 21:18:40) elapsed: 0.0 _TIME_ Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/prealignments/H9_PRO-seq_20_human_rDNA_unmap_R2.fq.gz` Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/prealignments/H9_PRO-seq_20_human_rDNA_unmap_R1.fq.gz` ### Split BAM file (06-14 21:18:40) elapsed: 0.0 _TIME_ Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_PE1.bam` Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_PE2.bam` ### Calculate NRF, PBC1, and PBC2 (06-14 21:18:40) elapsed: 0.0 _TIME_ Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_PE1.bam.bai` Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_bamQC.tsv` Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_unmap.bam` ### Split BAM by strand (06-14 21:18:40) elapsed: 0.0 _TIME_ Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_plus.bam` Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_minus.bam` ### Calculate TSS enrichment (06-14 21:18:40) elapsed: 0.0 _TIME_ Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_TSSenrichment.pdf` > `TSS enrichment` QC_hg38/H9_PRO-seq_20_TSSenrichment.pdf TSS enrichment QC_hg38/H9_PRO-seq_20_TSSenrichment.png PEPPRO _OBJ_ ### Calculate Pause Index (PI) (06-14 21:18:40) elapsed: 0.0 _TIME_ Missing stat 'Pause_index' Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/hg38_ensembl_tss.bed` Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/hg38_ensembl_gene_body.bed` Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_TSS_density.bed` Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_gene_body_density.bed` Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_pause_index.bed` > `join --nocheck-order -j4 -o 1.1 1.2 1.3 1.4 1.6 1.7 2.2 2.3 2.7 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_TSS_density.bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_gene_body_density.bed | awk -v OFS=' ' '{ if ($5 == "+"){print $1, $2, $8, $4, sqrt((($6+$9)/sqrt(($8-$2)^2))^2), ($6/sqrt(($3-$2)^2))/($9/sqrt(($8-$7)^2)), $5} else {print $1, $2, $8, $4, sqrt((($6+$9)/sqrt(($3-$7)^2))^2),($6/sqrt(($3-$2)^2))/($9/sqrt(($8-$7)^2)), $5}}' | env LC_COLLATE=C sort -k1,1 -k2,2n > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/tmp776j47gc` (315371,315391,315420)
Command completed. Elapsed time: 0:00:00. Running peak memory: 0.004GB. PID: 315371; Command: join; Return code: 0; Memory used: 0.001GB PID: 315420; Command: env; Return code: 0; Memory used: 0.004GB PID: 315391; Command: awk; Return code: 0; Memory used: 0.001GB > `awk '{print $5}' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/tmp776j47gc | sort -n | awk 'BEGIN{i=0} {s[i]=$1; i++;} END{print s[int(NR*0.5-0.5)]}'` Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_pause_index.bed` > `awk -v OFS=' ' '{ if ($5 > 0.0047564) {print $1, $2, $3, $4, $6, $7}}' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/tmp776j47gc > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_pause_index.bed` (315958)
Command completed. Elapsed time: 0:00:00. Running peak memory: 0.004GB. PID: 315958; Command: awk; Return code: 0; Memory used: 0.002GB > `sort -k5,5n /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_pause_index.bed | awk ' { a[i++]=$5; } END { x=int((i+1)/2); if (x < (i+1)/2) print (a[x-1]+a[x])/2; else print a[x-1]; }'` > `Pause_index` 33.2 PEPPRO _RES_ Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_pause_index.pdf` > `Rscript /scratch/jps3dp/tools/databio//peppro/tools/PEPPRO.R pi --annotate -i /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_pause_index.bed` (316144)
Pause index plot completed!Command completed. Elapsed time: 0:00:06. Running peak memory: 0.204GB. PID: 316144; Command: Rscript; Return code: 0; Memory used: 0.204GB > `Pause index` QC_hg38/H9_PRO-seq_20_pause_index.pdf Pause index QC_hg38/H9_PRO-seq_20_pause_index.png PEPPRO _OBJ_ Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_pause_index.bed.gz` > `pigz -f -p 4 -f /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_pause_index.bed` (325365)
Command completed. Elapsed time: 0:00:00. Running peak memory: 0.204GB. PID: 325365; Command: pigz; Return code: 0; Memory used: 0.002GB ### Calculate Fraction of Reads in pre-mature mRNA (06-14 21:18:47) elapsed: 7.0 _TIME_ Missing stat 'Plus_FRiP' > `samtools view -@ 4 -c -L /project/shefflab/genomes/hg38/refgene_anno/default/hg38_pre-mRNA.bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_plus.bam` 3633363.5 1411171 > `Plus_FRiP` 0.39 PEPPRO _RES_ Missing stat 'Minus_FRiP' > `samtools view -@ 4 -c -L /project/shefflab/genomes/hg38/refgene_anno/default/hg38_pre-mRNA.bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_minus.bam` 3633363.5 1313835 > `Minus_FRiP` 0.36 PEPPRO _RES_ Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/signal_hg38/H9_PRO-seq_20_gene_coverage.bed` > `grep -wf /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/chr_keep.txt /project/shefflab/genomes/hg38/refgene_anno/default/hg38_pre-mRNA.bed | bedtools sort -i stdin -faidx /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/hg38_gene_sort.bed` (331058,331060)
Command completed. Elapsed time: 0:00:01. Running peak memory: 0.204GB. PID: 331058; Command: grep; Return code: 0; Memory used: 0.004GB PID: 331060; Command: bedtools; Return code: 0; Memory used: 0.006GB > `bedtools coverage -sorted -counts -s -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/hg38_gene_sort.bed -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_PE1.bam -g /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/signal_hg38/H9_PRO-seq_20_gene_coverage.bed` (331425)
Command completed. Elapsed time: 0:00:08. Running peak memory: 0.204GB. PID: 331425; Command: bedtools; Return code: 0; Memory used: 0.038GB Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/raw/hg38_annotations.bed` > `ln -sf /project/shefflab/genomes/hg38/feat_annotation/default/hg38_annotations.bed.gz /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/raw/hg38_annotations.bed.gz` (331791)
Command completed. Elapsed time: 0:00:00. Running peak memory: 0.204GB. PID: 331791; Command: ln; Return code: 0; Memory used: 0.0GB > `pigz -f -p 4 -d -c /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/raw/hg38_annotations.bed.gz > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/raw/hg38_annotations.bed` (331792)
Command completed. Elapsed time: 0:00:00. Running peak memory: 0.204GB. PID: 331792; Command: pigz; Return code: 0; Memory used: 0.002GB ### Calculate cumulative and terminal fraction of reads in features (cFRiF/FRiF) (06-14 21:19:04) elapsed: 16.0 _TIME_ > `cut -f 4 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/raw/hg38_annotations.bed | sort -u` Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/Enhancer` > `awk -F' ' '{print>"/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/"$4}' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/raw/hg38_annotations.bed` (331800)
Command completed. Elapsed time: 0:00:01. Running peak memory: 0.204GB. PID: 331800; Command: awk; Return code: 0; Memory used: 0.002GB Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/Enhancer_sort.bed` > `cut -f 1 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/chr_order.txt | grep -wf - /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/Enhancer | cut -f 1-3 | bedtools sort -i stdin -faidx /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/Enhancer_sort.bed` (331810,331811,331812,331813)
Command completed. Elapsed time: 0:00:01. Running peak memory: 0.204GB. PID: 331810; Command: cut; Return code: 0; Memory used: 0.0GB PID: 331811; Command: grep; Return code: 0; Memory used: 0.002GB PID: 331813; Command: bedtools; Return code: 0; Memory used: 0.012GB PID: 331812; Command: cut; Return code: 0; Memory used: 0.001GB Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_Enhancer_plus_coverage.bed` > `bedtools coverage -sorted -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/Enhancer_sort.bed -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_plus.bam -g /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_Enhancer_plus_coverage.bed` (331818)
Command completed. Elapsed time: 0:00:03. Running peak memory: 0.204GB. PID: 331818; Command: bedtools; Return code: 0; Memory used: 0.007GB Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_Enhancer_minus_coverage.bed` > `bedtools coverage -sorted -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/Enhancer_sort.bed -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_minus.bam -g /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_Enhancer_minus_coverage.bed` (331830)
Command completed. Elapsed time: 0:00:03. Running peak memory: 0.204GB. PID: 331830; Command: bedtools; Return code: 0; Memory used: 0.007GB Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/Promoter` Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/Promoter_sort.bed` > `cut -f 1 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/chr_order.txt | grep -wf - /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/Promoter | cut -f 1-3 | bedtools sort -i stdin -faidx /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/Promoter_sort.bed` (331846,331847,331848,331849)
Command completed. Elapsed time: 0:00:00. Running peak memory: 0.204GB. PID: 331846; Command: cut; Return code: 0; Memory used: 0.0GB PID: 331847; Command: grep; Return code: 0; Memory used: 0.003GB PID: 331849; Command: bedtools; Return code: 0; Memory used: 0.009GB PID: 331848; Command: cut; Return code: 0; Memory used: 0.001GB Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_Promoter_plus_coverage.bed` > `bedtools coverage -sorted -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/Promoter_sort.bed -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_plus.bam -g /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_Promoter_plus_coverage.bed` (331851)
Command completed. Elapsed time: 0:00:03. Running peak memory: 0.204GB. PID: 331851; Command: bedtools; Return code: 0; Memory used: 0.009GB Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_Promoter_minus_coverage.bed` > `bedtools coverage -sorted -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/Promoter_sort.bed -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_minus.bam -g /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_Promoter_minus_coverage.bed` (331865)
Command completed. Elapsed time: 0:00:03. Running peak memory: 0.204GB. PID: 331865; Command: bedtools; Return code: 0; Memory used: 0.008GB Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/Promoter Flanking Region` Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/Promoter_Flanking_Region` > `mv "/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/Promoter Flanking Region" "/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/Promoter_Flanking_Region"` (331878)
Command completed. Elapsed time: 0:00:00. Running peak memory: 0.204GB. PID: 331878; Command: mv; Return code: 0; Memory used: 0.0GB Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/Promoter_Flanking_Region_sort.bed` > `cut -f 1 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/chr_order.txt | grep -wf - /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/Promoter_Flanking_Region | cut -f 1-3 | bedtools sort -i stdin -faidx /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/Promoter_Flanking_Region_sort.bed` (331879,331880,331881,331882)
Command completed. Elapsed time: 0:00:01. Running peak memory: 0.204GB. PID: 331879; Command: cut; Return code: 0; Memory used: 0.0GB PID: 331881; Command: cut; Return code: 0; Memory used: 0.001GB PID: 331880; Command: grep; Return code: 0; Memory used: 0.002GB PID: 331882; Command: bedtools; Return code: 0; Memory used: 0.012GB Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_Promoter_Flanking_Region_plus_coverage.bed` > `bedtools coverage -sorted -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/Promoter_Flanking_Region_sort.bed -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_plus.bam -g /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_Promoter_Flanking_Region_plus_coverage.bed` (331886)
Command completed. Elapsed time: 0:00:03. Running peak memory: 0.204GB. PID: 331886; Command: bedtools; Return code: 0; Memory used: 0.007GB Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_Promoter_Flanking_Region_minus_coverage.bed` > `bedtools coverage -sorted -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/Promoter_Flanking_Region_sort.bed -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_minus.bam -g /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_Promoter_Flanking_Region_minus_coverage.bed` (331896)
Command completed. Elapsed time: 0:00:03. Running peak memory: 0.204GB. PID: 331896; Command: bedtools; Return code: 0; Memory used: 0.007GB Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/5' UTR` Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/5_UTR` > `mv "/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/5' UTR" "/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/5_UTR"` (331902)
Command completed. Elapsed time: 0:00:00. Running peak memory: 0.204GB. PID: 331902; Command: mv; Return code: 0; Memory used: 0.0GB Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/5_UTR_sort.bed` > `cut -f 1 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/chr_order.txt | grep -wf - /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/5_UTR | cut -f 1-3 | bedtools sort -i stdin -faidx /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/5_UTR_sort.bed` (331903,331904,331905,331906)
Command completed. Elapsed time: 0:00:01. Running peak memory: 0.204GB. PID: 331903; Command: cut; Return code: 0; Memory used: 0.0GB PID: 331904; Command: grep; Return code: 0; Memory used: 0.003GB PID: 331906; Command: bedtools; Return code: 0; Memory used: 0.011GB PID: 331905; Command: cut; Return code: 0; Memory used: 0.001GB Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_5_UTR_plus_coverage.bed` > `bedtools coverage -sorted -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/5_UTR_sort.bed -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_plus.bam -g /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_5_UTR_plus_coverage.bed` (331912)
Command completed. Elapsed time: 0:00:03. Running peak memory: 0.204GB. PID: 331912; Command: bedtools; Return code: 0; Memory used: 0.007GB Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_5_UTR_minus_coverage.bed` > `bedtools coverage -sorted -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/5_UTR_sort.bed -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_minus.bam -g /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_5_UTR_minus_coverage.bed` (331923)
Command completed. Elapsed time: 0:00:03. Running peak memory: 0.204GB. PID: 331923; Command: bedtools; Return code: 0; Memory used: 0.006GB Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/3' UTR` Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/3_UTR` > `mv "/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/3' UTR" "/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/3_UTR"` (331931)
Command completed. Elapsed time: 0:00:00. Running peak memory: 0.204GB. PID: 331931; Command: mv; Return code: 0; Memory used: 0.0GB Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/3_UTR_sort.bed` > `cut -f 1 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/chr_order.txt | grep -wf - /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/3_UTR | cut -f 1-3 | bedtools sort -i stdin -faidx /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/3_UTR_sort.bed` (331932,331933,331934,331935)
Command completed. Elapsed time: 0:00:01. Running peak memory: 0.204GB. PID: 331932; Command: cut; Return code: 0; Memory used: 0.0GB PID: 331933; Command: grep; Return code: 0; Memory used: 0.003GB PID: 331935; Command: bedtools; Return code: 0; Memory used: 0.034GB PID: 331934; Command: cut; Return code: 0; Memory used: 0.001GB Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_3_UTR_plus_coverage.bed` > `bedtools coverage -sorted -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/3_UTR_sort.bed -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_plus.bam -g /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_3_UTR_plus_coverage.bed` (331939)
Command completed. Elapsed time: 0:00:03. Running peak memory: 0.204GB. PID: 331939; Command: bedtools; Return code: 0; Memory used: 0.007GB Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_3_UTR_minus_coverage.bed` > `bedtools coverage -sorted -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/3_UTR_sort.bed -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_minus.bam -g /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_3_UTR_minus_coverage.bed` (331949)
Command completed. Elapsed time: 0:00:03. Running peak memory: 0.204GB. PID: 331949; Command: bedtools; Return code: 0; Memory used: 0.007GB Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/Exon` Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/Exon_sort.bed` > `cut -f 1 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/chr_order.txt | grep -wf - /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/Exon | cut -f 1-3 | bedtools sort -i stdin -faidx /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/Exon_sort.bed` (331971,331972,331973,331974)
Command completed. Elapsed time: 0:00:03. Running peak memory: 0.204GB. PID: 331971; Command: cut; Return code: 0; Memory used: 0.0GB PID: 331972; Command: grep; Return code: 0; Memory used: 0.004GB PID: 331974; Command: bedtools; Return code: 0; Memory used: 0.158GB PID: 331973; Command: cut; Return code: 0; Memory used: 0.001GB Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_Exon_plus_coverage.bed` > `bedtools coverage -sorted -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/Exon_sort.bed -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_plus.bam -g /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_Exon_plus_coverage.bed` (331979)
Command completed. Elapsed time: 0:00:04. Running peak memory: 0.204GB. PID: 331979; Command: bedtools; Return code: 0; Memory used: 0.008GB Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_Exon_minus_coverage.bed` > `bedtools coverage -sorted -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/Exon_sort.bed -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_minus.bam -g /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_Exon_minus_coverage.bed` (331984)
Command completed. Elapsed time: 0:00:04. Running peak memory: 0.204GB. PID: 331984; Command: bedtools; Return code: 0; Memory used: 0.008GB Target exists: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/Intron` Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/Intron_sort.bed` > `cut -f 1 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/chr_order.txt | grep -wf - /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/Intron | cut -f 1-3 | bedtools sort -i stdin -faidx /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/Intron_sort.bed` (331989,331990,331991,331992)
Command completed. Elapsed time: 0:00:01. Running peak memory: 0.204GB. PID: 331989; Command: cut; Return code: 0; Memory used: 0.0GB PID: 331991; Command: cut; Return code: 0; Memory used: 0.001GB PID: 331990; Command: grep; Return code: 0; Memory used: 0.002GB PID: 331992; Command: bedtools; Return code: 0; Memory used: 0.077GB Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_Intron_plus_coverage.bed` > `bedtools coverage -sorted -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/Intron_sort.bed -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_plus.bam -g /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_Intron_plus_coverage.bed` (331995)
Command completed. Elapsed time: 0:00:05. Running peak memory: 0.204GB. PID: 331995; Command: bedtools; Return code: 0; Memory used: 0.03GB Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_Intron_minus_coverage.bed` > `bedtools coverage -sorted -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/Intron_sort.bed -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_minus.bam -g /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_Intron_minus_coverage.bed` (332002)
Command completed. Elapsed time: 0:00:04. Running peak memory: 0.204GB. PID: 332002; Command: bedtools; Return code: 0; Memory used: 0.02GB ### Plot cFRiF/FRiF (06-14 21:19:58) elapsed: 55.0 _TIME_ > `samtools view -@ 4 -q 10 -c -F4 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_plus.bam` Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_cFRiF.pdf` > `Rscript /scratch/jps3dp/tools/databio//peppro/tools/PEPPRO.R frif -s H9_PRO-seq_20 -z 3099922541 -n 1990532 -y cfrif --reads -o /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_cFRiF.pdf --bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_Enhancer_plus_coverage.bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_Promoter_plus_coverage.bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_Promoter_Flanking_Region_plus_coverage.bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_5_UTR_plus_coverage.bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_3_UTR_plus_coverage.bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_Exon_plus_coverage.bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_Intron_plus_coverage.bed` (332013)
Cumulative cfrif plot completed!Command completed. Elapsed time: 0:00:35. Running peak memory: 0.443GB. PID: 332013; Command: Rscript; Return code: 0; Memory used: 0.443GB > `cFRiF` QC_hg38/H9_PRO-seq_20_cFRiF.pdf cFRiF QC_hg38/H9_PRO-seq_20_cFRiF.png PEPPRO _OBJ_ Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_FRiF.pdf` > `Rscript /scratch/jps3dp/tools/databio//peppro/tools/PEPPRO.R frif -s H9_PRO-seq_20 -z 3099922541 -n 1990532 -y frif --reads -o /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_FRiF.pdf --bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_Enhancer_plus_coverage.bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_Promoter_plus_coverage.bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_Promoter_Flanking_Region_plus_coverage.bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_5_UTR_plus_coverage.bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_3_UTR_plus_coverage.bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_Exon_plus_coverage.bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_Intron_plus_coverage.bed` (332332)
Cumulative frif plot completed!Command completed. Elapsed time: 0:00:28. Running peak memory: 0.443GB. PID: 332332; Command: Rscript; Return code: 0; Memory used: 0.443GB > `FRiF` QC_hg38/H9_PRO-seq_20_FRiF.pdf FRiF QC_hg38/H9_PRO-seq_20_FRiF.png PEPPRO _OBJ_ ### Calculate mRNA contamination (06-14 21:21:02) elapsed: 64.0 _TIME_ Missing stat 'mRNA_contamination' Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/hg38_exons_sort.bed`,`/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/hg38_introns_sort.bed` > `grep -wf /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/chr_keep.txt /project/shefflab/genomes/hg38/refgene_anno/default/hg38_exons.bed | bedtools sort -i stdin -faidx /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/hg38_exons_sort.bed` (332406,332407)
Command completed. Elapsed time: 0:00:05. Running peak memory: 0.443GB. PID: 332407; Command: bedtools; Return code: 0; Memory used: 0.087GB PID: 332406; Command: grep; Return code: 0; Memory used: 0.004GB > `grep -wf /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/chr_keep.txt /project/shefflab/genomes/hg38/refgene_anno/default/hg38_introns.bed | bedtools sort -i stdin -faidx /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/chr_order.txt | bedtools sort -i stdin -faidx /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/hg38_introns_sort.bed` (332428,332429,332430)
Command completed. Elapsed time: 0:00:05. Running peak memory: 0.443GB. PID: 332428; Command: grep; Return code: 0; Memory used: 0.005GB PID: 332430; Command: bedtools; Return code: 0; Memory used: 0.007GB PID: 332429; Command: bedtools; Return code: 0; Memory used: 0.033GB Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_exons_coverage.bed`,`/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_introns_coverage.bed` > `bedtools coverage -sorted -counts -s -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/hg38_exons_sort.bed -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_PE1.bam -g /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_exons_coverage.bed` (332448)
Command completed. Elapsed time: 0:00:05. Running peak memory: 0.443GB. PID: 332448; Command: bedtools; Return code: 0; Memory used: 0.009GB > `bedtools coverage -sorted -counts -s -a /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/hg38_introns_sort.bed -b /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_PE1.bam -g /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/chr_order.txt > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_introns_coverage.bed` (332455)
Command completed. Elapsed time: 0:00:07. Running peak memory: 0.443GB. PID: 332455; Command: bedtools; Return code: 0; Memory used: 0.029GB Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_exons_rpkm.bed` > `awk -v OFS=' ' '{chrom[$4] = $1; if($4!=prev4) {chromStart[$4] = $2} strand[$4] = $6; readCount[$4] += $7; exonCount[$4] += 1; geneSizeKB[$4] += (sqrt(($3-$2+0.00000001)^2)/1000); gene[$4] = $4; chromEnd[$4]=$3; prev4=$4} END { for (a in readCount) { print chrom[a], chromStart[a], chromEnd[a], gene[a], (readCount[a]/3.6333635)/geneSizeKB[a], strand[a]}}' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_exons_coverage.bed | awk '$5>0' | sort -k4 > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_exons_rpkm.bed` (332462,332463,332464)
Command completed. Elapsed time: 0:00:01. Running peak memory: 0.443GB. PID: 332462; Command: awk; Return code: 0; Memory used: 0.009GB PID: 332464; Command: sort; Return code: 0; Memory used: 0.004GB PID: 332463; Command: awk; Return code: 0; Memory used: 0.001GB Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_introns_rpkm.bed` > `awk -v OFS=' ' '{chrom[$4] = $1; if($4!=prev4) {chromStart[$4] = $2} strand[$4] = $6; readCount[$4] += $7; exonCount[$4] += 1; geneSizeKB[$4] += (sqrt(($3-$2+0.00000001)^2)/1000); gene[$4] = $4; chromEnd[$4]=$3; prev4=$4} END { for (a in readCount) { print chrom[a], chromStart[a], chromEnd[a], gene[a], (readCount[a]/3.6333635)/geneSizeKB[a], strand[a]}}' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_introns_coverage.bed | awk '$5>0' | sort -k4 > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_introns_rpkm.bed` (332467,332468,332469)
Command completed. Elapsed time: 0:00:01. Running peak memory: 0.443GB. PID: 332467; Command: awk; Return code: 0; Memory used: 0.009GB PID: 332469; Command: sort; Return code: 0; Memory used: 0.004GB PID: 332468; Command: awk; Return code: 0; Memory used: 0.001GB Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_exon_intron_ratios.bed` > `join --nocheck-order -a1 -a2 -j4 /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_introns_rpkm.bed /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_exons_rpkm.bed | awk -v OFS=' ' 'NF==11 {print $7, $8, $9, $1, ($10/$5), $11}' | sort -k1,1 -k2,2n > /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_exon_intron_ratios.bed` (332472,332473,332474)
Command completed. Elapsed time: 0:00:00. Running peak memory: 0.443GB. PID: 332472; Command: join; Return code: 0; Memory used: 0.001GB PID: 332474; Command: sort; Return code: 0; Memory used: 0.003GB PID: 332473; Command: awk; Return code: 0; Memory used: 0.001GB > `awk '{print $5}' /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_exon_intron_ratios.bed | sort -n | awk ' { a[i++]=$1; } END { x=int((i+1)/2); if (x < (i+1)/2) print (a[x-1]+a[x])/2; else print a[x-1]; }'` > `mRNA_contamination` 1.31 PEPPRO _RES_ Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_mRNA_contamination.pdf` > `Rscript /scratch/jps3dp/tools/databio//peppro/tools/PEPPRO.R mrna -i /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_exon_intron_ratios.bed --annotate` (332480)
mRNA contamination plot completed!Command completed. Elapsed time: 0:00:05. Running peak memory: 0.443GB. PID: 332480; Command: Rscript; Return code: 0; Memory used: 0.316GB > `mRNA contamination` QC_hg38/H9_PRO-seq_20_mRNA_contamination.pdf mRNA contamination QC_hg38/H9_PRO-seq_20_mRNA_contamination.png PEPPRO _OBJ_ Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_exon_intron_ratios.bed.gz` > `pigz -f -p 4 -f /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/QC_hg38/H9_PRO-seq_20_exon_intron_ratios.bed` (332501)
Command completed. Elapsed time: 0:00:00. Running peak memory: 0.443GB. PID: 332501; Command: pigz; Return code: 0; Memory used: 0.002GB ### Produce bigWig files (06-14 21:21:31) elapsed: 29.0 _TIME_ Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/signal_hg38/H9_PRO-seq_20_plus_exact_body_0-mer.bw`,`/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/signal_hg38/H9_PRO-seq_20_plus_smooth_body_0-mer.bw` > `samtools index /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_plus.bam` (332509)
Command completed. Elapsed time: 0:00:02. Running peak memory: 0.443GB. PID: 332509; Command: samtools; Return code: 0; Memory used: 0.007GB > `/scratch/jps3dp/tools/databio//peppro/tools/bamSitesToWig.py -i /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_plus.bam -c /project/shefflab/genomes/hg38/fasta/default/hg38.chrom.sizes -o /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/signal_hg38/H9_PRO-seq_20_plus_exact_body_0-mer.bw -w /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/signal_hg38/H9_PRO-seq_20_plus_smooth_body_0-mer.bw -p 2 --variable-step --tail-edge --scale 3633363.5` (332515)
Cutting parallel chroms in half to accommodate two tracks. Registering input file: '/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_plus.bam' Temporary files will be stored in: 'tmp_H9_PRO-seq_20_plus_cuttrace_el5rfk2l' Processing with 1 cores... Discarding 132 chunk(s) of reads: ['chrM', 'chr1_KI270707v1_random', 'chr1_KI270708v1_random', 'chr1_KI270710v1_random', 'chr1_KI270714v1_random', 'chr2_KI270715v1_random', 'chr2_KI270716v1_random', 'chr3_GL000221v1_random', 'chr4_GL000008v2_random', 'chr5_GL000208v1_random', 'chr9_KI270717v1_random', 'chr9_KI270718v1_random', 'chr9_KI270720v1_random', 'chr14_GL000009v2_random', 'chr14_KI270724v1_random', 'chr14_KI270725v1_random', 'chr15_KI270727v1_random', 'chr22_KI270731v1_random', 'chr22_KI270732v1_random', 'chr22_KI270735v1_random', 'chr22_KI270737v1_random', 'chr22_KI270738v1_random', 'chr22_KI270739v1_random', 'chrY_KI270740v1_random', 'chrUn_KI270302v1', 'chrUn_KI270304v1', 'chrUn_KI270303v1', 'chrUn_KI270305v1', 'chrUn_KI270322v1', 'chrUn_KI270320v1', 'chrUn_KI270310v1', 'chrUn_KI270316v1', 'chrUn_KI270315v1', 'chrUn_KI270312v1', 'chrUn_KI270311v1', 'chrUn_KI270317v1', 'chrUn_KI270412v1', 'chrUn_KI270411v1', 'chrUn_KI270414v1', 'chrUn_KI270419v1', 'chrUn_KI270418v1', 'chrUn_KI270420v1', 'chrUn_KI270424v1', 'chrUn_KI270417v1', 'chrUn_KI270422v1', 'chrUn_KI270423v1', 'chrUn_KI270425v1', 'chrUn_KI270429v1', 'chrUn_KI270466v1', 'chrUn_KI270465v1', 'chrUn_KI270467v1', 'chrUn_KI270468v1', 'chrUn_KI270510v1', 'chrUn_KI270509v1', 'chrUn_KI270518v1', 'chrUn_KI270508v1', 'chrUn_KI270516v1', 'chrUn_KI270512v1', 'chrUn_KI270522v1', 'chrUn_KI270511v1', 'chrUn_KI270515v1', 'chrUn_KI270507v1', 'chrUn_KI270517v1', 'chrUn_KI270529v1', 'chrUn_KI270528v1', 'chrUn_KI270530v1', 'chrUn_KI270538v1', 'chrUn_KI270544v1', 'chrUn_KI270548v1', 'chrUn_KI270583v1', 'chrUn_KI270587v1', 'chrUn_KI270580v1', 'chrUn_KI270581v1', 'chrUn_KI270579v1', 'chrUn_KI270589v1', 'chrUn_KI270590v1', 'chrUn_KI270584v1', 'chrUn_KI270582v1', 'chrUn_KI270588v1', 'chrUn_KI270593v1', 'chrUn_KI270591v1', 'chrUn_KI270330v1', 'chrUn_KI270329v1', 'chrUn_KI270334v1', 'chrUn_KI270333v1', 'chrUn_KI270335v1', 'chrUn_KI270338v1', 'chrUn_KI270340v1', 'chrUn_KI270336v1', 'chrUn_KI270337v1', 'chrUn_KI270363v1', 'chrUn_KI270364v1', 'chrUn_KI270362v1', 'chrUn_KI270366v1', 'chrUn_KI270378v1', 'chrUn_KI270379v1', 'chrUn_KI270389v1', 'chrUn_KI270390v1', 'chrUn_KI270387v1', 'chrUn_KI270395v1', 'chrUn_KI270396v1', 'chrUn_KI270388v1', 'chrUn_KI270394v1', 'chrUn_KI270386v1', 'chrUn_KI270391v1', 'chrUn_KI270383v1', 'chrUn_KI270393v1', 'chrUn_KI270384v1', 'chrUn_KI270392v1', 'chrUn_KI270381v1', 'chrUn_KI270385v1', 'chrUn_KI270382v1', 'chrUn_KI270376v1', 'chrUn_KI270374v1', 'chrUn_KI270372v1', 'chrUn_KI270373v1', 'chrUn_KI270375v1', 'chrUn_KI270371v1', 'chrUn_KI270448v1', 'chrUn_KI270521v1', 'chrUn_GL000226v1', 'chrUn_KI270744v1', 'chrUn_KI270745v1', 'chrUn_KI270746v1', 'chrUn_KI270749v1', 'chrUn_KI270752v1', 'chrUn_KI270753v1', 'chrUn_KI270754v1', 'chrUn_KI270755v1', 'chrUn_KI270756v1', 'chrUn_KI270757v1', 'chrUn_GL000218v1'] Keeping 63 chunk(s) of reads: ['chr1', 'chr2', 'chr3', 'chr4', 'chr5', 'chr6', 'chr7', 'chr8', 'chr9', 'chr10', 'chr11', 'chr12', 'chr13', 'chr14', 'chr15', 'chr16', 'chr17', 'chr18', 'chr19', 'chr20', 'chr21', 'chr22', 'chrX', 'chrY', 'chr1_KI270706v1_random', 'chr1_KI270709v1_random', 'chr1_KI270711v1_random', 'chr1_KI270712v1_random', 'chr1_KI270713v1_random', 'chr9_KI270719v1_random', 'chr11_KI270721v1_random', 'chr14_GL000225v1_random', 'chr14_KI270722v1_random', 'chr14_GL000194v1_random', 'chr14_KI270723v1_random', 'chr14_KI270726v1_random', 'chr16_KI270728v1_random', 'chr17_GL000205v2_random', 'chr17_KI270729v1_random', 'chr17_KI270730v1_random', 'chr22_KI270733v1_random', 'chr22_KI270734v1_random', 'chr22_KI270736v1_random', 'chrUn_KI270442v1', 'chrUn_KI270435v1', 'chrUn_KI270438v1', 'chrUn_KI270519v1', 'chrUn_KI270539v1', 'chrUn_GL000195v1', 'chrUn_GL000219v1', 'chrUn_GL000220v1', 'chrUn_GL000224v1', 'chrUn_KI270741v1', 'chrUn_GL000213v1', 'chrUn_KI270743v1', 'chrUn_KI270747v1', 'chrUn_KI270748v1', 'chrUn_KI270750v1', 'chrUn_KI270751v1', 'chrUn_GL000214v1', 'chrUn_KI270742v1', 'chrUn_GL000216v2', 'chrEBV'] Reduce step (merge files)... Merging 63 files into output file: '/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/signal_hg38/H9_PRO-seq_20_plus_exact_body_0-mer.bw' Merging 63 files into output file: '/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/signal_hg38/H9_PRO-seq_20_plus_smooth_body_0-mer.bw'Command completed. Elapsed time: 0:16:49. Running peak memory: 1.33GB. PID: 332515; Command: /scratch/jps3dp/tools/databio//peppro/tools/bamSitesToWig.py; Return code: 0; Memory used: 1.33GB Target to produce: `/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/signal_hg38/H9_PRO-seq_20_minus_exact_body_0-mer.bw`,`/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/signal_hg38/H9_PRO-seq_20_minus_smooth_body_0-mer.bw` > `samtools index /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_minus.bam` (135538)
Command completed. Elapsed time: 0:00:02. Running peak memory: 1.33GB. PID: 135538; Command: samtools; Return code: 0; Memory used: 0.007GB > `/scratch/jps3dp/tools/databio//peppro/tools/bamSitesToWig.py -i /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_minus.bam -c /project/shefflab/genomes/hg38/fasta/default/hg38.chrom.sizes -o /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/signal_hg38/H9_PRO-seq_20_minus_exact_body_0-mer.bw -w /project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/signal_hg38/H9_PRO-seq_20_minus_smooth_body_0-mer.bw -p 2 --variable-step --tail-edge --scale 3633363.5` (135541)
Cutting parallel chroms in half to accommodate two tracks. Registering input file: '/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/aligned_hg38/H9_PRO-seq_20_minus.bam' Temporary files will be stored in: 'tmp_H9_PRO-seq_20_minus_cuttrace_g_w9nf7w' Processing with 1 cores... stdin is empty of data Discarding 131 chunk(s) of reads: ['chrM', 'chr1_KI270706v1_random', 'chr1_KI270707v1_random', 'chr1_KI270708v1_random', 'chr1_KI270709v1_random', 'chr1_KI270710v1_random', 'chr1_KI270712v1_random', 'chr2_KI270715v1_random', 'chr2_KI270716v1_random', 'chr4_GL000008v2_random', 'chr9_KI270717v1_random', 'chr9_KI270719v1_random', 'chr9_KI270720v1_random', 'chr11_KI270721v1_random', 'chr14_GL000009v2_random', 'chr14_KI270722v1_random', 'chr14_KI270723v1_random', 'chr14_KI270725v1_random', 'chr17_KI270729v1_random', 'chr17_KI270730v1_random', 'chr22_KI270731v1_random', 'chr22_KI270732v1_random', 'chr22_KI270735v1_random', 'chr22_KI270736v1_random', 'chr22_KI270738v1_random', 'chr22_KI270739v1_random', 'chrY_KI270740v1_random', 'chrUn_KI270302v1', 'chrUn_KI270304v1', 'chrUn_KI270303v1', 'chrUn_KI270305v1', 'chrUn_KI270322v1', 'chrUn_KI270320v1', 'chrUn_KI270310v1', 'chrUn_KI270316v1', 'chrUn_KI270315v1', 'chrUn_KI270312v1', 'chrUn_KI270311v1', 'chrUn_KI270317v1', 'chrUn_KI270412v1', 'chrUn_KI270411v1', 'chrUn_KI270414v1', 'chrUn_KI270419v1', 'chrUn_KI270418v1', 'chrUn_KI270420v1', 'chrUn_KI270424v1', 'chrUn_KI270417v1', 'chrUn_KI270422v1', 'chrUn_KI270423v1', 'chrUn_KI270425v1', 'chrUn_KI270429v1', 'chrUn_KI270466v1', 'chrUn_KI270465v1', 'chrUn_KI270467v1', 'chrUn_KI270435v1', 'chrUn_KI270468v1', 'chrUn_KI270510v1', 'chrUn_KI270509v1', 'chrUn_KI270518v1', 'chrUn_KI270508v1', 'chrUn_KI270516v1', 'chrUn_KI270512v1', 'chrUn_KI270519v1', 'chrUn_KI270522v1', 'chrUn_KI270511v1', 'chrUn_KI270515v1', 'chrUn_KI270507v1', 'chrUn_KI270517v1', 'chrUn_KI270529v1', 'chrUn_KI270528v1', 'chrUn_KI270530v1', 'chrUn_KI270539v1', 'chrUn_KI270538v1', 'chrUn_KI270544v1', 'chrUn_KI270548v1', 'chrUn_KI270583v1', 'chrUn_KI270587v1', 'chrUn_KI270580v1', 'chrUn_KI270581v1', 'chrUn_KI270579v1', 'chrUn_KI270590v1', 'chrUn_KI270584v1', 'chrUn_KI270582v1', 'chrUn_KI270588v1', 'chrUn_KI270591v1', 'chrUn_KI270330v1', 'chrUn_KI270329v1', 'chrUn_KI270334v1', 'chrUn_KI270335v1', 'chrUn_KI270338v1', 'chrUn_KI270340v1', 'chrUn_KI270336v1', 'chrUn_KI270363v1', 'chrUn_KI270364v1', 'chrUn_KI270362v1', 'chrUn_KI270366v1', 'chrUn_KI270378v1', 'chrUn_KI270379v1', 'chrUn_KI270389v1', 'chrUn_KI270390v1', 'chrUn_KI270387v1', 'chrUn_KI270395v1', 'chrUn_KI270396v1', 'chrUn_KI270388v1', 'chrUn_KI270394v1', 'chrUn_KI270386v1', 'chrUn_KI270391v1', 'chrUn_KI270383v1', 'chrUn_KI270393v1', 'chrUn_KI270384v1', 'chrUn_KI270392v1', 'chrUn_KI270381v1', 'chrUn_KI270385v1', 'chrUn_KI270382v1', 'chrUn_KI270376v1', 'chrUn_KI270374v1', 'chrUn_KI270372v1', 'chrUn_KI270373v1', 'chrUn_KI270375v1', 'chrUn_KI270371v1', 'chrUn_KI270448v1', 'chrUn_KI270521v1', 'chrUn_GL000226v1', 'chrUn_GL000213v1', 'chrUn_KI270744v1', 'chrUn_KI270745v1', 'chrUn_KI270748v1', 'chrUn_KI270751v1', 'chrUn_KI270754v1', 'chrUn_KI270755v1', 'chrUn_GL000214v1'] Keeping 64 chunk(s) of reads: ['chr1', 'chr2', 'chr3', 'chr4', 'chr5', 'chr6', 'chr7', 'chr8', 'chr9', 'chr10', 'chr11', 'chr12', 'chr13', 'chr14', 'chr15', 'chr16', 'chr17', 'chr18', 'chr19', 'chr20', 'chr21', 'chr22', 'chrX', 'chrY', 'chr1_KI270711v1_random', 'chr1_KI270713v1_random', 'chr1_KI270714v1_random', 'chr3_GL000221v1_random', 'chr5_GL000208v1_random', 'chr9_KI270718v1_random', 'chr14_GL000225v1_random', 'chr14_GL000194v1_random', 'chr14_KI270724v1_random', 'chr14_KI270726v1_random', 'chr15_KI270727v1_random', 'chr16_KI270728v1_random', 'chr17_GL000205v2_random', 'chr22_KI270733v1_random', 'chr22_KI270734v1_random', 'chr22_KI270737v1_random', 'chrUn_KI270442v1', 'chrUn_KI270438v1', 'chrUn_KI270589v1', 'chrUn_KI270593v1', 'chrUn_KI270333v1', 'chrUn_KI270337v1', 'chrUn_GL000195v1', 'chrUn_GL000219v1', 'chrUn_GL000220v1', 'chrUn_GL000224v1', 'chrUn_KI270741v1', 'chrUn_KI270743v1', 'chrUn_KI270746v1', 'chrUn_KI270747v1', 'chrUn_KI270749v1', 'chrUn_KI270750v1', 'chrUn_KI270752v1', 'chrUn_KI270753v1', 'chrUn_KI270756v1', 'chrUn_KI270757v1', 'chrUn_KI270742v1', 'chrUn_GL000216v2', 'chrUn_GL000218v1', 'chrEBV'] Reduce step (merge files)... Merging 64 files into output file: '/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/signal_hg38/H9_PRO-seq_20_minus_exact_body_0-mer.bw' Merging 64 files into output file: '/project/shefflab/processed/peppro/paper/6.11.2020/results_pipeline/H9_PRO-seq_20/signal_hg38/H9_PRO-seq_20_minus_smooth_body_0-mer.bw'Command completed. Elapsed time: 0:16:35. Running peak memory: 1.33GB. PID: 135541; Command: /scratch/jps3dp/tools/databio//peppro/tools/bamSitesToWig.py; Return code: 0; Memory used: 1.281GB ### Pipeline completed. Epilogue * Elapsed time (this run): 0:36:25 * Total elapsed time (all runs): 4:44:32 * Peak memory (this run): 1.3301 GB * Pipeline completed time: 2020-06-14 21:54:59