Submitting
Sequenced BACs

Robert Buels

SGN

Purpose and Scope

Provide well-organized archive of sequencing data for the tomato genome sequencing project, including:

finished BAC sequences

chromatograms

assembly files

annotations

BAC Submission Guidelines

A comprehensive BAC submission format exists (see handout, pp. 4-5).

If we all follow this format, the data will be much easier to work with.

Problems

Not all submissions follow the submission guidelines.

Irregularities in naming, directory structure, submission file contents.

FTP site does not separate finished and unfinished sequences.

Result: data is harder to work with than it should be!

Problems

To address these problems:

We improved the directory structure of the FTP site.

Corrected existing submissions to follow submission guidelines.

We made a new submission system that checks submission guidelines.

New FTP Structure

tomato_genome/bacs/

chr01/

chr02/

...

chr12/

bacs.seq

finished_bacs.seq

README

validate_submission.pl

New FTP Structure

tomato_genome/bacs/

chr01/

chr02/

...

chr12/

bacs.seq

finished_bacs.seq

README

validate_submission.pl

all seqs

New FTP Structure

tomato_genome/bacs/

chr01/

chr02/

...

chr12/

bacs.seq

finished_bacs.seq

README

validate_submission.pl

all seqs

finished seqs

New FTP Structure

tomato_genome/bacs/

chr01/

chr02/

...

chr12/

bacs.seq

finished_bacs.seq

README

validate_submission.pl

all seqs

finished seqs

validation script

New FTP Structure

tomato_genome/bacs/

chr01/

unfinished/

finished/

C01HBa0088L02.all.xml

C01HBa0088L02.seq

C01HBa0088L02.tar.gz

C01HBa0216G16.all.xml

C01HBa0216G16.seq

C01HBa0216G16.tar.gz

...

New FTP Structure

tomato_genome/bacs/

chr01/

unfinished/

finished/

C01HBa0088L02.all.xml

C01HBa0088L02.seq

C01HBa0088L02.tar.gz

C01HBa0216G16.all.xml

C01HBa0216G16.seq

C01HBa0216G16.tar.gz

...

separate finished and unfinished

New FTP Structure

tomato_genome/bacs/

chr01/

unfinished/

finished/

C01HBa0088L02.all.xml

C01HBa0088L02.seq

C01HBa0088L02.tar.gz

C01HBa0216G16.all.xml

C01HBa0216G16.seq

C01HBa0216G16.tar.gz

...

separate finished and unfinished

submission

New FTP Structure

tomato_genome/bacs/

chr01/

unfinished/

finished/

C01HBa0088L02.all.xml

C01HBa0088L02.seq

C01HBa0088L02.tar.gz

C01HBa0216G16.all.xml

C01HBa0216G16.seq

C01HBa0216G16.tar.gz

...

separate finished and unfinished

submission

sequence

New FTP Structure

tomato_genome/bacs/

chr01/

unfinished/

finished/

C01HBa0088L02.all.xml

C01HBa0088L02.seq

C01HBa0088L02.tar.gz

C01HBa0216G16.all.xml

C01HBa0216G16.seq

C01HBa0216G16.tar.gz

...

separate finished and unfinished

annotations

submission

sequence

New Submission System

Developed to automatically check submission formatting, analyze submissions, and publish them on the FTP site.

Also improves turn-around time since it is mostly automatic.

New Submission System

checks submission guidelines

C01HBa0001A01.tar.gz

C01HBa0001A01/

chromat_dir/

edit_dir/

phd_dir/

seq_dir/

C01HBa0001A01.seq

>C01HBa0001A01

ATGCCTACGAT...

New Submission System

checks submission guidelines

C01HBa0001A01.tar.gz

C01HBa0001A01/

chromat_dir/

edit_dir/

phd_dir/

seq_dir/

C01HBa0001A01.seq

>C01HBa0001A01

ATGCCTACGAT...

submission is tar.gz file, named for BAC

New Submission System

checks submission guidelines

C01HBa0001A01.tar.gz

C01HBa0001A01/

chromat_dir/

edit_dir/

phd_dir/

seq_dir/

C01HBa0001A01.seq

>C01HBa0001A01

ATGCCTACGAT...

submission is tar.gz file, named for BAC

self-named subdirectory

New Submission System

checks submission guidelines

C01HBa0001A01.tar.gz

C01HBa0001A01/

chromat_dir/

edit_dir/

phd_dir/

seq_dir/

C01HBa0001A01.seq

>C01HBa0001A01

ATGCCTACGAT...

submission is tar.gz file, named for BAC

self-named subdirectory

self-named sequence file

New Submission System

checks submission guidelines

C01HBa0001A01.tar.gz

C01HBa0001A01/

chromat_dir/

edit_dir/

phd_dir/

seq_dir/

C01HBa0001A01.seq

>C01HBa0001A01

ATGCCTACGAT...

submission is tar.gz file, named for BAC

self-named subdirectory

self-named sequence file

self-named

sequence

validate_submission.pl

Checks the formatting of your submission.

Avoids having to retry uploads to correct formatting.

~$ ./validate_submission.pl C01HBa0001A01.tar.gz

C01HBa0088L02.tar.gz passed.

~$

On your machine...

Submission Process

Use validate_submission.pl to verify that your submission is properly formatted.

Upload with scp to upload.sgn.cornell.edu, using your login and password.

When finished, the submission cron job will notice it and process it.

Submission Process

If properly formatted, you will see it on the FTP site within 3 - 4 hours.

Otherwise, you will receive an email asking you to correct the format and retry the submission.

Questions?

Suggestions for improvement of FTP site?

Suggestions for submission process?

Robert Buels

2005-09-20T17:13:01