What SEED Do I Use?

March 17, 2011

Ross Overbeek

A number of distinct SEEDs have emerged over the years.  Almost all users will find they wish to use just the following two:

1.     The PubSEED will rapidly become the central SEED for use by the research community.  It will support access to the largest collection of genomes.  The constant influx of new genomes will be to the PubSEED first.  The PubSEED will support the ability for registered users to make annotations and subsystems (unfortunately, that implies that they will be able to overwrite the work of others, too).  We will support the ability for registered users to install genomes from RAST directly into the PubSEED.  Access and update capabilities, by genome, will require establishing a notion of ownership of genomes.  Users will be allowed to copy a genome creating a version with ownership rights that they control.  We will architect a few basic rules, and then we will do our best to develop tools that support reconciliation of conflicts, backup and recovery to specific points in time, and so forth.  This SEED will be the center of much of our work.

2.     The UC-SEED (i.e., the University of Chicago SEED) will be rebuilt periodically  as a copy of the PubSEED.  It is a place where classes can be held, students can do annotations and build subsystems, and so forth.  Subsystems built in the UC-SEED can be exported to the Clearinghouse, and then imported to the PubSEED.  This procedure allows users to save work and make it available, if they wish.  However, the whole system is rebuilt and the existing contents destroyed on a periodic basis (usually between semesters, and after several weeks in which there will be a posted notice on the front page).

Ownership of Genomes

I am now discussing a basic position relating to genomes that is not yet fully implemented.  I believe that it will move to the position I describe within 4-6 months.

There will soon be hundreds of thousands of genomes.  Many of these genomes will be either identical or almost identical.  In some cases the genomic sequence data will be identical, but distinct user groups will insist on the ability to annotate isolated copies that are protected from unauthorized updates.  Determination of a protocol that effectively supports both sharing and isolated annotation will require support for effectively managing privileges and interactions in a way that minimally constrains experts attempting to contribute.

It will rapidly become critical that we be able to talk about genomes, contigs, genes, and proteins and to easily detect whether two references are to the "same" entity.  As we move into a world with hundreds of thousands of genomes, some with identical sequence, and others with sequences that differ by only a few characters, it will become critical that we support basic ID Correspondence Services in a consistent manner.

We suggest employing the following set of definitions for what it means to be "the same" for genomes, contigs, genes, and proteins.

1.     Two sequences are the same if the MD5 functions of the uppercase versions of the sequences are identical.

2.     Two contigs are considered the same if their DNA sequences are the same. 

3.     Two genomes are considered the same iff

a.     They have the same number of contigs.

b.     The MD5 function of the sorted and concatenated contig MD5s match.  We call the MD5 function of the sorted and concatenated contigs the MD5 of the genome.

4.     Two genes are considered identical if they are in genomes that are the same, and

a.     They occur in contigs that are the same,

b.     They have identical start and stop positions in the two contigs. 

5.     Two proteins are the same if their sequences are the same (note that this is not a notion that is equivalent to saying that they are the gene products of two genes that are the same).

We will support the ability to rapidly determine which genomes, genes, and proteins are identical.  Further, we will support the capability of users defining sets of representative genomes and limiting displays to any selected set.

We are architecting the SEED environment as a framework that will be able to effectively integrate initially thousands, and within a short period millions, of distinct genomes.   Genomes will enter the collection from a growing number of sources.   Registerying a Genome will amount to claiming unique IDs for the genome and the features that occur within the genome.  This will inevitably lead to multiple registrations for identical genomes.  Further, while we will not support alteration of the sequence of a genome (i.e., such a change would lead to the creation and registration of a new genome), we will support addition and deletion of features on a genome.  A deletion will lead to recording a change in status (retaining a complete record of the deleted feature indefinitely).  The addition of a feature would require the acquisition on a new ID.  Changing the start location of a gene would cause deletion of the existing feature and addition of a new feature, which would inherit the appropriate attributes from the deleted feature.

The SEED environment will support the maintenance of genomes and features via a set of services that will include:

1.     acquire_a_genome_ID            returns a genome ID to a registered user

2.     acquire_a_feature_ID(Genome,Contig,Start,Stop)  returns a feature ID

3.     delete_a_feature(ID)               requires a update privileges

4.     reactivate a deleted genome(ID)    requires update privileges    

Registered users will be able to make any of these operations against genomes for which they have the required privileges.  Users owning genomes will have the ability to restrict access to a specified set of users.  That is, we will support private genomes that are not seen by everyone, and we will support the ability of owners to change the status of a genome (from private to public and vice versa).

Perhaps a short summary of the decision procedures on access/update rights would be as follows:

1.     We have registered users.  Users are either superusers or normal users.

2.     We have genomes.  Genomes are either private or public.

3.     Anyone attempting to access a genome or a feature of a genome will be given access if and only if

a.     the genome is public, or

b.     the user is a superuser, or

c.     the user either owns the genome or has been granted access to the genome.

4.     Anyone attempting to update a genome (which includes annotating features, deleting features, and adding features) will be allowed to make the update iff and only if

a.     the genome is public, or

b.     the user is a superuser, or

c.     the user either owns the genome or has been granted write privileges to the genome.

Setting Up an Annotation Group

If a group wishes to use the SEED Environment as a resource for supporting annotation and analysis of their genome, they would begin by registering each member of the group, and then establishing a group containing those members.

They would select the genomes they wish to annotate (probably by importing a newly-annotated genome from RAST into the PubSEED).  They would decide whether access and update privileges should be restricted to the group or not.

Then, they would use the framework we currently use to support our annotators to examine and edit annotations, construct metabolic models, or whatever.  The set of genomes that would be simultaneously be edited could all be public or all be private.  If private, they would be imported from RAST or as copies of existing genomes.

Access of Data Via the Servers

Most users of the SEED will use a web browser.  However, a growing body of users will also start using our SEED Servers, which support a well-defined API to access and update data from a SEED.  We will run servers for the PubSEED.  See

http://servers.nmpdr.org/servers

 for a discussion of the servers, the APIs used to access them, and the command-line services supported via the servers.  We believe that research groups may wish to use or help extend this growing confederation of servers.

Maintenance of Subsystems

The PubSEED, and UC-SEED both support development of subsystems.  From any of these platforms, subsystems can be exported to the Clearinghouse, and they can be imported into any other SEED (if you have the appropriate privileges).  We anticipate that students in classes would use the UC-SEED to avoid destroying the work of others.  Users wishing to make a more permanent contribution would use the PubSEED.

We will try to install a few basic rules to prevent bloodshed in instances in which incompatible annotations must be reconciled.  They would be something like

1.     You may overwrite any annotation that is not in a subsystem or is a duplicate in a subsystem (i.e., a case in which two genes currently have the same assigned functional role).

2.     Before overwriting a function in which a gene plays a unique role in someone else's subsystem, email them and ask for permission.  If they do not respond with a few days, proceed.

As the number of genomes grows rapidly, we believe that fewer and fewer annotators will actually construct and maintain comprehensive subsystems.  Rather, there will be a growing number of subsystems that contain only a subset of the actual genomes that have the machinery.  To handle this situation, we will periodically produce estimates, for each genome, of the subsystems that should contain the genome.  These will not impact any of the subsystems, but will allow users to have a reasonable estimate of the molecular machinery that can be identified.  This mimics what is now done in RAST, where a new genome contains estimates of which genes go into which subsystems, but these estimates do not actually impact the subsystems themselves.

The real point is that subsystems will no longer be thought of as comprehensive.  Up to this point the goal was to provide the tools needed to support manual curation of subsystems that were to contain as many genomes as possible from the existing collection.   The goal will shift.  We will think of subsystems as containing a diverse collection of instances needed to support accurate projection over the entire collection. The PubSEED will be used to house as complete a collection as possible, to support experimentation, and will inevitably lead to conflicts (that, hopefully, enrich the overall collection and get resolved peaceably).

An Exercise to Get a List of Homologs of Virulence Factors

Suppose that you have a file containing data relating to known virulence factors. For example, here are the first few lines of a file given to me by a participant in one of our tutorials (These columns are separated by tab characters):

VFG_id gi_id Gene_name Product Organism VF_id

VFG1293 21282773        hla     Alpha-Hemolysin precursor       Staphylococcus aureus MW2       VF0001

VFG1798 46588   hlb     beta-hemolysin  Staphylococcus aureus   VF0002

The second column contains GI numbers (without the usual "gi|" prefix).  The goal was to extract the set of FIGfams containing these genes, along with a list of the PEGs (and their functions).

We can do these easily in two steps. We'll use the "svr" command line tools in this example. If you are using Windows, be sure to remember to use the myRAST shell,  or, if you are using a Mac,  ensure that the svr commands are in your path with this command in your shell window:

export PATH=$PATH:/Applications/myRAST.app/bin

Our solution requires that the servers use the data from the PSEED, so be sure to run this command in your shell window:

Mac users:  export SAS_SERVER=PSEED

Windows Users: set SAS_SERVER=PSEED

 Now, to begin,  let's try

svr_ids_to_figfams -c=2 -source=NCBI < known.virulence.factors 2> no.matches | svr_figfams_to_ids > with.figfams

This produces two files.  The file "no.matches" contains a list of the lines from the input file ("known.virulence.factors") that have not yet been placed into FIGfams. The second file ("with.figfams") is a file in which two columns have been added: a FIGfam number and a PEG from that FIGfam.  Now, if we wished to add a function of the PEG, as well as a readable version of the genome it is from, we would use

svr_gene_data function genome-name < with.figfams > desired.table.txt

If the user needs to see the "aliases" for the PEGs that have been found, they might try

svr_aliases_of -c=9 < desired.table.txt > desired.table.aliases.txt

but be warned: we return all of the IDs of protein sequences that we know about that have the same protein sequence (from a potentially large collection of very similar genomes).

Finally, we should note that the initial virulence factors may have multiple entries leading to the same FIGfam (and, hence, to the same sets of PEGs).  This can lead to the situation  where multiple roles, while not identical, would have information on the same PEG.  To collapse the table to unique, you need to sort on the 9th field, assuming tab-delimited fields.

This can be done using

sort -k 9 -t $'\t' -u < desired.table.aliases.txt  > sorted.txt

Let us consider an somewhat more useful alternative.

We begin by just connecting the aliases in column 2 to PEGs in the SEED:

svr_aliases_to_pegs -c=2 -source=NCBI -protein=1 < known.virulence.factors 2> no.matches.1 > with.peg

This produces a file (no.matches.1) of lines that could not be connected to PEGs.

Now,

svr_ids_to_figfams < with.peg > with.ff 2> no.matches.to.ff

can be used to add a FIGfam function and FIGfam ID to the end of each line

for which the identified PEG is in a FIGfam.  Those lines that do not have a PEG

get written to a file (no.matches.to.ff).

Now, it is worth noting that we may have many lines pointing at the same FIGfam (i.e.,

many lines with the same FIGfam ID in the last column, which is column 9).

By using

   sort -u -k 9 -t$'\t' with.ff > with.ff.sorted

we sort the file on the FIGfam ID, deleting all but one of each set that share

a common FIGfam ID.

Now,

svr_figfams_to_ids < with.ff.sorted > with.ff.peg

can be used to expand each input line to a set of lines, each one containing a 

distinct PEG from the FIGfam.  Using 

svr_gene_data function genome-name < with.ff.peg > complete

we add the PEG function and the genome-name to the end of each row.

Finally, we run

svr_aliases_of -c 10 < complete > complete.with.aliases

to add known aliases for each PEG as the last column in the table.

1. Turn on your wireless adapter if it is not already active.
2. Connect to wireless network ANLTCSG-Guest (if you have a normal 802.11b/g/n adapter) or ANLTCSA-Guest (if you have a 5Ghz-band 802.11a adapter).
3. Bring up a web browser and browse to wireless.anl.gov (or any other website). Instead of the website you've chosen, an ANL network page will come up.
4. Read the agreement and click the link at the bottom, then fill out the form that comes up.
5. You may need to wait a bit and turn your wireless off and back on again, or possibly reboot (but probably not that). If the registration page comes up again, reload the page in the browser. It might have cached the registration page for you and helpfully prevented you from seeing the real page.
6. If you are using Windows, you may be asked what type of network this is. Choose Public.
   

On Aug 31 - Sept 2 we held a tutorial on use of the SEED servers, RAST, and myRAST to annotate genomes, formulate metabolic reconstructions, and model metabolic networks.  We think that it went pretty well.  Links to the presentations used can be found here.  We will use basically the same agenda on the four more classes that are now scheduled. These tutorials will be offered at Argonne National Laboratory on the following dates: 

·        Oct 4-6 (Now full)

·       Oct 25-27

·       Nov 1-3

·       Nov 15-17

If you are interested in attending any of these tutorials, please send a note to rast-tutorial@lists.mcs.anl.gov specifying the dates of interest (Oct 25-27, Nov 1-3 or Nov 15-17).  

If

1. The Annotators SEED (A-SEED) remains the SEED used to support manual curations, which are largely cast as creation and maintenance of subsystems.
2. The PATRIC-SEED (P-SEED)  contains all of the complete prokaryotic genomes from RefSeq, as well as all of the complete genomes from the A-SEED. 

1. Create A-FIGfams-1, the restriction of P-FIGfams to A-SEED Genomes
   
   The first step is to take the current P-FIGfams, which may have numerous assertions relating to the extended set of genomes maintained in P-SEED, and restricting those protein sets to genomes from A-SEED.  Protein sets that condense to singleton sets are removed in the process.
2. Create A-FIGfams-2 by Updating the A-FIGfams-1 in the A-SEED
   
   This step (in effect) uses the current subsystems in the A-SEED, as well as any non-subsystem clustering done in the P-SEED, to create a reconciled set of FIGfams for the A-SEED.  A kmer-update is generated from the contents of A-FIGfams-2.
3. Recall Protein Functions and Subsystems in P-SEED
   
   This, in effect, imports the manual subsystem maintenance into the annotations of P-SEED.
4. Update P-FIGfams to P-FIGfams-1 in the Updated P-SEED
   
   Since this is an update of P-FIGfams using the updated subsystem   annotations, it retains the clustering work done in P-SEED, while keeping the annotations imported from A-SEED.  Create an updated set of kmers from P-FIGfams-1.
5. Install the new Kmers into the Kmer Server
   
   This leaves one with the largest set of Kmers (suitable for recognition of pan-genomes, and offering a larger set of signature kmers than is possible using just genomes from the A-SEED).
6. Recall the Functions of  the Proteins in P-SEED using the New Kmers
   
   The change should be minor, but will induce consistency.  The subsystem assertions can then be finally updated.

This completes the steps needed to update the A-FIGfams (to A-FIGfams-2), the annotations in the P-SEED, P-FIGfams (to P-FIGfams-1) and the Kmers.

It remains essential that we maintain the list of induced changes and do manual inspections to enforce quality control on the process.