Changed codes:

    WCDSSELECTED    -> WCDS
    WCDS            -> WCDS_OTHER
    WSPLICESELECTED -> WSPLICE
    WSPLICE         -> WSPLICE_OTHER


git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/trunk@644 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/trunk@518 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

For UD slices we also generate g. descriptions on the chromosome reference. We
now also apply the roll rule there and use correct ranges and sequences on the
reverse strand. Fixes #75.


git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/trunk@495 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/trunk@476 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

The old way of using the configuration file was by instantiating a Config
object which read the file. This instance was passed to every function and
object that might need it.

The new way is by simply calling config.get('name') to get the configuration
value for 'name'. This lazily reads the configuration file and the contents
are cached for future calls.


git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/branches/implicit-config-branch@408 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

This is a rebase of commit r202 to the current branch. Please not that this
is not a very clean implementation and still contains several Todo's. But
they don't affect functionality outside issue #38.

Please see the issue for more information.



git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/branches/refactor-mutalyzer-branch@325 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/branches/refactor-mutalyzer-branch@276 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

- Warning can also be shown for a roll on the reverse strand.
- Only the roll warning about the selected strand is shown.
- Roll of insertion where the inserted sequence changes now has correct
  description on the reverse strand.



git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/branches/refactor-mutalyzer-branch@275 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/branches/refactor-mutalyzer-branch@268 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/branches/refactor-mutalyzer-branch@267 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

This implements the revert mentioned in the previous commit and now this
branch fully merged the first part of exon-deletions-branch. The second
part of that branch is about unknown offsets, but that implementation is
not yet fit to be merged.



git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/branches/refactor-mutalyzer-branch@234 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

This merges the first part of exon-deletions-branch (the second part concerns
unknown exon offsets, denoted with the ? character). Except that we now keep
track of removed splice sites in the transcript, instead of in the Mutator
instance. I guess the latter was better, so we will probably revert to storing
this in the Mutator instance.

Issue #35.



git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/branches/refactor-mutalyzer-branch@233 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/branches/refactor-mutalyzer-branch@214 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/branches/refactor-mutalyzer-branch@212 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/branches/refactor-mutalyzer-branch@211 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/trunk@112 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

- Added misc_RNA, ncRNA, rRNA, tRNA and tmRNA as valid RNA fields.

GenRecord.py:
- Enabled the transcription of non-coding RNA.

Web.py:
- Increased the version number and release date.



git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/trunk@102 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

mutalyzer.conf:
- Increased the UpdateInterval window to three weeks (apparently 7 was too
  small since we sometimes miss an update).

handler.py:
- Added mimetype guessing for all files in base/ (to make images cacheable and
  to reduce browser errors).

UCSC_update.py:
- Fixed a bug that made the update program crash.

GenRecord.py:
- Added a function __checkExonList() that checks whether CDS start and CDS stop
  are in the list of gathered exons.
- Added error handling for incomplete exon lists.

Web.py:
- Updated the release date.

menu.html:
- Fixed a syntax bug.

style.css:
- Fixed a syntax bug.



git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/trunk@89 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

- Made the ``multiple transcripts found'' message more descriptive.
- Made the ``in frame stop codon'' check work for all organisms (and
  organelles).

GBparser.py:
- Added mitochondrial DNA to the list of possible molecular types to be
  checked, it will be treated like genomic DNA.
- Allowed genes without CDS list or mRNA list (only a CDS location) as a
  genomic description only if there are no other CDS's for that gene. 

GenRecord.py:
- Fixed a bug in the naming of the link method (it should also apply when mRNA
  is constructed from a CDS location only).


git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/trunk@87 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

Lots of comments





git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/trunk@85 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

src/
    Mutalyzer.py
	Batch Jobs omit additional mutalyzer results when an error occures

    LRGParser.py
	Added the locus tag, transcriptProduct and proteinProduct
    
    GenRecord.py
	Removed transLongName & protLongName.
	Substituted by transcriptProduct and proteinProduct

    File.py
	Empty lines in batch files are preserved
	A better summary of errors is displayed
	A threshold of 5% is set which omits erroreneous entries
	Added the skip flag ~! for entries to be skipped at all
	    Used for empty lines and lines that do not follow
	    the formatting of either the new of old style
	
    Db.py
	Added the option to set a flag when inserting an entry

templates/
    batch.html
	Moved the error below the input form


TODO:
    File.py
	Add threshold to config file
	


git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/trunk@76 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

vanished. Fixed a large number of bugs.

Db.txt, errorcodes.txt:
- Renamed and converted to LaTeX.

bibliography.bib:
- Added references for the TRM.

TechnicalReference.tex:
- Documented a large part of the modules.

Mutalyzer.py:
- Added a __formatRange() function for better output of a range.
- Added a __checkIntronPosition() function that checks the validity of an 
  intronic notation.
- Fixed a bug that made a change in the first 9 nucleotides result in an
  unknown protein description.
- Modified the error- and warning messages for large strings (see Mutator.py)
  and ranges.
- Added a filter for indexing by protein isoform.

Mutator.py:
- Replaced the calculation of restriction sites using sets with code that does
  so using multisets. This allows the deletion of a site while an other site
  with the same name remains to be detected. Restriction sites are also
  reported in a more compact way.
- Extracted the part of the visualisation where large strings are represented
  by their pre- and suffix and a number of omitted nucleotides. This is now
  available as a function visualiseLargeString().

Parser.py:
- Separated the visualisation of a parse error from the error message. The 
  visualisation is now put in the output object as "parseError".

Output.py:
- Added documentation.

GBparser.py:
- Moved the reverse-complement logic for the __findMismatch() function to the
  function itself.
- Added an extra check for retrieving slicing information.
- Added a filter for poorly annotated exons.
- Added documentation.

Retriever.py:
- Added file type information (for lrg and gb files).
- Added sanity checks for the snpConvert() function.
- Added documentation.

GenRecord.py:
- Added a new type "construction" to the linking methods, meaning that the
  mRNA is generated from the CDS.

Web.py:
- Added an email variable for output to the website.

index.py:
- Added error information, to hide parts of the output.
- Added file type information (see Retriever.py) to make download links more
  sensible.
- Modified the syntax checker to work with the new parser output (see
  Parser.py).
- Fixed a bug (selecting the human build) in the position converter.
- Renamed download() to webservices().
- Renamed ConversionChecker to PositionConverter.

disclaimer.html:
- Added.

check.html:
- Modified the template to work with the new restriction site format and the
  new parse error format.
- Made lots of things conditional.

skel.html:
- Added.

snp.html:
- Added an example.

menu.html:
- Modified the layout, added some links, etc.

Other templates:
- Mainly edited the layout and indentation.



git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/trunk@75 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

mRNA and CDS fields). Added a SNP converter. Implemented a first check for
variants that hit splice sites. Added a chromosomal position for NC slices.
Added a check for the use of intronic positions in a transcript reference 
sequence.

mutalyzer.conf:
- Added variables for splice site mutation detection.

Mutator.py:
- Merged the restriction sites (added, deleted) in one object.

Output.py:
- Changed the return type of getIndexedOutput() and getOutput() from None to
  an empty list for convenience.

Config.py:
- Added variables for splice site mutation detection.

GBparser.py:
- Completely rewrote this module.
  - Will now collect all CDS, mRNA and exon information.
  - Tries to match a CDS and mRNA based upon the range.
    - When this succeeds, try to match on protein, locus tag or product name.
    - If all fails and there is only one option left, link it.
  - Will remove genes that are not fully annotated (half outside the record
    for example).

Retriever.py:
- Added a snpConvert() function.
- Added a check for uploaded records or slices that have no sequence in them
  (a complete contig for example).

GenRecord.py:
- Added variables to cope with chromosomal coordinates.
- Added a toChromPos() function to convert a g. notation to a g. on a
  chromosome.
- Added a addToChromDescription() function to generate a chromosomal
  description of a variant.
- Modified some warnings concerning missing mRNA or missing CDS.
- Added a checkIntron() function that gives a warning when a variant hits a
  splice site.

Web.py:
- Added a nomenclature version variable.
- Added a urlEncode() function to generate valid links.

File.py:
- Made the getMimeType() function public.

Mutalyzer.py:
- Added a __intronicPosition() function that checks whether the user used an
  intronic position.
- Added checks for illegal use of intronic positions.
- Fixed a bug in the __toProtDescr() function.
- Added more checks for the translation and transcription of transcripts.
- Added a better CDS start site mutation detection.
- Added a chromosomal description if available.
- Added more information to the legend (product and linking method).
- Merged the restriction sites (added, deleted) in one object.

handler.py:
- Added guessing of mime types for downloadable files (it used to default to
  text/plain).

index.py:
- Added a snp() function to interface with the snp() function of the Retriever.
- Added a checkForward() function to accommodate for HTTP GET links.
- Removed the `version' variable from all functions, it is now moved to the
  menu.

check.html:
- Modified to cope with the new functionality.

gbupload.html:
- Restructured the layout.

snp.html:
- New page for the SNP converter.

menu.html:
- Modified to cope with the new functionality.

style.css:
- Modified to make some better tables and helper boxes.

index.html:
- Completely rewritten it.

parse.html:
- Restructured the layout to make it uniform with the name checker.



git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/trunk@73 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

Mutalyzer.py
    * Updated the batch output generator

handler.py
    * Make result file downloadable

webservice.py
    * Make webservice compatible with the new Mapper module

VarInfo.py
    * Updated functions to reflect new Mapper module

index.py
    * Added a redirect option to move get messages to the check form
    * Updated check to use the Session from redirect
    * implented __checkInt to catch user errors in upload
    * Added a progress option to get updates about batch progress
    * updated batch to show batch progress after submit

#Templates
interface.js
    * Added the AJAX request to get a batch progress counter
    * Started the linkify to change all Mutalyzer instances to clickable links

batch.html
    * Added a hidden form to support the batch progress

gbupload.html
    * Display Errors 



#Modules
Output.py
    * Added the getMessagesWithErrorCode

LRGParser.py
    * Made the seq a BioSequence for the restriction function

Retriever.py
    * Added BatchFlags support to prevent jobs to hang at the same problem
    * Updated some returnvalues to reflect errors

Scheduler.py
    * Implemented the Flags to skip or alter other batch entries
    * Cleaned up the nameCheckerbatch significantly
    * Added a try/except clause around the namechecker to prevent batchhangs
    * Changed output fileextension to txt

Mapper.py
    * Cleaned up Module
    * mainTranscript function updated to reflect webservice needs
    * Correct errors when retrieving fields from DB in _FieldsFromDb

Db.py
    * Impemented entriesLeftForJob to get info about how a job is doing
    * Added updateBatchDb and skipBatchDb to skip/alter batch entries




git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/trunk@71 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

notation. Implemented the detection of the addition or removal of restriction 
sites. Added additional information about the transcript under scrutiny (if
any), like the g. and c. positions of the exons, as well as the CDS start
and stop.

install.sh:
- Added a rewrite rule to link to the bug tracker.

Db.txt:
- Updated the definition of the GBInfo table.

errorcodes.txt:
- Added a new error (EPOS).

templates/base/images:
- Made new images for the template (temporary).

style.css:
- Added a new class to make a form button look like a clickable link (used in
  check.html).
- Added a new <td> class for right aligned text.

check.html:
- Made a lot of output conditional, depending on what Mutalyzer returns.
- Made the submit forms conditional, depending on the method of retrieval:
  - GET ->  No forms.
  - POST -> Full output.
- Added exon and CDS information.
- Added restriction site information, coupled to the `raw variants'.
- Added a download link for the reference sequence.
- Made the affected transcripts clickable (post to Mutalyzer).
- Added highlighting of changed amino acids for the protein changes.

gbupload.html:
- Added a conditional output section for uploaded sequences.

menu.html:
- Changed the logos.
- Added a link to the bug tracker.

index.html:
- Cleaned up the layout (indenting).
- Added link to Gen2Phen.

Mutator.py:
- Implemented the detection of the addition or removal of restriction sites.
  - Of each raw variant and its flanking sequences, two sets of restriction
    sites is constructed. One for the original subsequence and one for the
    mutated subsequence.
  - Additions and deletions can be detected by using the minus operator for
    sets.
- Changed the `visualisation' output list from a bare list to a list of 
  triples.
- Added verbose output to all raw variants.

Parser.py:
- Added productions for the EX and IVS position notations.

Output.py:
- Added a function that returns the data at the index in a list. Neither the
  list, nor the index need to be present, in which case None will be returned.
  This is convenient in index.py.

GBparser.py:
- Added a version variable to the record class (not used yet).
- Fixed a bug in matching transcripts and CDS by using a locus tag.

Retriever.py:
- Added return values for downloadrecord() and uploadrecord(), so they can be
  used in index.py.

GenRecord.py:
- Added translate and transcribe variables to the Locus class, to be used to
  skip translation of non coding transcripts and to skip the processing of 
  incomplete transcripts as a whole.

Web.py:
- Revived the old tal parser to return a more basic html page (used when GET
  instead of POST is used in index.py). Mainly for LOVD.

Scheduler.py:
- Testing.

Crossmap.py:
- Updated comment.
- Added:
  - getSpliceSite(), get the g. position of a splice site.
  - numberOfIntrons(), get the number of introns in a transcript.
  - numberOfExons(), get the number of exons in a transcript.

index.py:
- check():
  - Removed checks by using the getIndexedOutput() function.
  - Added a call to the `old' tal parser in case of a GET request.
  - Added more output (exon/CDS info, restriction sites, ...).
- upload():
  - Implemented calls to the uploadrecord() and downloadrecord() functions of
    the Retriever module.

Mutalyzer.py:
- Implemented a new protein visualisation function that highlights the changes.
- Used __toProtDescr() (and therefore the findInFrameDescription() and 
  findFrameShift() functions) to return the extent of the change, as well as a
  description of it. The extent is coded as a triple: (from, original end,
  variant end). This is used for the new visualisation.
- Added a check for an inversion of length 1 (actually a substitution).
- Added translation from EX and IVS positions (and ranges) to standard HGVS 
  positions.
- Added a check for out of boundary positions (outside the reference sequence).
- Added exon and CDS information to the output.
- Made the translation of a CDS depend on the translate member variable of the
  Locus class.



git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/trunk@66 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

- Implemented the checking of shifting a deletion or insertion over a splice 
  site boundary in a transcript.

Db.txt:
- Added table definitions for the new Link table (used for the linkage of mRNA
  and CDS in the absence of locus tags).

GBparser.py:
- Moved all functions to a GBparser class.
- Added a transcriptToProtein() function that queries the database for a link
  between a transcript- and a protein accession number, if it is not found, the
  NCBI is queried to retrieve this link (and it is added to the local database).
- Modified the createGBRecord() function to work with the new linkage function.

Config.py:
- Added an email variable for the GBparser module (temporarily).

GenRecord.py:
- Added a function to the Gene class that creates a new locus tag.
- Added a function to the Gene class that finds a link between a transcript and
  a CDS, based on the protein accession number.

Db.py:
- Added the functions getProtAcc(), getmrnaAcc() and insertLink() for the
  linkage of transcript and CDS.

gbupload.html:
- Changed the orientation input field to a select box.

Mutalyzer.py:
- Implemented the checking of shifting a deletion or insertion over a splice 
  site boundary in a transcript.
- Fixed a bug in the description of a frameshift.



git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/trunk@59 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/branches/lrg@55 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

handling for incomplete and poorly annotated GenBank files.

menu.html:
- Template for all pages. Includes a menu and such.

index.html:
- The main page.

base/:
- Pictures, javascript, stylesheet, etc.

interface.js:
- Temporary script for gbupload.html.

download.html:
- Adjusted to work with the menu.html template.

gbupload.html:
- Adjusted to work with the menu.html template.

check.html:
- Extended the template to use more variables to increase flexibility.

index.py:
- Removed the capturing of the Mutalyzer output in the check() function,
  replaced it with retrieval of data from the Output object. 
- Added a new index() function (see index.html).
- Implemented the upload() function (see gbupload.html), currently it can only
  retrieve genes and chromosome slices from the NCBI.

Web.py:
- Enabled the parsing of the menu.html template in the tal() function.

handler.py:
- Modified to work with the new tal() function of Web.py.

Output.py:
- getMessages() now returns a list of messages, instead of printing it to 
  standard output.

mail.txt:
- Template message for finished batch jobs.

Mutalyzer.py:
- __bprint() now puts the visualisation of a protein in the Output object, 
  instead of printing it to standard output.
- Made a new protein printing function that can highlight the changes, it is
  not used yet.
- Added a __cdsLen() function that is used to see if a variant leads to a
  frameshift or not.
- Added extention of a protein in the protein description function.
- Added some error handling for `?' as a position.
- Added error messages for invalid genes and transcript variants in the 
  input (suggesting possible options).
- DNA / protein descriptions are now stored in the Output object.  
- Added a legend.

GenRecord.py:
- Added functions listLoci() and listGenes() for suggestions when invalid 
  input is given.
- Added a preprocessing step in the GenBank file parsing to deal with 
  missing locus tags (not functional yet).
- Added some error handling for incomplete transcripts.



git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/trunk@52 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

section included in record.

Extracted code to create a GenRecord.Record from a GenBank file from
GenRecord.py and included it in GBparser.py

Updated the Retriever Module to facilitate two different retriever-types;
LRGRetriever and GBRetriever.

Updated the Db module with one method to update a LRG record in the database


NOTE: Ensembl data is parsed but not yet included in the GenRecord.Record,
because there is no clear usage for this. This behaviour can be changed in the
LRGparser.py file


git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/branches/lrg@51 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

compatible with the mutalyzer engine at the moment, so it is disabled. 

Added basic unittests to test the validity of Records


git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/branches/lrg@49 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/branches/lrg@46 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

record. Added a legend for transcripts and proteins.

Install.txt:
- Added dependency information.

Mutalyzer.py:
- Rewrote the roll function to roll both ways at the same time.
- Rewrote the trim function (fixed a lcs bug).
- Moved the semantic checking of raw variants to separate functions.
- Removed lots of conversion code. We now convert to g. first and check this
  to all transcripts.
- Added protein descriptions for all transcripts.

GenRecord.py:
- Added more gathering information (locus tags, transcript- and protein 
  identifiers, etc.)
- Made a separate function that checks the gathered information after 
  retrieval. This can be used later on by the LRG parser.
- Modified the name() function to give better descriptions.

Parser.py:
- Added code to distinguish between GenBank records and LRG records.



git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/trunk@41 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/branches/lrg@38 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

distribution as it is under heavy development.

Most modules will have minor changes because of a difference in set up of both
the Db and Config module.

install.sh:
- Added functionality to enable the cron restart of the Batch Checker.
- Added the auto-generation of a .htaccess file.
- Added permission settings.

mutalyzer.conf:
- Added configuration options for the Scheduler, File and GenRecord modules.

Db.txt:
- Described how to make the new ChrName tables for hg18 and hg19.

errorcodes.txt:
- Added classifications to the messages.

doc:
- Made a set up for the documentation.
  TechnicalReference:
  - This will be a technical document that describes the internals of the
    project. It is only meant for developers.
  API:
  - This is a description of the API, it is auto generated by the mkapidoc.sh
    script. Also only meant for developers.

Mutalyzer.py:
- Added a new roll function that will always find both boundaries.
- Implemented a new protein naming scheme.
- Fixed the trimming of a delins.
- Rewrote the processing of a variant. 
  - Moved post processing of the GenBank record to the GenRecord module.
  - Moved the crossmapper instance to the GenBank module, to make one instance
    per transcript variant.
  - Moved the naming of a variant to the GenBank module, as is strongly
    interacts with the crossmapper instance.
- Moved the constructCDS function to the GenRecord module.

handler.py:
- Added functionality for the batch checker (retrieve results).
- Added functionality for the genbank uploader (retrieve GenBank files).

webservice.py:
- Modified to work with the new Db module.

UCSC_update.py:
- Modified to work with the new Db module.

GenRecord.py:
- Replaced the dictionary structure with a nested list structure to make
  iteration more convenient.
- Added names to the Locus and Gene objects.
- Added all information needed to do a crossmapping in the Locus object.
- Wrote functions to find Loci and Genes.
- Wrote a function that expands a description of a variant (coupled to a Locus).

Mutator.py:
- Added documentation.

Parser.py:
- Added documentation.

Web.py:
- Added documentation.
- Added a function that checks whether a string is an e-mail address.

Scheduler.py:
- Implemented a batch scheduler that uses a MySQL database for queueing.

File.py:
- Implemented a CSV, XLS and ODS parser for use in the Scheduler module.

Output.py:
- Added documentation.

Mapper.py:
- Modified the complex object initialisation.

Config.py:
- Made subclasses to configure the separate modules.

Db.py:
- Added documentation.
- Split the Db modules into different classes, according to functionality, they
  all inherit the query function from the Db base class.
- Added chromosome accession number to name conversion functions and vice versa.
- Added functionality for the batch checker.

Crossmap.py:
- Added documentation.

Retriever.py:
- Added documentation.
- Added fall back functionality when searching for a gene.

index.py:
- Added a batch submit interface.

batch.html:
- The layout of the batch submit interface.



git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/trunk@30 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

list with priorities, and all output is stored in a dictionary. This dictionary
can be read at a later time.
The Retriever module is changed to accommodate for uploaded GenBank files and
slices of a chromosome.
A Scheduler was added for batch checking.

mutalyzer.conf:
- Added variables for the Retriever module:
  - maxDldSize, minDldSize ; Maximum and minimum sizes for slices and uploaded 
    GenBank files.
- Added variables for the Output module:
  - loglevel, outputlevel ; Specify default verbosity levels for logging and 
    output.
- Added variables for the Scheduler module:
  - processName ; Name of the running scheduler.

Db.txt:
- Added information on how to create the newly used tables GbInfo, BatchQueue
  and BatchJob.

errorcodes.txt:
- Short description of the error codes used in the Output module.

webservice.py:
- Added a Complex class to test more complicated return types (see the web_dev
  branch).
- Modified the code to work with the new Output module.

index.py:
- Made a first upload page.

Mutalyzer.py:
- Modified the code to work with the new Output module.

UCSC_update.py:
- Modified the code to work with the new Output module.

Variant_info.py:
- Modified the code to work with the new Output module.

GenRecord.py:
- Added documentation.

Mutator.py:
- Modified the code to work with the new Output module.

Misc.py:
- New file, used for generating unique IDs.

Parser.py:
- Made a change to the definition of an UD accession number.
- Modified the code to work with the new Output module.

Scheduler.py:
- Made a batch checker scheduler.
  - isDaemonRunning() ; See if we need to be started.
  - process() ; Start the batch checker.
  - addJob() ; Add jobs to a queue in the database.

Output.py:
- Added a Message class to store all debug, info, warning, error and fatal
  messages.
  - If a message is given that exceeds the configured log level, it will be
    logged immediately.
- A function is added to the Output class to read all messages that exceed a 
  certain verbosity level.
- A function is added to create a named list as an output node.
- With the getOutput function the content of this list can be retrieved.

Config.py:
- Several sub-classes were added for each configurable module.

Db.py:
- Added documentation.
- Added functionality that is used by the Retriever module.
- Added functionality that is used by the Scheduler module.

Retriever.py:
- Added functionality to be able to use custom GenBank files and chromosome
  slices.
  - Information on these created files are stored in a database to be able to
    re-create them when the cache is cleaned.
  - The hash of each file is stored for error detection.

BatchChecker.py:
- A wrapper that is called either from the addJob() function from the Scheduler
  module, or from cron. It dispatches a background process that processes the
  batch jobs.

gbupload.html:
- Test template for uploading files (copied from Mutalyzer 1.0.4).

sp.py:
- Some test with a complex return type.

download.html:
- Did some first tests with a METAL template.



git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/trunk@26 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

The classes Mutator, Output and Db are now derived classes of Config. The
class Retriever is a derived class of Output. This reduces the amount of
code and variable passing significantly.

mutalyzer.conf:
- Converted the dbName variable to a dbName list, to accommodate for more than
  one database.
- Added variables flanksize, maxvissize and flankclipsize for the visualisation
  in Mutator.py (instead of the previous alignment).

Mutalyzer.py:
- Resolved the range-swap issues.
- Resolved the reverse-complement (cosmetic) issues.
- Fixed a cosmetic bug in the __bprint() function.
- Added a new function __nsplice(), to accommodate for a CDS extension.
- Added a function __toProtDescr(), that gives a protein description in case
  of a simple substitution.
- Added functionality for n. m. and EST notations.
- Added functionality for other species (translation tables).
- Corrected the roll-rule for insertions.
- Added fallbacks for missing CDS and mRNA lists and positions (for an EST for
  example).
- Added an input check for wrong gene symbols.
- Added a temporary exception for in frame stop codons.

webservice.py:
- Added the private functions __checkBuild(), __checkChrom() and __checkPos()
  that do routine checks in a number of services.
- Added a 'build' variable to getTranscripts(), getTranscriptsRange() and
  getGeneName()
- Added exceptions that raise a Fault() object to make the client receive a
  SOAP exception.

UCSC_update.py:
- Added functionality for more than one database.

Variant_info.py:
- Added functionality for more than one database.
- Added functionality to deal with non-coding transcripts.

Genrecord.py:
- Made a RecordObj() object that consists of the old 'genelist' dictionary, 
  combined with 'mol_type' and 'organelle' variables. Also, a fake gene named
  'source' is included to accommodate for sequences that do not contain any
  annotated genes (an EST for example).
- The Locus object is extended with a 'txTable' variable to accommodate for
  different organelles (mitochondria) and other species.

Mutator.py:
- Replaced the alignment visualisation by a home-made one. Also see the
  variables that were added to the configuration file to alter the behaviour of
  this visualisation.
- Modified the shiftpos() function when inserting something on a splice site
  boundary (now it extends the exon).

Output.py:
- Minor modifications for the new inheritance scheme.

Config.py:
- Minor modifications for the new inheritance scheme.

Db.py:
- Minor modifications for the new inheritance scheme.
- Added functionality to handle multiple databases.
- Added an isChrom() function, used by the webservices check functions.

Crossmap.py:
- Made a change in the usage of the '__STOP' variable. It is set to
  transcription stop if there is no stop codon present. This makes conversion
  to an n. notation trivial.

Retriever.py:
- Minor modifications for the new inheritance scheme.
- Added a check for invalid accession numbers (or versions).
- Added a check for erroneous genbank files that can occur when the NCBI is
  overloaded. The erroneous file is purged and the user can try again.

templates/sp.py
- Modified the sample code to accommodate for the new 'build' variable.



git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/trunk@20 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

The main structure is as follows:

/                     ; Root of the installation.
  - Install.txt
  - Todo.txt
  - Obsoleted.txt
  - mutalyzer.conf
  - var/              ; Variable data.
      - cache/
      - mutalyzer.log
  - templates/        ; HTML, XML, JavaScript, etc.
  - src/
    - Mutalyzer.py
    - Variant_info.py
    - Services/       ; Webservices.
    - Clients/        ; Example clients for webservices.
    - Modules/        ; The core modules.
    - Interfaces/     ; Interfaces to mod_python.

Apart from changes that were needed to deal with this new structure, no changes
in the code were made.


git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/trunk@14 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

- Todo.txt.
- handler.py: A general handler for mod_python. This handler dispatches
  SOAP services and a normal HTML publisher. Furthermore, it is able to handle
  raw requests to dump HTML or XML files. This handler is TAL enabled.
- webservice.py: A publisher for webservices. When a new webservice is
  added, this is the entrypoint for the server side (just like index.py, add
  a function).
- getTranscripts.py: A webservice that reports all transcripts that overlap 
  with a certain genomic position (chomosome, position). 
- getGeneName.py: A webservice that finds the gene name of a given transcript.
- service.wsdl: This is the definition of the interface for webservices.
  A client must download this file and parse it to obtain a programming 
  interface, then the client can use this interface just like any local 
  function.
- Obsoleted.txt: A list of things that will be deleted in the future (but are
  still functional for backwards compatibility).
- client/sp.py: A test client for the two webservices.

Renamed Main.py to Mutalyzer.py.

Modified:
- mutalyzer.conf: Added a configurable date prefix for logging.
- Install.txt: 
  - To reflect the difference in configuration of apache to work with the new 
    handler (requires less configuration).
  - Added TAL as a new dependency.
- html/check.html: Made it a full TAL template. Title, version and output are
  now separated from the HTML design.
- Mutator.py: Made the shiftpos() function public, this is needed for insertion
  checking.
- Parser.py: Updated the comments.
- Variant_info: Made all internal functions private.
- Output.py: Updated the comments.
- Config.py: Modified to reflect the changes in mutalyzer.conf.
- Db.py:
  - We now keep the handle to the database open until the object is deleted.
  - Added a destructor that closes the handle to the database.
  - Added getTranscripts(): Get a list of transcripts, given a chromosome and a
    position on that chromosome.
  - Added get_GeneName(): Get the gene name of a given transcript.
- Retriever.py: Updated the comments.
- Crossmap.py:
  - Made a patch that handles a CDS start on the first position of the 
    transcript.
  - Added more unit tests.
- index.py: 
  - Added a switch for older versions of LOVD, to generate the expected output 
    in Variant_info.
  - Made this publisher compatible with TAL.
- Mutalyzer.py: Made all internal functions private.



git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/trunk@13 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

can be sent to this object, which sends errors and explicit logging messages
to a log (defined in mutalyzer.conf) and warning messages to standard output.
Behaviour of this object may change in the future, adding severity and logging
above a certain severity level is one option that would increase debugging 
possibilities.
- NiceName() returns a short description of the calling program (we can not 
  use the default __name__ here.
- ErrorMsg() Print the nice name of the calling module, an error message and 
  log it. Also increase an error counter.
- WarningMsg() Print the nice name of the calling module and an error message.
  Also increase a warning counter.
- LogMsg() Only log the message (nice name of the calling module and the 
  message itself).
- Summary() Give the number of errors and warnings.
- A unit test is also defined (it does not do much at this moment).

Added index.py: The web interface to mutalyzer, it is dependent on mod-python,
we chose for this interface to eliminate the need for php. Also apache is now
added to the list of dependencies. The configuration of mod-python is described
in Install.txt.

GenRecord.py: 
- Moved the splice() function to Main.py. 
- Added an exon list to the Plist class. This list can be used as a fallback
  in case the mRNA tag is missing from a GenBank file.
- Added an empty unit test.

Mutator.py:
- Added a standard alignment for visualisation. This will probably be replaced
  in newer versions.

Parser.py:
- Added functionality for the new output module.
- Made the parser gracefully return, instead of exit on a parse error. This is
  needed for the web interface.

Variant_info.py:
- Added functionality for the new output module. Note that output generated by
  this program should go to a different log, something for a future version.
- Fixed a bug that occurred when a CDS start or stop was on an exon boundary.

Main.py (heavily under development, names of functions are not very descriptive
  yet):
- Added functionality for the new output module.
- Fixed a bug in the roll() function, it returned a wrong value for forward
  genes.
- Added the function bprint(), it formats a large string to be printed in an
  insightful way (like GenBank does it), it also prints the offsets at the
  beginning of each line.
- Obsoleted the ErrorMsg() and WarningMsg() functions.
- Added constructCDS(), a function that is able to construct a CDS from an
  mRNA list, CDS start and CDS stop. In the future we would like to work 
  without a CDS list, so this function will be obsoleted.
- Added the splice() function (from GenRecord.py).
- Made a function rv() that is able to process a RawVar. This function is
  seperated from the ppp() function to be able to work with an allele 
  description.
- Added splicing.
- Added translation to a protein.
- Added a function rrr() which is to be called from Main.py itself or from 
  index.py.

Config.py:
- Added a log variable for Output.py.

Crossmap.py:
- Fixed a bug concerning genes where the entire CDS is in one exon.
- Added more uncertainty handling.

Retriever.py:
- Added functionality for the new output module.
- Added handling of accession numbers with no version. It downloads the latest
  version, and gives a warning.



git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/trunk@10 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

- src/c2g.py
  - This program returns the genomic coordinates of a position in c. notation,
    it depends on the Config, Db Crossmap and Parser modules.
  - The input consists of the following variables:
    - LOVD_ver      ; The version of LOVD, ignored for now.
    - build         ; The build of the human genome, assumed to be hg19 for now.
    - accno.version ; The accession number and version on which the c. 
                      positions are defined.
    - var           ; The variation (without an accession number).
  - If the accession number and version are present in the database, it returns
    all positions (one or two) in g. notation relative to the chromosome.
- c2g.php 
  - This is a web interface for c2g.py, it is called in the following way:
    c2g.php?LOVD_ver=a&build=b&acc=c.d&var=e where a, b, c, d and e are 
    described above (note that the variation (e) should be HTML encoded).

Altered (with respect to c2g.py):
- Install.txt
  - Documented how to set up the c2g program.
- Db.py
  - Added two new query functions:
    - get_NM_info    ; Get exonStarts, exonEnds, cdsStart, cdsEnd and strand
                       information, given an NM accession number.
    - get_NM_version ; Get the NM version, given an NM accession number.
  - Made a general query function, that is called by the specific query 
    functions.
- Crossmap.py
  - Added helper functions for the output of c2g.
    - c2str ; Returns a string given mainsgn, main, offsgn and offset.
    - c2g   ; Returns a genomic position given mainsgn, main, offsgn and offset.

Altered (for Mutalyzer itself):
- GenRecord.py
  - Added default locus tag handling.
- Mutator.py
  - Added a duplication function.
- Parser.py
  - Added argument passing:
    - Substitutions: Arg1, Arg2 (Arg1>Arg2 for example).
    - Deletions: Arg1 (delArg1).
    - Ins: Arg1 (insArg1).
- Retriever.py
  - Added code to make the cache directory if it does not exist. This 
    eliminates the need for the clean.sh script.
- Main.py (most of the new functions have to be migrated elsewhere)
  - Altered the roll function to be able to roll in both directions.
  - Made a palindrome snoop function that finds the smallest string that is 
    not invariant under reverse complement, this function is also used to 
    detect perfect palindromes.
  - Made PtLoc2main and PtLoc2offset; get integers from the locations returned
    by the nomenclature parser.
  - Made Error and Warning message wrapper functions.
  - Made a function that tests if a string is an integer.
  - Made the parsing less verbose.
  - Extracted the positions from raw variations.
  - Wrote code to deal with reversed ranges.
  - Added error handling and warning messages for:
    - Substitutions.
    - Deletions.
    - Duplications.
    - Inversions.
    - Insertions.

Removed:
  - clean.sh
    - It is no longer needed, since all temporary files and directories are no
      longer in subversion.


git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/trunk@7 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1