Added help and about pages.

Extended the TRM.

Mutalyzer.py:
- Added a __checkDNA() funtion for input checking.
- Added a warning message for ``rolled'' insertions that are not duplications.
- Added a warning for unsupported RNA descriptions.

Web.py:
- Added a release date variable.
- Increased version numbers.

index.py:
- Added functions for the new pages (help.html and about.html)
- Fixed a cosmetic bug that occurred when using html tags in the input.

menu.html:
- Made the top and bottom links compatible with Internet Explorer.
- Added a link to the about page.
- Added a link to the name generator page.
- Removed some comment, altered some layout.

index.html:
- Moved some technical data to the about page.



git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/trunk@78 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

=====
src/Modules/Mapper.py
    * LOVDERR fixed
	Correct return string when the Variant-info requests a reference
	that could not be found in the database

ADDED FEATURES
==============
Name Generator

src/index.py
    * Added the Name Generator reference

templates/menu.html
    * Updated the Menu to include the Name Generator

templates/generator.html
    * Main HTML file for the name generator

templates/base/js/generator.js
    * Main JS file for the name generator

TODO
====
Name Generator
    Add Protein description, allel and exotic names





git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/trunk@77 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

src/
    Mutalyzer.py
	Batch Jobs omit additional mutalyzer results when an error occures

    LRGParser.py
	Added the locus tag, transcriptProduct and proteinProduct
    
    GenRecord.py
	Removed transLongName & protLongName.
	Substituted by transcriptProduct and proteinProduct

    File.py
	Empty lines in batch files are preserved
	A better summary of errors is displayed
	A threshold of 5% is set which omits erroreneous entries
	Added the skip flag ~! for entries to be skipped at all
	    Used for empty lines and lines that do not follow
	    the formatting of either the new of old style
	
    Db.py
	Added the option to set a flag when inserting an entry

templates/
    batch.html
	Moved the error below the input form


TODO:
    File.py
	Add threshold to config file
	


git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/trunk@76 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

vanished. Fixed a large number of bugs.

Db.txt, errorcodes.txt:
- Renamed and converted to LaTeX.

bibliography.bib:
- Added references for the TRM.

TechnicalReference.tex:
- Documented a large part of the modules.

Mutalyzer.py:
- Added a __formatRange() function for better output of a range.
- Added a __checkIntronPosition() function that checks the validity of an 
  intronic notation.
- Fixed a bug that made a change in the first 9 nucleotides result in an
  unknown protein description.
- Modified the error- and warning messages for large strings (see Mutator.py)
  and ranges.
- Added a filter for indexing by protein isoform.

Mutator.py:
- Replaced the calculation of restriction sites using sets with code that does
  so using multisets. This allows the deletion of a site while an other site
  with the same name remains to be detected. Restriction sites are also
  reported in a more compact way.
- Extracted the part of the visualisation where large strings are represented
  by their pre- and suffix and a number of omitted nucleotides. This is now
  available as a function visualiseLargeString().

Parser.py:
- Separated the visualisation of a parse error from the error message. The 
  visualisation is now put in the output object as "parseError".

Output.py:
- Added documentation.

GBparser.py:
- Moved the reverse-complement logic for the __findMismatch() function to the
  function itself.
- Added an extra check for retrieving slicing information.
- Added a filter for poorly annotated exons.
- Added documentation.

Retriever.py:
- Added file type information (for lrg and gb files).
- Added sanity checks for the snpConvert() function.
- Added documentation.

GenRecord.py:
- Added a new type "construction" to the linking methods, meaning that the
  mRNA is generated from the CDS.

Web.py:
- Added an email variable for output to the website.

index.py:
- Added error information, to hide parts of the output.
- Added file type information (see Retriever.py) to make download links more
  sensible.
- Modified the syntax checker to work with the new parser output (see
  Parser.py).
- Fixed a bug (selecting the human build) in the position converter.
- Renamed download() to webservices().
- Renamed ConversionChecker to PositionConverter.

disclaimer.html:
- Added.

check.html:
- Modified the template to work with the new restriction site format and the
  new parse error format.
- Made lots of things conditional.

skel.html:
- Added.

snp.html:
- Added an example.

menu.html:
- Modified the layout, added some links, etc.

Other templates:
- Mainly edited the layout and indentation.



git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/trunk@75 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

Mutalyzer.py
    * Typo

handler.py
    * Reverted the download of sp.py to the original version -r25

VarInfo.py
    * Fixed a return bug of the mainMapping

index.py
    * dropped redirect function

#######Modules#######

LRGParser.py
    * added a few checks for existing XML tags

GBparser.py
    * __tagLocus product tag fixed if not existing

Mapper.py
    * _FieldsFromDb fixed if versions is ""

#######Templates######
sp.py
    * reverted to version 25


    


git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/trunk@74 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

mRNA and CDS fields). Added a SNP converter. Implemented a first check for
variants that hit splice sites. Added a chromosomal position for NC slices.
Added a check for the use of intronic positions in a transcript reference 
sequence.

mutalyzer.conf:
- Added variables for splice site mutation detection.

Mutator.py:
- Merged the restriction sites (added, deleted) in one object.

Output.py:
- Changed the return type of getIndexedOutput() and getOutput() from None to
  an empty list for convenience.

Config.py:
- Added variables for splice site mutation detection.

GBparser.py:
- Completely rewrote this module.
  - Will now collect all CDS, mRNA and exon information.
  - Tries to match a CDS and mRNA based upon the range.
    - When this succeeds, try to match on protein, locus tag or product name.
    - If all fails and there is only one option left, link it.
  - Will remove genes that are not fully annotated (half outside the record
    for example).

Retriever.py:
- Added a snpConvert() function.
- Added a check for uploaded records or slices that have no sequence in them
  (a complete contig for example).

GenRecord.py:
- Added variables to cope with chromosomal coordinates.
- Added a toChromPos() function to convert a g. notation to a g. on a
  chromosome.
- Added a addToChromDescription() function to generate a chromosomal
  description of a variant.
- Modified some warnings concerning missing mRNA or missing CDS.
- Added a checkIntron() function that gives a warning when a variant hits a
  splice site.

Web.py:
- Added a nomenclature version variable.
- Added a urlEncode() function to generate valid links.

File.py:
- Made the getMimeType() function public.

Mutalyzer.py:
- Added a __intronicPosition() function that checks whether the user used an
  intronic position.
- Added checks for illegal use of intronic positions.
- Fixed a bug in the __toProtDescr() function.
- Added more checks for the translation and transcription of transcripts.
- Added a better CDS start site mutation detection.
- Added a chromosomal description if available.
- Added more information to the legend (product and linking method).
- Merged the restriction sites (added, deleted) in one object.

handler.py:
- Added guessing of mime types for downloadable files (it used to default to
  text/plain).

index.py:
- Added a snp() function to interface with the snp() function of the Retriever.
- Added a checkForward() function to accommodate for HTTP GET links.
- Removed the `version' variable from all functions, it is now moved to the
  menu.

check.html:
- Modified to cope with the new functionality.

gbupload.html:
- Restructured the layout.

snp.html:
- New page for the SNP converter.

menu.html:
- Modified to cope with the new functionality.

style.css:
- Modified to make some better tables and helper boxes.

index.html:
- Completely rewritten it.

parse.html:
- Restructured the layout to make it uniform with the name checker.



git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/trunk@73 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/trunk@72 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

Mutalyzer.py
    * Updated the batch output generator

handler.py
    * Make result file downloadable

webservice.py
    * Make webservice compatible with the new Mapper module

VarInfo.py
    * Updated functions to reflect new Mapper module

index.py
    * Added a redirect option to move get messages to the check form
    * Updated check to use the Session from redirect
    * implented __checkInt to catch user errors in upload
    * Added a progress option to get updates about batch progress
    * updated batch to show batch progress after submit

#Templates
interface.js
    * Added the AJAX request to get a batch progress counter
    * Started the linkify to change all Mutalyzer instances to clickable links

batch.html
    * Added a hidden form to support the batch progress

gbupload.html
    * Display Errors 



#Modules
Output.py
    * Added the getMessagesWithErrorCode

LRGParser.py
    * Made the seq a BioSequence for the restriction function

Retriever.py
    * Added BatchFlags support to prevent jobs to hang at the same problem
    * Updated some returnvalues to reflect errors

Scheduler.py
    * Implemented the Flags to skip or alter other batch entries
    * Cleaned up the nameCheckerbatch significantly
    * Added a try/except clause around the namechecker to prevent batchhangs
    * Changed output fileextension to txt

Mapper.py
    * Cleaned up Module
    * mainTranscript function updated to reflect webservice needs
    * Correct errors when retrieving fields from DB in _FieldsFromDb

Db.py
    * Impemented entriesLeftForJob to get info about how a job is doing
    * Added updateBatchDb and skipBatchDb to skip/alter batch entries




git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/trunk@71 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/trunk@68 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

		- update errorcodes.txt
		- rewrite some webservice.py functions
		- File.py ods files are not read in correctly. Temp file deleted before it
			can be used

    
CHANGES:
		- mutalyzer.conf
				Added pidfile reference
				Added Output Headers for the Batch Jobs

		- Db.txt
				Updated the Db.txt file so that it reflects the database changes
					* BatchJob now contains an Arg1 attribute, which is used by the
						Conversion Batch to store the build version. This could be
						changed to use the Filter Field.
					* Dropped the Var table
					* BatchQueue -> AccNo, Gene, Variant are dropped and replaced by
						single column input. This fields contains a single full
						mutalyzer variant description.

		- Install.txt
				Added the python-daemon dependency

		- handler.py
				Added a downloads handler for the old and new batch files
				Other handlers are changed so the filter for extensions now matches the
				trailing characters

		- webservice.py
				Made the functions compatible with the new Mapper.Converter class
				*** This webservice.py can use a bit more structure ***

		- index.py
				Moved a lot of logic from the index file to the Mapper.py module
				Implemented the batch handler, which has multiple entry points
				positionConverter now uses the Mapper.Converter class

		- BatchChecker.py
				Fixed the multiprocessing bugs. Now uses python-daemon to spawn a
				process and uses the var/batch.pid as a lockfile for the process

		---- Modules

		- Ouput.py
				Added getSoapMessages, which returns Soap compatible Messages for over
				the wire
				Added getBatchMessages, which returns messages above %level and filters
				the ParseError to one line
		
		- LRGparser.py
				Included the usage of transcription.translate and transcribe for the
				main transcript

		- Config.py
				Added the Batch Headers to the configfile
				Added a Batch Config object for the PID file

		- Scheduler.py
				Added the SyntaxCheck Batch
				Added the Conversion Batch
				Have the BatchProcesser read the Argument from the Batch
				Changed outputfile format to csv

		- File.py
				Changed the csv sniffer to drop the ":" delimiter in batch entries
				Updated some Job logic to detect all lines with basic errors

		- Mapper.py
				Practically a complete refactor of the module
				Added the Converter class for chromosome to c and back conversions
				Added a SoapMessage class which instances can be send over the wire
				Updated the Mapper.Mapping class to use SoapMessage messages
				Basically create correct mappings between c 2 chrom 2 c [tested]
				Reduced the number of Database calls needed significantly

		- Db.py
				Implemented getAllFields to retrieve complete dataset of Fields of
				intereset, this reduced the number of database calls
				Updated the Db logic to encompass the new Batch Job logic:
						Input entries are 1 field
						Jobs have a type
						Jobs can have arguments (1 at the moment, use BITFLAGS)


		---- Templates

		REMOVED 
				- batchCheck.html				-> All Batch jobs are united in batch.html
				- batch_convert.html		->  								"
				- convert.html					-> Replaced by converter.html

		CHANGED
				- batch.html
						Included big file type help
						Made template compatible with the three different batchTypes

				- gbupload.html & menu.html
						Fixed an onload javascript bug which caused a stream of javascript
						errors on pages other than gbupload
		
				- interface.js
						Added the changeBatch and toggle_visibility functions

		ADDED
				- downloads
						Added example download files for old & new batchfiles


NOTES:
		- change permission on var/batch.pid to have apache read/write it
		- 



git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/branches/lrg@67 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

notation. Implemented the detection of the addition or removal of restriction 
sites. Added additional information about the transcript under scrutiny (if
any), like the g. and c. positions of the exons, as well as the CDS start
and stop.

install.sh:
- Added a rewrite rule to link to the bug tracker.

Db.txt:
- Updated the definition of the GBInfo table.

errorcodes.txt:
- Added a new error (EPOS).

templates/base/images:
- Made new images for the template (temporary).

style.css:
- Added a new class to make a form button look like a clickable link (used in
  check.html).
- Added a new <td> class for right aligned text.

check.html:
- Made a lot of output conditional, depending on what Mutalyzer returns.
- Made the submit forms conditional, depending on the method of retrieval:
  - GET ->  No forms.
  - POST -> Full output.
- Added exon and CDS information.
- Added restriction site information, coupled to the `raw variants'.
- Added a download link for the reference sequence.
- Made the affected transcripts clickable (post to Mutalyzer).
- Added highlighting of changed amino acids for the protein changes.

gbupload.html:
- Added a conditional output section for uploaded sequences.

menu.html:
- Changed the logos.
- Added a link to the bug tracker.

index.html:
- Cleaned up the layout (indenting).
- Added link to Gen2Phen.

Mutator.py:
- Implemented the detection of the addition or removal of restriction sites.
  - Of each raw variant and its flanking sequences, two sets of restriction
    sites is constructed. One for the original subsequence and one for the
    mutated subsequence.
  - Additions and deletions can be detected by using the minus operator for
    sets.
- Changed the `visualisation' output list from a bare list to a list of 
  triples.
- Added verbose output to all raw variants.

Parser.py:
- Added productions for the EX and IVS position notations.

Output.py:
- Added a function that returns the data at the index in a list. Neither the
  list, nor the index need to be present, in which case None will be returned.
  This is convenient in index.py.

GBparser.py:
- Added a version variable to the record class (not used yet).
- Fixed a bug in matching transcripts and CDS by using a locus tag.

Retriever.py:
- Added return values for downloadrecord() and uploadrecord(), so they can be
  used in index.py.

GenRecord.py:
- Added translate and transcribe variables to the Locus class, to be used to
  skip translation of non coding transcripts and to skip the processing of 
  incomplete transcripts as a whole.

Web.py:
- Revived the old tal parser to return a more basic html page (used when GET
  instead of POST is used in index.py). Mainly for LOVD.

Scheduler.py:
- Testing.

Crossmap.py:
- Updated comment.
- Added:
  - getSpliceSite(), get the g. position of a splice site.
  - numberOfIntrons(), get the number of introns in a transcript.
  - numberOfExons(), get the number of exons in a transcript.

index.py:
- check():
  - Removed checks by using the getIndexedOutput() function.
  - Added a call to the `old' tal parser in case of a GET request.
  - Added more output (exon/CDS info, restriction sites, ...).
- upload():
  - Implemented calls to the uploadrecord() and downloadrecord() functions of
    the Retriever module.

Mutalyzer.py:
- Implemented a new protein visualisation function that highlights the changes.
- Used __toProtDescr() (and therefore the findInFrameDescription() and 
  findFrameShift() functions) to return the extent of the change, as well as a
  description of it. The extent is coded as a triple: (from, original end,
  variant end). This is used for the new visualisation.
- Added a check for an inversion of length 1 (actually a substitution).
- Added translation from EX and IVS positions (and ranges) to standard HGVS 
  positions.
- Added a check for out of boundary positions (outside the reference sequence).
- Added exon and CDS information to the output.
- Made the translation of a CDS depend on the translate member variable of the
  Locus class.



git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/trunk@66 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/branches/lrg@65 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/branches/lrg@63 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/trunk@61 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

    - Schedular.py
	addJob calls Popen and returns immediatly this kills the spawned process
	and leaves an unjoined child. This must be fixed with e.g. multiprocessing
    - 

CHANGES:
    - templates
	* menu.html	- include navigation to new features
	* check.html	- <pre>mutalyzer output</pre>
	* batch.html	- change Name Checker Batch Output
    - Modules
	* Db.py		- addJob  function attributes include jobType
			  getJobs function returns JobID & JobType
	* Schedular.py  - process function now support different jobtypes
			  addJob  function new attribute jobType
					   disabled BatchChecker -> TODO
	* LRGparser.py  - removed the names of transcript in the updatable
			    section
    - /
	* index.py	- batchCheck fn cleaned up and implemented
	* Mutalyzer.py  - added _batchOutput function to generate a batch output
			  entry in the output object, which is used in the batch
			  version of the NameChecker


NOTES:
    - Modules
	* Schedular.py	- __sendMail doesn't handle mail errors.
			  addJob doesn't start the batchchecker
    - /
	* index.py	- batchConvert fn in the middle of development, cleaning
			  up previous version: batchConversion
			- checkingSyntax function needs to be moved to
			  another section and be reused in the batchSyntaxCheck



git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/branches/lrg@60 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

- Implemented the checking of shifting a deletion or insertion over a splice 
  site boundary in a transcript.

Db.txt:
- Added table definitions for the new Link table (used for the linkage of mRNA
  and CDS in the absence of locus tags).

GBparser.py:
- Moved all functions to a GBparser class.
- Added a transcriptToProtein() function that queries the database for a link
  between a transcript- and a protein accession number, if it is not found, the
  NCBI is queried to retrieve this link (and it is added to the local database).
- Modified the createGBRecord() function to work with the new linkage function.

Config.py:
- Added an email variable for the GBparser module (temporarily).

GenRecord.py:
- Added a function to the Gene class that creates a new locus tag.
- Added a function to the Gene class that finds a link between a transcript and
  a CDS, based on the protein accession number.

Db.py:
- Added the functions getProtAcc(), getmrnaAcc() and insertLink() for the
  linkage of transcript and CDS.

gbupload.html:
- Changed the orientation input field to a select box.

Mutalyzer.py:
- Implemented the checking of shifting a deletion or insertion over a splice 
  site boundary in a transcript.
- Fixed a bug in the description of a frameshift.



git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/trunk@59 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/branches/lrg@58 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/trunk@56 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/branches/lrg@55 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/trunk@53 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

handling for incomplete and poorly annotated GenBank files.

menu.html:
- Template for all pages. Includes a menu and such.

index.html:
- The main page.

base/:
- Pictures, javascript, stylesheet, etc.

interface.js:
- Temporary script for gbupload.html.

download.html:
- Adjusted to work with the menu.html template.

gbupload.html:
- Adjusted to work with the menu.html template.

check.html:
- Extended the template to use more variables to increase flexibility.

index.py:
- Removed the capturing of the Mutalyzer output in the check() function,
  replaced it with retrieval of data from the Output object. 
- Added a new index() function (see index.html).
- Implemented the upload() function (see gbupload.html), currently it can only
  retrieve genes and chromosome slices from the NCBI.

Web.py:
- Enabled the parsing of the menu.html template in the tal() function.

handler.py:
- Modified to work with the new tal() function of Web.py.

Output.py:
- getMessages() now returns a list of messages, instead of printing it to 
  standard output.

mail.txt:
- Template message for finished batch jobs.

Mutalyzer.py:
- __bprint() now puts the visualisation of a protein in the Output object, 
  instead of printing it to standard output.
- Made a new protein printing function that can highlight the changes, it is
  not used yet.
- Added a __cdsLen() function that is used to see if a variant leads to a
  frameshift or not.
- Added extention of a protein in the protein description function.
- Added some error handling for `?' as a position.
- Added error messages for invalid genes and transcript variants in the 
  input (suggesting possible options).
- DNA / protein descriptions are now stored in the Output object.  
- Added a legend.

GenRecord.py:
- Added functions listLoci() and listGenes() for suggestions when invalid 
  input is given.
- Added a preprocessing step in the GenBank file parsing to deal with 
  missing locus tags (not functional yet).
- Added some error handling for incomplete transcripts.



git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/trunk@52 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

section included in record.

Extracted code to create a GenRecord.Record from a GenBank file from
GenRecord.py and included it in GBparser.py

Updated the Retriever Module to facilitate two different retriever-types;
LRGRetriever and GBRetriever.

Updated the Db module with one method to update a LRG record in the database


NOTE: Ensembl data is parsed but not yet included in the GenRecord.Record,
because there is no clear usage for this. This behaviour can be changed in the
LRGparser.py file


git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/branches/lrg@51 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/branches/lrg@50 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

compatible with the mutalyzer engine at the moment, so it is disabled. 

Added basic unittests to test the validity of Records


git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/branches/lrg@49 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/branches/lrg@46 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/branches/lrg@45 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/branches/lrg@44 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/trunk@42 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

record. Added a legend for transcripts and proteins.

Install.txt:
- Added dependency information.

Mutalyzer.py:
- Rewrote the roll function to roll both ways at the same time.
- Rewrote the trim function (fixed a lcs bug).
- Moved the semantic checking of raw variants to separate functions.
- Removed lots of conversion code. We now convert to g. first and check this
  to all transcripts.
- Added protein descriptions for all transcripts.

GenRecord.py:
- Added more gathering information (locus tags, transcript- and protein 
  identifiers, etc.)
- Made a separate function that checks the gathered information after 
  retrieval. This can be used later on by the LRG parser.
- Modified the name() function to give better descriptions.

Parser.py:
- Added code to distinguish between GenBank records and LRG records.



git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/trunk@41 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/branches/lrg@40 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/branches/lrg@39 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/branches/lrg@38 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/branches/lrg@36 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/trunk@31 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

distribution as it is under heavy development.

Most modules will have minor changes because of a difference in set up of both
the Db and Config module.

install.sh:
- Added functionality to enable the cron restart of the Batch Checker.
- Added the auto-generation of a .htaccess file.
- Added permission settings.

mutalyzer.conf:
- Added configuration options for the Scheduler, File and GenRecord modules.

Db.txt:
- Described how to make the new ChrName tables for hg18 and hg19.

errorcodes.txt:
- Added classifications to the messages.

doc:
- Made a set up for the documentation.
  TechnicalReference:
  - This will be a technical document that describes the internals of the
    project. It is only meant for developers.
  API:
  - This is a description of the API, it is auto generated by the mkapidoc.sh
    script. Also only meant for developers.

Mutalyzer.py:
- Added a new roll function that will always find both boundaries.
- Implemented a new protein naming scheme.
- Fixed the trimming of a delins.
- Rewrote the processing of a variant. 
  - Moved post processing of the GenBank record to the GenRecord module.
  - Moved the crossmapper instance to the GenBank module, to make one instance
    per transcript variant.
  - Moved the naming of a variant to the GenBank module, as is strongly
    interacts with the crossmapper instance.
- Moved the constructCDS function to the GenRecord module.

handler.py:
- Added functionality for the batch checker (retrieve results).
- Added functionality for the genbank uploader (retrieve GenBank files).

webservice.py:
- Modified to work with the new Db module.

UCSC_update.py:
- Modified to work with the new Db module.

GenRecord.py:
- Replaced the dictionary structure with a nested list structure to make
  iteration more convenient.
- Added names to the Locus and Gene objects.
- Added all information needed to do a crossmapping in the Locus object.
- Wrote functions to find Loci and Genes.
- Wrote a function that expands a description of a variant (coupled to a Locus).

Mutator.py:
- Added documentation.

Parser.py:
- Added documentation.

Web.py:
- Added documentation.
- Added a function that checks whether a string is an e-mail address.

Scheduler.py:
- Implemented a batch scheduler that uses a MySQL database for queueing.

File.py:
- Implemented a CSV, XLS and ODS parser for use in the Scheduler module.

Output.py:
- Added documentation.

Mapper.py:
- Modified the complex object initialisation.

Config.py:
- Made subclasses to configure the separate modules.

Db.py:
- Added documentation.
- Split the Db modules into different classes, according to functionality, they
  all inherit the query function from the Db base class.
- Added chromosome accession number to name conversion functions and vice versa.
- Added functionality for the batch checker.

Crossmap.py:
- Added documentation.

Retriever.py:
- Added documentation.
- Added fall back functionality when searching for a gene.

index.py:
- Added a batch submit interface.

batch.html:
- The layout of the batch submit interface.



git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/trunk@30 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

Modified the merged changes to work with the new output module.

Modified the layout of the project.
- Removed the Clients, Interfaces and Services subdirectories.

handler.py, Web.py, index.py:
- Modified for the new layout.

Variant_info.py is renamed to VarInfo.py because of a conflict of names with 
the function Variant_info() in index.py.
VarInfo.py:
- Modified for the new layout.

webservice.py:
- Modified for the new layout and for the new Output module.

Mapper.py:
- Added by the merge and modified for the new Output module.



git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/trunk@27 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

list with priorities, and all output is stored in a dictionary. This dictionary
can be read at a later time.
The Retriever module is changed to accommodate for uploaded GenBank files and
slices of a chromosome.
A Scheduler was added for batch checking.

mutalyzer.conf:
- Added variables for the Retriever module:
  - maxDldSize, minDldSize ; Maximum and minimum sizes for slices and uploaded 
    GenBank files.
- Added variables for the Output module:
  - loglevel, outputlevel ; Specify default verbosity levels for logging and 
    output.
- Added variables for the Scheduler module:
  - processName ; Name of the running scheduler.

Db.txt:
- Added information on how to create the newly used tables GbInfo, BatchQueue
  and BatchJob.

errorcodes.txt:
- Short description of the error codes used in the Output module.

webservice.py:
- Added a Complex class to test more complicated return types (see the web_dev
  branch).
- Modified the code to work with the new Output module.

index.py:
- Made a first upload page.

Mutalyzer.py:
- Modified the code to work with the new Output module.

UCSC_update.py:
- Modified the code to work with the new Output module.

Variant_info.py:
- Modified the code to work with the new Output module.

GenRecord.py:
- Added documentation.

Mutator.py:
- Modified the code to work with the new Output module.

Misc.py:
- New file, used for generating unique IDs.

Parser.py:
- Made a change to the definition of an UD accession number.
- Modified the code to work with the new Output module.

Scheduler.py:
- Made a batch checker scheduler.
  - isDaemonRunning() ; See if we need to be started.
  - process() ; Start the batch checker.
  - addJob() ; Add jobs to a queue in the database.

Output.py:
- Added a Message class to store all debug, info, warning, error and fatal
  messages.
  - If a message is given that exceeds the configured log level, it will be
    logged immediately.
- A function is added to the Output class to read all messages that exceed a 
  certain verbosity level.
- A function is added to create a named list as an output node.
- With the getOutput function the content of this list can be retrieved.

Config.py:
- Several sub-classes were added for each configurable module.

Db.py:
- Added documentation.
- Added functionality that is used by the Retriever module.
- Added functionality that is used by the Scheduler module.

Retriever.py:
- Added functionality to be able to use custom GenBank files and chromosome
  slices.
  - Information on these created files are stored in a database to be able to
    re-create them when the cache is cleaned.
  - The hash of each file is stored for error detection.

BatchChecker.py:
- A wrapper that is called either from the addJob() function from the Scheduler
  module, or from cron. It dispatches a background process that processes the
  batch jobs.

gbupload.html:
- Test template for uploading files (copied from Mutalyzer 1.0.4).

sp.py:
- Some test with a complex return type.

download.html:
- Did some first tests with a METAL template.



git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/trunk@26 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

	changed dbName = "hg18"
In webservice.py
	changed the names of arguments v1, v2, v3 and v4 in all methods to 'more readable' names
	changed C and D to Conf and Database in all methods
	
	method extractChange was added, which extracts from a complete HGVS variant description the part after the coordinates (positions)
	and the start position of the variant
	
	method cTogConversion was added, which converts a complete HGVS variant description in c. notation to g. notation

	method gTocConversion was added, whicht converts a complete HGVS variant description in g. notation to c. notation

In Mapper.py
	method conversionToCoding was added, which converts non-star c. positions to star c.positions
this comment should have been added with the previous commit


git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/branches/web_dev@25 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/branches/web_dev@24 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1

mappingInfo() and transcriptInfo. These methods are used, respectively,
when a variant is present, or when its not.
Mapper.py is the new version of Variant_info.py, with the two methods
described above defined here: mainMapping and mainTranscript
Config.py and Crossmap.py were not actually changed.


git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/branches/web_dev@23 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1