1. 08 Mar, 2017 1 commit
    • Hoogenboom, Jerry's avatar
      FDSTools v1.1.0.dev3: Fixes and pipelining enhancements · be8dbe46
      Hoogenboom, Jerry authored
      * General changes in v1.1.0.dev3:
        * Allele name heuristics: don't produce insertions at the end of the
          prefix or at the beginning of the suffix; just include extra STR
          blocks.
        * FDSTools will no longer crash with a 'column not found' error when
          an input file is empty. This situation is now treated as if the
          expected columns existed, but no lines of actual data were present.
          This greatly helps in tracking down issues in pipelines involving
          multiple tools, as tools will now shutdown gracefully if an upstream
          tool fails to write output.
      * Allelefinder v1.0.1:
        * Fixed crash that occurred when converting sequences to allele names
          format while no library file was provided.
        * Don't crash when output pipe is closed.
      * BGAnalyse v1.0.1:
        * Don't crash when output pipe is closed.
      * BGCorrect v1.0.2:
        * Don't crash on empty input files.
        * Don't crash when output pipe is closed.
      * BGEstimate v1.1.2:
        * Don't crash when output pipe is closed.
      * BGHomRaw v1.0.1:
        * Clarified the 'Allele x of marker y has 0 reads' error message with
          the sample tag.
        * Don't crash when output pipe is closed.
      * BGHomStats v1.0.1:
        * Error messages about the input data now contain the sample tag of
          the sample that triggered the error.
        * Don't crash when output pipe is closed.
      * BGMerge v1.0.3:
        * Don't crash when output pipe is closed.
      * BGPredict v1.0.2:
        * Don't crash on empty input files.
        * Don't crash when output pipe is closed.
      * FindNewAlleles v1.0.1:
        * Don't crash on empty input files.
        * Don't crash when output pipe is closed.
      * Libconvert v1.1.2:
        * Don't crash when output pipe is closed.
      * Library v1.0.3:
        * Don't crash when output pipe is closed.
      * Seqconvert v1.0.2:
        * Don't crash when output pipe is closed.
      * Samplestats v1.1.1:
        * Don't crash on empty input files.
        * Don't crash when output pipe is closed.
      * Stuttermark v1.5.1:
        * Don't crash on empty input files.
        * Don't crash when output pipe is closed.
      * Stuttermodel v1.1.2:
        * Don't crash when output pipe is closed.
      * TSSV v1.1.0 (additionally):
        * When running analysis in parallel, make tasks of 1 million alignments.
          Previously, this was 10k reads, with the number of alignments per task
          depending on the size of the library file. This caused memory issues for
          huge libraries like whole mt interval libraries.
        * Don't crash when output pipe is closed.
      * Vis v1.0.4:
        * Don't crash when output pipe is closed.
      be8dbe46
  2. 20 Sep, 2016 1 commit
    • Hoogenboom, Jerry's avatar
      Developing towards v0.0.6 (v0.0.6.dev1) · 5a2addb7
      Hoogenboom, Jerry authored
      * General changes in v0.0.6.dev1:
        * Tools that take a list of files as their argument (through the -i option or
          as positionals) now explicitly support glob patterns.  This means they will
          interpret '*' and '?' characters as wildcards for 'zero or more characters'
          and 'any one character', respectively.  On Unix-like systems this is
          generally done by the shell, but on Windows one had to specify every file
          name completely.
      * BGEstimate v1.1.1:
        * Added option -p/--profiles which can be used to provide a previously
          created background noise profiles file.  BGEstimate will read starting
          values from this file instead of assuming zero noise.
      * BGMerge v1.0.2:
        * Small code changes to facilitate explicit glob pattern matching support.
      * Pipeline v1.0.1:
        * The Pipeline tool will no longer check the existence of the files specified
          for the -S/--in-samples option; instead, this is left to the downstream
          tools to find out, consistent with how this works with the other input file
          options.
      * Allelevis v2.0.1:
        * Added tooltip support to HTML visualisations.  Moving the mouse pointer
          over a node or edge in the graph now displays a tooltip giving allele names
          and sample counts.
      * Stuttermodelvis v2.0.1:
        * Changed the unit in the horizontal axis title from 'bp' to 'nt'.
      * Library v1.0.1:
        * Updated some of the comments describing the sections.
      5a2addb7
  3. 26 Jul, 2016 1 commit
    • Hoogenboom, Jerry's avatar
      Various bug fixes and refinements throughout FDSTools · 08cf6ddd
      Hoogenboom, Jerry authored
      * Global changes in v0.0.4:
        * FDSTools will now print profiling information to stdout when the -d/--debug
          option was specified.
        * Fixed bug where specifying '-' as the output filename would be taken
          literally, while it should have been interpreted as 'write to standard out'
          (Affected tools: BGCorrect, Samplestats, Seqconvert, Stuttermark).
        * Added more detailed license information to FDSTools.
      * BGEstimate v1.1.0:
        * Added a new option -g/--min-genotypes (default: 3). Only alleles that occur
          in at least this number of unique heterozygous genotypes will be
          considered. This is to avoid 'contamination' of the noise profile of one
          allele with the noise of another. If homozygous samples are available for
          an allele, this filter is not applied to that allele. Setting this option
          to 1 effectively disables it. This option has the same cascading effect as
          the -s/--min-samples option, that is, if one allele does not meet the
          threshold, the samples with this allele are excluded which may cause some
          of the other alleles of these samples to fall below the threshold as well.
      * Stuttermodel v1.1.0:
        * Stuttermodel will now only output a fit for one strand if it could also
          obtain a fit for the other strand (for the same marker, unit, and stutter
          depth). This new behaviour can be disabled with a new -O/--orphans option.
        * Fixed bug that caused Stuttermodel to output only the raw data points for
          -1 and +1 stutter when normal output was supressed.
      * BGCorrect v1.0.1:
        * Added new column 'weight' to the output. The value in this column expresses
          the number of times that the noise profile of that allele fitted in the
          sample.
      * Samplestats v1.0.1:
        * Samplestats will now round to 4 or 5 significant digits if a value is
          above 1000 or 10000, respectively. Previously, this was only done for the
          combined 'Other sequences' values.
        * The 'Other sequences' lines will now also include values for
          total_recovery, forward_recovery, and reverse_recovery.
        * The total_recovery, forward_recovery, and reverse_recovery columns are no
          longer placed to the left of all the other columns generated by
          Samplestats.
        * The help text for Samplestats erroneously listed the X_recovery_pct instead
          of X_recovery.
        * Added support for the new 'weight' column produced by BGCorrect when the
          -a/--filter-action option is set to 'combine'.
      * BGPredict v1.0.1:
        * Greatly reduced memory usage.
        * BGPredict will now output nonzero values below the threshold set by
          -n/--min-pct if the predicted noise ratio of the same stutter on the other
          strand is above the threshold. Previously, values below the threshold were
          clipped to zero, which may cause unnecessarily high strand bias in the
          predicted profile.
      * BGMerge v1.0.1:
        * Reduced memory usage.
      * TSSV v1.0.1:
        * Renamed the '--is_fastq' option to '--is-fastq'. It was the only option
          with an underscore instead of a hyphen in FDSTools.
        * Fixed crash that would occur if -F/--sequence-format was set to anything
          other than 'raw'.
      * Libconvert v1.0.1:
        * Specifying '-' as the first positional argument to libconvert will now
          correctly interpret this as "read from stdin" instead of throwing a "file
          not found" error (or reading from a file named "-" if it exists).
      * Seqconvert v1.0.1:
        * Internal naming of the first positional argument was changed from 'format'
          to 'sequence-format'. This was done for consistency with the
          -F/--sequence-format option in other tools, giving it the same name in
          Pipeline configuration files.
      * Vis v1.0.1:
        * Added -j/--jitter option for Stuttermodelvis (default: 0.25).
        * Vis would not allow the -n/--min-abs and the -s/--min-per-strand options to
          be set to 0.
      * Stuttermodelvis v1.0.0beta2:
        * HTML visualisations now support drawing raw data points on top of the fit
          functions. The points can be drawn with an adjustable jitter to reduce
          overlap.
        * Fixed a JavaScript crash that would occur in HTML visualisations if the
          Repeat unit or Marker name filter resulted in an invalid regular expression
          (e.g., when the entered value ends with a backslash).
        * Reduced Vega graph spec complexity by using the new Rank transform to
          position the subgraphs.
        * HTML visualisations made with the -O/--online option of the Vis tool will
          now contain https URLs instead of http.
      * Samplevis v1.0.1:
        * Fixed a JavaScript crash that would occur in HTML visualisations if the
          Marker name filter resulted in an invalid regular expression (e.g., when
          the entered value ends with a backslash).
        * Reduced Vega graph spec complexity by using the new Rank transform to
          position the subgraphs.
        * Fixed a glitch where clicking the 'Truncate sequences to' label would
          select the marker spacing input.
        * The 'Notes' table cells with 'BGPredict' in them now get a light orange
          background to warn the user that their background profile was computed.
          If a sequence was explicitly 'not corrected', 'not in ref db', or
          'corrected as background only', the same colour is used.
        * The message bar at the bottom of Samplevis HTML visualisations will now
          grow no larger than 3 lines. A scroll bar will appear as needed.
        * HTML visualisations made with the -O/--online option of the Vis tool will
          now contain https URLs instead of http.
      * BGRawVis v1.0.1:
        * Fixed a JavaScript crash that would occur in HTML visualisations if the
          Marker name filter resulted in an invalid regular expression (e.g., when
          the entered value ends with a backslash).
        * Reduced Vega graph spec complexity by using the new Rank transform to
          position the subgraphs.
        * HTML visualisations made with the -O/--online option of the Vis tool will
          now contain https URLs instead of http.
      * Profilevis v1.0.1:
        * Fixed a JavaScript crash that would occur in HTML visualisations if the
          Marker name filter resulted in an invalid regular expression (e.g., when
          the entered value ends with a backslash).
        * Reduced Vega graph spec complexity by using the new Rank transform to
          position the subgraphs.
        * HTML visualisations made with the -O/--online option of the Vis tool will
          now contain https URLs instead of http.
      * Allelevis v1.0.0beta2:
        * Fixed potential crash/corruption that could occur with very unfortunate
          combinations of sample names and marker names.
        * HTML visualisations made with the -O/--online option of the Vis tool will
          now contain https URLs instead of http.
        * Added two more colours to the legend, such that a maximum of 22 markers is
          now supported without re-using colours.
      * Updated bundled D3 to v3.5.17.
      * Updated bundled Vega to v2.6.0.
      08cf6ddd
  4. 08 Mar, 2016 1 commit
    • Hoogenboom, Jerry's avatar
      Version numbering for everything · da8d1d1b
      Hoogenboom, Jerry authored
      * Turned all '0.1dev' tool version numbers to '1.0.0'.
      * Changed Stuttermark's version number from '1.5' to '1.5.0'.
      * Added version numbers to the visualisations.
      * Updated README.rst to include all tools, but removed the usage details
        of Stuttermark because it is highly impractical to include usage
        details for all tools in the README file. I'll leave that to the
        -h/--help option and the yet-to-write FDSTools User's Handbook.
      da8d1d1b
  5. 18 Jan, 2016 1 commit
    • Hoogenboom, Jerry's avatar
      Various fixes and improvements · 4f9286e4
      Hoogenboom, Jerry authored
      Fixed:
      * Fixed a crash in BGMerge.
      * Fixed bug in BGCorrect that resulted in incorrect values in the
        *_add and *_corrected columns (yes, you, 8685a304).
      * Fixed a glitch in BGCorrect that prevented it from ever writing
        corrected_bgestimate in the correction_flags column.
      
      Improved:
      * BGEstimate will now include the sample tag in the error messages for
        missing alleles and alleles with 0 reads.
      * Strand bias lines in Samplevis are now clamped to the 0-100% range.
        BGCorrect may cause forward read percentages outside this range.
      
      Visualisations:
      * Updated Vega to version 2.4.2.
      * Fixed drag-'n-drop behaviour for HTML visualisations in Internet
        Explorer and Firefox.
      * Fixed the Save Image link when viewing HTML visualisations in
        Internet Explorer 10 and above.
      * Added http-equiv="X-UA-Compatible" content="IE=edge" meta-tag to all
        visualisations to prevent Internet Explorer from entering quirks mode.
      * Samplevis:
        * Fixed glitch that would sometimes cause a second horizontal scroll
          bar to appear.
        * Graphs now render much more quickly when 'Split markers' is on, and
          Chrome no longer crashes on large sample files with this option set.
      4f9286e4
  6. 16 Nov, 2015 1 commit
    • Hoogenboom, Jerry's avatar
      Various fixes and improvements · 313867bc
      Hoogenboom, Jerry authored
      Fixed:
      * The 'to' base in variants called on mtDNA was incorrect. This bug could also cause FDSTools to crash.
      * FDSTools would crash if you tried to generate an allele name for a primer dimer of an mtDNA marker. (Now, you get an insane but entirely accurate allele name instead.)
      * Fixed bug that caused some perfectly valid mtDNA allele names to be rejected when attempting to convert them back to raw sequences.
      
      Improved:
      * You can now also specify the ending position of the markers in the FDSTools library. If you do, you may also additionally specify a second start position (and optionally also a second end position, and so on). FDSTools will interpret this as that the marker is the concatenation of each of these fragments. This was primarily introduced to support mtDNA fragments that contain (somewhere in the middle) the origin of mtDNA base numbering.
      * More helpful error message when format violations are detected while parsing the library file.
      * More helpful error message when the -e/--tag-expr regular expression could not be compiled.
      * Added a paragraph about sequence alignment caching to the help text of Seqconvert.
      * Added a 'flags' column to BGCorrect output, which gives information about the data that was used to do the correction.
      
      Background noise profiles:
      * Removed -C/--cross-tabular option from BGEstimate, BGPredict, and BGMerge and also removed the ability to read files in this format.
      * BGEstimate, BGHomStats, and BGPredict now add a column 'tool' with their name to the output.
      313867bc
  7. 04 Nov, 2015 1 commit
    • Hoogenboom, Jerry's avatar
      Implemented support for non-STR markers, improved file handling and more · 1083919c
      Hoogenboom, Jerry authored
      Additions and improvements to the FDSTools library file format:
      * New [genome_position] section in FDSTools-style library files allows
      for specifying the chromosome and position of each marker.
      * New [no_repeat] section in FDSTools-style library files allows for
      including non-STR markers.
      * Comma/semicolon/space-separated values in FDSTools-style library files
      can now also be separated by tab characters and multiple consecutive
      separators are no longer collapsed (with the exception of whitespace).
      * If no prefix and/or suffix has been specified for an alias, the
      prefix/suffix of the marker itself is used.
      * Implemented support for non-STR markers (e.g. SNP clusters) and mtDNA
      markers. Allele names of the latter follow mtDNA nomenclature.
      * Improved the logic of generating STR allele names for sequences that
      have a prefix or suffix sequence that was not included in the library
      file.
      * Updated and clarified various explanatory texts in generated FDSTools
      library files.
      
      Fixed:
      * Fixed a bug that caused prefix/suffix variants in aliases to go
      missing in allele names.
      
      Improved file handling:
      * Library files are now closed immediately after parsing them.
      * Sample data input files are opened one at a time now.
      
      Visualisations:
      * Updated Vega to version 2.3.1.
      * Worked around a bug in Google Chrome that caused the 'Save image' link
      to stop working after having been used once.
      1083919c
  8. 01 Sep, 2015 1 commit
    • jhoogenboom's avatar
      Cleanup and minor enhancements · 03fc3d49
      jhoogenboom authored
      * BGCorrect and Stuttermark will now exit with an error message if
        more than one input file for the same sample is specified and no
        separate output files are given. Previously these tools would
        just overwrite the output file repeatedly, discarding the output
        of all but the last data file of the sample.
      * Removed to main() functions and related stubs from the tools
        because they are not actually runnable directly anyway.
      * Added some more help text to some of the tools.
      * Doubled the size of the marker name filter input element on the
        HTML visualisations.
      03fc3d49
  9. 12 Aug, 2015 1 commit
    • jhoogenboom's avatar
      Introducing BGMerge · 6207d485
      jhoogenboom authored
      * New tool BGMerge can be used to merge background noise profiles
        (e.g., merge BGPredict output with a database previously
        obtained from BGEstimate).
      * Fixed two major bugs in BGPredict that resulted in incorrect fit
        functions being used.
      * BGEstimate, BGPredict, BGHomStats, Blame, and StutterModel no
        longer crash if a library file is specified.
      * Added reverse strand profile estimation to BGPredict.
      6207d485