Skip to content
Snippets Groups Projects
Select Git revision
  • dependabot/pip/flask-2.3.2
  • dependabot/pip/redis-4.4.4
  • dependabot/pip/werkzeug-2.2.3
  • dependabot/pip/suds-1.0.0
  • dependabot/pip/lxml-4.9.1
  • master default
  • release-2.0.35
  • release
  • retirement
  • release-2.0.34
  • link-version
  • jfjlaros-patch-2
  • jfjlaros-patch-1
  • update-requirements
  • release-2.0.31
  • nc_db
  • lrg_pending_fix
  • crossmap_module
  • fix-batch-snp-conv
  • drop-reference-gi
  • v2.0.35
  • v2.0.34
  • v2.0.33
  • v2.0.32
  • v2.0.31
  • v2.0.30
  • v2.0.29
  • v2.0.28
  • v2.0.27
  • v2.0.26
  • v2.0.25
  • v2.0.24
  • v2.0.23
  • v2.0.22
  • v2.0.21
  • v2.0.20
  • v2.0.19
  • v2.0.18
  • v2.0.17
  • v2.0.16
40 results
Search by author
  • Any Author
  • Laros Laros j.f.j.laros
  • Mihai Lefter Mihai Lefter mlefter
  • git mirror git mirror gitmirror
  • wgallard wgallard wgallard
4 authors
  1. Oct 13, 2015
  3. Jul 09, 2015
  5. May 31, 2015
  7. May 18, 2015
  9. Nov 24, 2014
  11. Oct 21, 2014
  13. Oct 20, 2014
    • Vermaat's avatar
      Use unicode strings · 2a4dc3c1
      Vermaat authored 
      Don't fix what ain't broken. Unfortunately, string handling in Mutalyzer
really is broken. So we fix it.

Internally, all strings should be represented by unicode strings as much as
possible. The main exception are large reference sequence strings. These can
often better be BioPython sequence objects, since that is how we usually get
them in the first place.

These changes will hopefully make Mutalyzer more reliable in working with
incoming data. As a bonus, they're a first (small) step towards Python 3
compatibility [1].

Our strategy is as follows:

1. We use `from __future__ import unicode_literals` at the top of every file.
2. All incoming strings are decoded to unicode (if necessary) as soon as
   possible.
3. Outgoing strings are encoded to UTF8 (if necessary) as late as possible.
4. BioPython sequence objects can be based on byte strings as well as unicode
   strings.
5. In the database, everything is UTF8.
6. We worry about uploaded and downloaded reference files and batch jobs in a
   later commit.

Point 1 will ensure that all string literals in our source code will be
unicode strings [2].

As for point 4, sometimes this may even change under our eyes (e.g., calling
`.reverse_complement()` will change it to a byte string). We don't care as
long as they're BioPython objects, only when we get the sequence out we must
have it as unicode string. Their contents are always in the ASCII range
anyway.

Although `Bio.Seq.reverse_complement` works fine on Python byte strings (and
we used to rely on that), it crashes on a Python unicode string. So we take
care to only use it on BioPython sequence objects and wrote our own reverse
complement function for unicode strings (`mutalyzer.util.reverse_complement`).

As for point 5, SQLAlchemy already does a very good job at presenting decoding
from and encoding to UTF8 for us.

The Spyne documentation has the following to say about their `String` and
`Unicode` types [3]:

> There are two string types in Spyne: `spyne.model.primitive.Unicode` and
> `spyne.model.primitive.String` whose native types are `unicode` and `str`
> respectively.
>
> Unlike the Python `str`, the Spyne `String` is not for arbitrary byte
> streams. You should not use it unless you are absolutely, positively sure
> that you need to deal with text data with an unknown encoding. In all other
> cases, you should just use the `Unicode` type. They actually look the same
> from outside, this distinction is made just to properly deal with the quirks
> surrounding Python-2's `unicode` type.
>
> Remember that you have the `ByteArray` and `File` types at your disposal
> when you need to deal with arbitrary byte streams.
>
> The `String` type will be just an alias for `Unicode` once Spyne gets ported
> to Python 3. It might even be deprecated and removed in the future, so make
> sure you are using either `Unicode` or `ByteArray` in your interface
> definitions.

So let's not ignore that and never use `String` anymore in our webservice
interface.

For the command line interface it's a bit more complicated, since there seems
to be no reliable way to get the encoding of command line arguments. We use
`sys.stdin.encoding` as a best guess.

For us to interpret a sequence of bytes as text, it's key to be aware of their
encoding. Once decoded, a text string can be safely used without having to
worry about bytes. Without unicode we're nothing, and nothing will help
us. Maybe we're lying, then you better not stay. But we could be safer, just
for one day. Oh-oh-oh-ohh, oh-oh-oh-ohh, just for one day.

[1] https://docs.python.org/2.7/howto/pyporting.html
[2] http://python-future.org/unicode_literals.html
[3] http://spyne.io/docs/2.10/manual/03_types.html#strings
      2a4dc3c1
  15. Oct 15, 2014
    • Vermaat's avatar
      Fix several error cases in LOVD2 getGS call · bcef1633
      Vermaat authored 
      The `getGS` website view for LOVD2 would report "transcript not found" if
the genomic reference has multiple transcripts annotated or if the variant
description raises an error in the variant checker.
      bcef1633
  17. Sep 22, 2014
  19. Aug 27, 2014
  21. Feb 17, 2014
  23. Jan 22, 2014
    • Vermaat's avatar
      Use fixtures in the unit tests · c49d49f0
      Vermaat authored 
      This is The Good Stuff. The entire test suite can now be run without
having to setup a database, running the batch checker, any of the web
services or the website. It even passes without an internet connection.
In, like, 30 seconds! Awesome!

This means tests don't randomly fail after some reference sequence
changes on the NCBI server and it doesn't take an entire configured
server with mapping database setup to run the tests. Those are things
of the past! No more frustrations, Mutalyzer is testable!

Going down now...

The mountain screamed three times today
I guess it thought it'd like to play
How much does one have to pay
To fry a peak and melt away
Launching titan's breath on mine
The sweating measure lands on time

And the old man, down by the river
Well he walks up and he walks on down
To the spaceship that's parked at your doorstep
And it's waiting to take you away now

Goin' down now
Goin' down now

Looking for the rate that crowed
He's hooked up down in Mexico
Slap my nerve now give me more
It's my disaster friend, not yours

And the old man, down by the river
Well he walks up and he walks on down
To the spaceship that's parked at your doorstep
And it's waiting to take you away now

And the last one, it's down by the river
Where he gets up and he walks on down
To the spaceship that's parked at your doorstep
And it's waiting to take you away now

It's down by the river, it's always this way now
It's down by the river, it's always this way now

Going down now
Going down now
now, now, now

down, down, down
      c49d49f0
  25. Jan 10, 2014
    • Vermaat's avatar
      Port Mapping database module to SQLAlchemy · e9bf1bc9
      Vermaat authored 
      This introduces a proper notion of genome assemblies. Transcript
mappings for alle genome assemblies are in the same database, which
is better for maintenance. Updating transcript mappings is also
simplified a lot, especially from NCBI mapview files where we now
require a preprocessing sort on the input file.

Overall, this port touches a lot of Mutalyzer code, so beware.
      e9bf1bc9
  27. Jan 04, 2014
  29. Dec 23, 2013
    • Vermaat's avatar
      Fix unit tests with SQLAlchemy · 94df7c07
      Vermaat authored 
      This involves making the SQLAlchemy session reconfigurable at run-time,
which is done automatically on updating the Mutalyzer configuration using
configuration update callbacks.
      94df7c07
  31. Dec 19, 2013
  33. Dec 13, 2013
  35. Sep 18, 2013
  37. Jan 14, 2013
  39. Oct 05, 2012
  41. Aug 21, 2012
  43. Jul 11, 2012
  45. Jun 07, 2012
  47. May 30, 2012
  49. May 09, 2012
    • Vermaat's avatar
      Refactor name checker to accept GET requests · d5f1e49d
      Vermaat authored 
      Until now, the name checker form used POST requests. As a special case for
linking from LOVD, GET requests were handled by showing results without the
full interface (header, menu, etc). Ordinary linking was done to a separate
checkForward location which then set a cookie with the variant name and a
redirect to the name checker results. This was all very hackish and somewhat
broken (see Track issue #94).

This commit refactors the name checker to use GET requests, so ordinary
bookmarking of result pages is possible. All old entrypoints are handled with
a redirect for backwards compatibility.

Worth mentioning is that variant descriptions in the name checker are now
limited in length by the maximum query string length. However, this maximum
is several thousand characters with Internet Explorer having the lowest
maximum of just over 2000. Longer descriptions are not practically checked
with the name checker web interface anyway, so this should not be a problem.


git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/trunk@521 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1
      d5f1e49d
  51. Feb 20, 2012
  53. Jan 26, 2012
  55. Jan 08, 2012
  57. Nov 24, 2011
  59. Nov 10, 2011
  61. Oct 21, 2011
  63. Oct 10, 2011
  65. Sep 29, 2011
  67. Sep 27, 2011
  69. Sep 20, 2011