Skip to content
Snippets Groups Projects
Select Git revision
  • dependabot/pip/flask-2.3.2
  • dependabot/pip/redis-4.4.4
  • dependabot/pip/werkzeug-2.2.3
  • dependabot/pip/suds-1.0.0
  • dependabot/pip/lxml-4.9.1
  • master default
  • release-2.0.35
  • release
  • retirement
  • release-2.0.34
  • link-version
  • jfjlaros-patch-2
  • jfjlaros-patch-1
  • update-requirements
  • release-2.0.31
  • nc_db
  • lrg_pending_fix
  • crossmap_module
  • fix-batch-snp-conv
  • drop-reference-gi
  • v2.0.35
  • v2.0.34
  • v2.0.33
  • v2.0.32
  • v2.0.31
  • v2.0.30
  • v2.0.29
  • v2.0.28
  • v2.0.27
  • v2.0.26
  • v2.0.25
  • v2.0.24
  • v2.0.23
  • v2.0.22
  • v2.0.21
  • v2.0.20
  • v2.0.19
  • v2.0.18
  • v2.0.17
  • v2.0.16
40 results
Search by author
  • Any Author
  • Laros Laros j.f.j.laros
  • Mihai Lefter Mihai Lefter mlefter
  • git mirror git mirror gitmirror
  • wgallard wgallard wgallard
4 authors
  1. Oct 20, 2014
    • Vermaat's avatar
      Use unicode strings · 2a4dc3c1
      Vermaat authored 
      Don't fix what ain't broken. Unfortunately, string handling in Mutalyzer
really is broken. So we fix it.

Internally, all strings should be represented by unicode strings as much as
possible. The main exception are large reference sequence strings. These can
often better be BioPython sequence objects, since that is how we usually get
them in the first place.

These changes will hopefully make Mutalyzer more reliable in working with
incoming data. As a bonus, they're a first (small) step towards Python 3
compatibility [1].

Our strategy is as follows:

1. We use `from __future__ import unicode_literals` at the top of every file.
2. All incoming strings are decoded to unicode (if necessary) as soon as
   possible.
3. Outgoing strings are encoded to UTF8 (if necessary) as late as possible.
4. BioPython sequence objects can be based on byte strings as well as unicode
   strings.
5. In the database, everything is UTF8.
6. We worry about uploaded and downloaded reference files and batch jobs in a
   later commit.

Point 1 will ensure that all string literals in our source code will be
unicode strings [2].

As for point 4, sometimes this may even change under our eyes (e.g., calling
`.reverse_complement()` will change it to a byte string). We don't care as
long as they're BioPython objects, only when we get the sequence out we must
have it as unicode string. Their contents are always in the ASCII range
anyway.

Although `Bio.Seq.reverse_complement` works fine on Python byte strings (and
we used to rely on that), it crashes on a Python unicode string. So we take
care to only use it on BioPython sequence objects and wrote our own reverse
complement function for unicode strings (`mutalyzer.util.reverse_complement`).

As for point 5, SQLAlchemy already does a very good job at presenting decoding
from and encoding to UTF8 for us.

The Spyne documentation has the following to say about their `String` and
`Unicode` types [3]:

> There are two string types in Spyne: `spyne.model.primitive.Unicode` and
> `spyne.model.primitive.String` whose native types are `unicode` and `str`
> respectively.
>
> Unlike the Python `str`, the Spyne `String` is not for arbitrary byte
> streams. You should not use it unless you are absolutely, positively sure
> that you need to deal with text data with an unknown encoding. In all other
> cases, you should just use the `Unicode` type. They actually look the same
> from outside, this distinction is made just to properly deal with the quirks
> surrounding Python-2's `unicode` type.
>
> Remember that you have the `ByteArray` and `File` types at your disposal
> when you need to deal with arbitrary byte streams.
>
> The `String` type will be just an alias for `Unicode` once Spyne gets ported
> to Python 3. It might even be deprecated and removed in the future, so make
> sure you are using either `Unicode` or `ByteArray` in your interface
> definitions.

So let's not ignore that and never use `String` anymore in our webservice
interface.

For the command line interface it's a bit more complicated, since there seems
to be no reliable way to get the encoding of command line arguments. We use
`sys.stdin.encoding` as a best guess.

For us to interpret a sequence of bytes as text, it's key to be aware of their
encoding. Once decoded, a text string can be safely used without having to
worry about bytes. Without unicode we're nothing, and nothing will help
us. Maybe we're lying, then you better not stay. But we could be safer, just
for one day. Oh-oh-oh-ohh, oh-oh-oh-ohh, just for one day.

[1] https://docs.python.org/2.7/howto/pyporting.html
[2] http://python-future.org/unicode_literals.html
[3] http://spyne.io/docs/2.10/manual/03_types.html#strings
      2a4dc3c1
  3. Feb 12, 2013
  5. Jan 14, 2013
  7. Oct 05, 2012
  9. Aug 21, 2012
  11. Aug 20, 2012
  13. Aug 04, 2012
  15. Jul 16, 2012
  17. Jul 12, 2012
  19. Feb 20, 2012
  21. Feb 18, 2012
    • Laros's avatar
      Added the Variant Description Extractor as a web interface. · b8640a1f
      Laros authored 
      describe.py:
- Module that provides the Variant Description Extractor functions.

__init__.py:
- Added an automated copyright year update.

website.py:
- Added the Variant Description Extractor web interface.

templates/descriptionExtract.html:
-  Template page for the Variant Description Extractor.

templates/snp.html:
templates/menu.html:
templates/converter.html:
templates/index.html:
templates/parse.html:
- Cosmetic changes.

Added a presentation.



git-svn-id: https://humgenprojects.lumc.nl/svn/mutalyzer/trunk@479 eb6bd6ab-9ccd-42b9-aceb-e2899b4a52f1
      b8640a1f
  23. Feb 01, 2012
  25. Jan 31, 2012
  27. Jan 30, 2012
  29. Jan 29, 2012
  31. Jan 28, 2012
  33. Jan 26, 2012
  35. Jan 22, 2012
  37. Jan 15, 2012
  39. Jan 13, 2012
  41. Sep 07, 2011
  43. Sep 05, 2011
  45. Aug 19, 2011
  47. Jul 27, 2011
  49. Jul 26, 2011
  51. Jul 25, 2011
  53. Jul 21, 2011
  55. Jul 12, 2011
  57. Apr 12, 2011
  59. Apr 08, 2011
  61. Apr 05, 2011