diff --git a/more_python/03_more_python_goodness_1.ipynb b/more_python/03_more_python_goodness_1.ipynb
new file mode 100644
index 0000000000000000000000000000000000000000..fed127b7737b5afc7c750d115064263b649d908b
--- /dev/null
+++ b/more_python/03_more_python_goodness_1.ipynb
@@ -0,0 +1,941 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# More Python Goodness (1)\n",
+ "***"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Table of contents\n",
+ "\n",
+ "1. [Working with scripts](#scripts)\n",
+ "2. [The standard library](#stdlib)\n",
+ "3. [String methods](#stringmethods)\n",
+ "4. [Comments and docstrings](#docstrings)\n",
+ "5. [Detour: PEP8 and other PEPs](#peps)\n",
+ "6. [Errors and exceptions](#exceptions)\n",
+ "7. Working with modules\n",
+ "8. Examples from the standard library\n",
+ "9. Reading and writing files\n",
+ "10. Assignment: Finding the most common 7-mer in a FASTA file\n",
+ "11. Further reading"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "<a id=\"scripts\"></a>\n",
+ "## Working with scripts\n",
+ "\n",
+ "Interpreters are great for *prototyping*, but not really suitable if you want to **share** or **release** code. To do so, we write our Python commands in scripts (and later, modules).\n",
+ "\n",
+ "A **script** is a simple text file containing Python instructions to execute."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Executing scripts\n",
+ "\n",
+ "There are two common ways to execute a script:\n",
+ "\n",
+ "1. As an argument of the Python interpreter command.\n",
+ "2. As a standalone executable (with the appropriate shebang line & file mode).\n",
+ "\n",
+ "IPython gives you a third option:\n",
+ "\n",
+ "<ol start=\"3\">\n",
+ " <li>As an argument of the `%run` magic.\n",
+ "</ol>"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Writing your script\n",
+ "\n",
+ "Let's start with a simple GC calculator. Open your text editor, and write the following Python statements (remember your indentations):\n",
+ "```python\n",
+ "def calc_gc_percent(seq):\n",
+ " at_count, gc_count = 0, 0\n",
+ " for char in seq:\n",
+ " if char in ('A', 'T'):\n",
+ " at_count += 1\n",
+ " elif char in ('G', 'C'):\n",
+ " gc_count += 1\n",
+ " \n",
+ " return gc_count * 100.0 / (gc_count + at_count) \n",
+ "\n",
+ "print \"The sequence 'CAGG' has a %GC of {:.2f}\".format(\n",
+ " calc_gc_percent(\"CAGG\"))\n",
+ "```\n",
+ "Save the file as `seq_toolbox.py` (you can use any other name if you like) and go to your shell."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Running the script\n",
+ "\n",
+ "Let's try the first method: using your script as an argument:\n",
+ "\n",
+ " $ python seq_toolbox.py\n",
+ "\n",
+ "Is the output as you expect?\n",
+ "\n",
+ "For the second method, we need to do two more things:\n",
+ "\n",
+ "1. Open the script in your editor and add the following line to the very top:\n",
+ "\n",
+ " #!/usr/bin/env python\n",
+ "\n",
+ "2. Save the file, go back to the shell, and allow the file to be executed:\n",
+ "\n",
+ " $ chmod +x seq_toolbox.py\n",
+ "\n",
+ "You can now execute the file directly:\n",
+ "\n",
+ " $ ./seq_toolbox.py\n",
+ "\n",
+ "Is the output the same as the previous method?\n",
+ "\n",
+ "Finally, try out the third method. Open an IPython interpreter session and do:\n",
+ "\n",
+ " %run seq_toolbox.py"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "<a id=\"stdlib\"></a>\n",
+ "## The standard library\n",
+ "\n",
+ "Our script is nice and dandy, but we don't want to edit the source file everytime we calculate a sequence's GC.\n",
+ "\n",
+ "The **standard library** is a collection of Python modules (or functions, for now) that comes packaged with a default Python installation. They're not part of the language per se, more like a *batteries included* thing."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Our first standard library module: `sys`\n",
+ "\n",
+ "We'll start by using the simple `sys` module to make our script more flexible.\n",
+ "\n",
+ "Standard library (and other modules, as we'll see later) can be used via the `import` statement, for example:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 1,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import sys"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Like other objects so far, we can peek into the documentation of these modules using `help`, or the IPython `?` shortcut. For example:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "sys?"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### The `sys.argv` list\n",
+ "\n",
+ "The `sys` module provides a way to capture command line arguments with its `argv` object. This is a list of arguments supplied when invoking the current Python session. Not really useful for an interpreter session, but very handy for scripts."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 3,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "['-c',\n",
+ " '-f',\n",
+ " '/home/martijn/.ipython/profile_default/security/kernel-9535cbe5-e69a-4c48-91f2-80a022c362b9.json',\n",
+ " \"--IPKernelApp.parent_appname='ipython-notebook'\",\n",
+ " '--profile-dir',\n",
+ " '/home/martijn/.ipython/profile_default',\n",
+ " '--parent=1']"
+ ]
+ },
+ "execution_count": 3,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "sys.argv"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 4,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "['-c',\n",
+ " '-f',\n",
+ " '/home/martijn/.ipython/profile_default/security/kernel-9535cbe5-e69a-4c48-91f2-80a022c362b9.json']"
+ ]
+ },
+ "execution_count": 4,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "sys.argv[:3]"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Improving our script with `sys.argv`\n",
+ "\n",
+ "To use `sys.argv` in our script, open a text editor and edit the script by adding an import statement, capturing the `sys.argv` value, and editing our last `print` line:\n",
+ "\n",
+ "```python\n",
+ " #!/usr/bin/env python\n",
+ " import sys\n",
+ "\n",
+ " def calc_gc_percent(seq):\n",
+ " at_count, gc_count = 0, 0\n",
+ " for char in seq:\n",
+ " if char in ('A', 'T'):\n",
+ " at_count += 1\n",
+ " elif char in ('G', 'C'):\n",
+ " gc_count += 1\n",
+ "\n",
+ " return gc_count * 100.0 / (gc_count + at_count) \n",
+ "\n",
+ " input_seq = sys.argv[1]\n",
+ " print \"The sequence '{}' has a %GC of {:.2f}\".format(\n",
+ " input_seq, calc_gc_percent(input_seq))\n",
+ "```\n",
+ "To test it, you can run the following command in your shell:\n",
+ "\n",
+ " $ python seq_toolbox.py CAGG\n",
+ "\n",
+ "Try it with `./seq_toolbox.py` instead. What happens?"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "<a id=\"stringmethods\"></a>\n",
+ "## String methods\n",
+ "\n",
+ "Try running the script with `'cagg'` as the input sequence. What happens?\n",
+ "\n",
+ "As we saw earlier, many objects, like those of type `list`, `dict`, or `str`, have useful methods defined on them. One way to squash this potential bug is by using Python's string method `upper`. Let's first check out some commonly used string functions.\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 5,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "my_str = 'Hello again, ipython!'"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 6,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "'HELLO AGAIN, IPYTHON!'"
+ ]
+ },
+ "execution_count": 6,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "my_str.upper()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 7,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "'hello again, ipython!'"
+ ]
+ },
+ "execution_count": 7,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "my_str.lower()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 8,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "'Hello Again, Ipython!'"
+ ]
+ },
+ "execution_count": 8,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "my_str.title()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 9,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "True"
+ ]
+ },
+ "execution_count": 9,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "my_str.startswith('H')"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 10,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "False"
+ ]
+ },
+ "execution_count": 10,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "my_str.startswith('h')"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 11,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "['Hello again', ' ipython!']"
+ ]
+ },
+ "execution_count": 11,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "my_str.split(',')"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 12,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "'Hello again, lumc!'"
+ ]
+ },
+ "execution_count": 12,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "my_str.replace('ipython', 'lumc')"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 13,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "2"
+ ]
+ },
+ "execution_count": 13,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "my_str.count('n')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Improving our script with `upper()`\n",
+ "\n",
+ "Let's use `upper()` to fortify our function. It should now look something like this:\n",
+ "```python\n",
+ " def calc_gc_percent(seq):\n",
+ " at_count, gc_count = 0, 0\n",
+ " for char in seq.upper():\n",
+ " if char in ('A', 'T'):\n",
+ " at_count += 1\n",
+ " elif char in ('G', 'C'):\n",
+ " gc_count += 1\n",
+ "\n",
+ " return gc_count * 100.0 / (gc_count + at_count) \n",
+ "```\n",
+ "And run it (in whichever way you prefer). Do you get the expected output?"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "<a id=\"docstrings\"></a>\n",
+ "## Comments and docstrings\n",
+ "\n",
+ "There's a golden rule in programming: write code for humans (this includes you in 6 months). Python provides two ways to accomplish this: comments and docstrings.\n",
+ "\n",
+ "### Comments\n",
+ "\n",
+ "Any lines prepended with `#` are **comments**, making them ignored by the interpreter. Comments can be freeform text; anything that helps in understanding the code\n",
+ "\n",
+ "### Docstrings\n",
+ "\n",
+ "**Docstrings** are Python's way of attaching proper documentation to objects. Officially, the first string literal that occurs in a module, function, class, or method definition is used as that object's docstring.\n",
+ "\n",
+ "In practice, *triple-quoted strings* are used, to handle newlines easier.\n",
+ "\n",
+ "Remember how we used the `help` function (or IPython's `?` shortcut) to get information about an object, function, or module? This actually prints that object's docstring."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Improving our script with comments and docstrings\n",
+ "\n",
+ "Open your script again in a text editor, and add the following comments and docstrings: \n",
+ "```python\n",
+ " #!/usr/bin/env python\n",
+ " import sys\n",
+ "\n",
+ " def calc_gc_percent(seq):\n",
+ " \"\"\"\n",
+ " Calculates the GC percentage of the given sequence.\n",
+ "\n",
+ " Arguments:\n",
+ " - seq - the input sequence (string).\n",
+ "\n",
+ " Returns:\n",
+ " - GC percentage (float).\n",
+ "\n",
+ " The returned value is always <= 100.0\n",
+ " \"\"\"\n",
+ " at_count, gc_count = 0, 0\n",
+ " # Change input to all caps to allow for non-capital\n",
+ " # input sequence.\n",
+ " for char in seq.upper():\n",
+ " if char in ('A', 'T'):\n",
+ " at_count += 1\n",
+ " elif char in ('G', 'C'):\n",
+ " gc_count += 1\n",
+ "\n",
+ " return gc_count * 100.0 / (gc_count + at_count) \n",
+ "\n",
+ " input_seq = sys.argv[1]\n",
+ " print \"The sequence '{}' has a %GC of {:.2f}\".format(\n",
+ " input_seq, calc_gc_percent(input_seq))\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "<a id=\"peps\"></a>\n",
+ "## Detour: PEP8 and other PEPs\n",
+ "\n",
+ "Since comments and docstrings are basically free-form text, whether it's useful or not depends heavily on the developer. To mitigate this, the Python community has come up with practical conventions. They are documented in a document called **PEP8**.\n",
+ "\n",
+ "Complementary to PEP8, there is **PEP257** which is for docstrings specifically. It's not a must to follow these conventions, but *very* encouraged to do so.\n",
+ "\n",
+ "Python Enhancement Proposals, or **PEP**s, are how Python grows. There are hundreds of them now, all have to be approved by our BDFL.\n",
+ "\n",
+ "> [PEP8: Style Guide for Python Code](http://www.python.org/dev/peps/pep-0008/)\n",
+ "\n",
+ "> [PEP257: Docstring Conventions](http://www.python.org/dev/peps/pep-0257/)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "<a id=\"exceptions\"></a>\n",
+ "## Errors and exceptions\n",
+ "\n",
+ "Try running the script with `ACTG123` as the argument. What happens? Is this acceptable behavior?\n",
+ "\n",
+ "Sometimes we want to put safeguards to handle invalid inputs. In this case we only accept `ACTG`, all other characters are invalid.\n",
+ "\n",
+ "Python provides a way to break out of the normal execution flow, by raising what's called as an **exception**. We can raise exceptions ourselves as well, by using the `raise` statement."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### The `ValueError` built-in exception\n",
+ "\n",
+ "One of the most often used exceptions is the builtin exception `ValueError`. It is used on occasions where inappropriate argument values are used, for example when trying to convert the string `A` to an integer:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 14,
+ "metadata": {},
+ "outputs": [
+ {
+ "ename": "ValueError",
+ "evalue": "invalid literal for int() with base 10: 'A'",
+ "output_type": "error",
+ "traceback": [
+ "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m\n\u001b[1;31mValueError\u001b[0m Traceback (most recent call last)",
+ "\u001b[1;32m<ipython-input-14-0da6d315d7ad>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m()\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[0mint\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m'A'\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m",
+ "\u001b[1;31mValueError\u001b[0m: invalid literal for int() with base 10: 'A'"
+ ]
+ }
+ ],
+ "source": [
+ "int('A')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "`ValueError` is the appropriate exception to raise when your function is called with argument values it cannot handle."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Improving our script by handling invalid inputs\n",
+ "\n",
+ "Open your script, and edit the `if` clause to add our exception:\n",
+ "```python\n",
+ "def calc_gc_percent(seq):\n",
+ " \"\"\"\n",
+ " Calculates the GC percentage of the given sequence.\n",
+ "\n",
+ " Arguments:\n",
+ " - seq - the input sequence (string).\n",
+ "\n",
+ " Returns:\n",
+ " - GC percentage (float).\n",
+ "\n",
+ " The returned value is always <= 100.0\n",
+ " \"\"\"\n",
+ " at_count, gc_count = 0, 0\n",
+ " # Change input to all caps to allow for non-capital\n",
+ " # input sequence.\n",
+ " for char in seq.upper():\n",
+ " if char in ('A', 'T'):\n",
+ " at_count += 1\n",
+ " elif char in ('G', 'C'):\n",
+ " gc_count += 1\n",
+ " else:\n",
+ " raise ValueError(\n",
+ " \"Unexpeced character found: {}. Only \"\n",
+ " \"ACTGs are allowed.\".format(char))\n",
+ " \n",
+ " return gc_count * 100.0 / (gc_count + at_count)\n",
+ "```\n",
+ "Try running the script again with `ACTG123` as the argument. What happens now?"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Handling corner cases\n",
+ "\n",
+ "Try running the script with `''` (two quote signs) as the argument. What happens? Why? Is this a valid input?\n",
+ "\n",
+ "We don't always want to let exceptions stop program flow, sometimes we want to provide alternative flow. The `try ... except` block allows you to do this.\n",
+ "\n",
+ "The syntax is:\n",
+ "\n",
+ " try:\n",
+ " # Statements that may raise exceptions.\n",
+ " # [...]\n",
+ " except {exception type}:\n",
+ " # What to do when the exceptionis raised.\n",
+ " # [...]"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Improving our script by handling corner cases\n",
+ "\n",
+ "Let's change our script by adding a `try ... except` block:\n",
+ "\n",
+ "```python\n",
+ " def calc_gc_percent(seq):\n",
+ " \"\"\"\n",
+ " Calculates the GC percentage of the given sequence.\n",
+ "\n",
+ " Arguments:\n",
+ " - seq - the input sequence (string).\n",
+ "\n",
+ " Returns:\n",
+ " - GC percentage (float).\n",
+ "\n",
+ " The returned value is always <= 100.0\n",
+ " \"\"\"\n",
+ " at_count, gc_count = 0, 0\n",
+ " # Change input to all caps to allow for non-capital\n",
+ " # input sequence.\n",
+ " for char in seq.upper():\n",
+ " if char in ('A', 'T'):\n",
+ " at_count += 1\n",
+ " elif char in ('G', 'C'):\n",
+ " gc_count += 1\n",
+ " else:\n",
+ " raise ValueError(\n",
+ " \"Unexpeced character found: {}. Only \"\n",
+ " \"ACTGs are allowed.\".format(char))\n",
+ "\n",
+ " # Corner case handling: empty input sequence.\n",
+ " try:\n",
+ " return gc_count * 100.0 / (gc_count + at_count)\n",
+ " except ZeroDivisionError:\n",
+ " return 0.0\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Detour: Exception handling best practices\n",
+ "\n",
+ "#### Aim for a minimal `try` block\n",
+ "\n",
+ "We want to be able to pinpoint the statements that may raise the exceptions so we can tailor our handling.\n",
+ "\n",
+ "Example of code that violates this principle:\n",
+ "```python\n",
+ " try:\n",
+ " my_function()\n",
+ " my_other_function()\n",
+ " except ValueError:\n",
+ " my_fallback_function()\n",
+ "```\n",
+ "A better way would be:\n",
+ "```python\n",
+ " try:\n",
+ " my_function()\n",
+ " except ValueError:\n",
+ " my_fallback_function()\n",
+ " my_other_function()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "#### Be specific when handling exceptions\n",
+ "\n",
+ "The following code is syntactically valid, but *never* use it in your real scripts / programs:\n",
+ "```python\n",
+ " try:\n",
+ " my_function()\n",
+ " except:\n",
+ " my_fallback_function()\n",
+ "```\n",
+ "*Always* use the full exception name when handling exceptions, to make for a much cleaner code:\n",
+ "```python\n",
+ " try:\n",
+ " my_function()\n",
+ " except ValueError:\n",
+ " my_fallback_function()\n",
+ " except TypeError:\n",
+ " my_other_fallback_function()\n",
+ " except IndexError:\n",
+ " my_final_function()\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "#### Look Before You Leap (LBYL) vs Easier to Ask for Apology (EAFP)\n",
+ " \n",
+ "We could have written our last exception block like so:\n",
+ "```python\n",
+ " if gc_count + at_count == 0:\n",
+ " return 0.0\n",
+ " return gc_count * 100.0 / (gc_count + at_count)\n",
+ "```\n",
+ "Both approaches are correct and have their own plus and minuses in general. However in this case, I would argue that EAFP is better since it makes the code more readable."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Improving our script by handling more corner cases\n",
+ "\n",
+ "Now try running your script without any arguments at all. What happens?\n",
+ "\n",
+ "Armed with what you now know, how would you handle this situation?"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 15,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from IPython.core.display import HTML\n",
+ "def custom_style():\n",
+ " style = open('styles/notebook.css', 'r').read()\n",
+ " return HTML('<style>' + style + '</style>')\n",
+ "def custom_script():\n",
+ " script = open('styles/notebook.js', 'r').read()\n",
+ " return HTML('<script>' + script + '</script>')"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 16,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "<style>/*\n",
+ " https://github.com/CamDavidsonPilon/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers\n",
+ "*/\n",
+ "@font-face {\n",
+ " font-family: \"Computer Modern\";\n",
+ " src: url('http://mirrors.ctan.org/fonts/cm-unicode/fonts/otf/cmunss.otf');\n",
+ "}\n",
+ "div.cell{\n",
+ " width:800px;\n",
+ " margin-left:16% !important;\n",
+ " margin-right:auto;\n",
+ "}\n",
+ "h1 {\n",
+ " font-family: Helvetica, serif;\n",
+ "}\n",
+ "h4{\n",
+ " margin-top:12px;\n",
+ " margin-bottom: 3px;\n",
+ " }\n",
+ "div.text_cell_render{\n",
+ " font-family: Computer Modern, \"Helvetica Neue\", Arial, Helvetica, Geneva, sans-serif;\n",
+ " line-height: 145%;\n",
+ " font-size: 130%;\n",
+ " width:800px;\n",
+ " margin-left:auto;\n",
+ " margin-right:auto;\n",
+ "}\n",
+ ".CodeMirror{\n",
+ " font-family: \"Source Code Pro\", source-code-pro,Consolas, monospace;\n",
+ "}\n",
+ ".prompt{\n",
+ " display: None;\n",
+ "}\n",
+ ".text_cell_render .exercise {\n",
+ " font-weight: 300;\n",
+ " /*font-size: 22pt;*/\n",
+ " color: #4057A1;\n",
+ " font-style: italic;\n",
+ " /*margin-bottom: .5em;\n",
+ " margin-top: 0.5em;\n",
+ " display: block;*/\n",
+ "}\n",
+ ".text_cell_render .example {\n",
+ " font-weight: 300;\n",
+ " color: #40A157;\n",
+ " font-style: italic;\n",
+ "}\n",
+ "\n",
+ ".warning{\n",
+ " color: rgb( 240, 20, 20 )\n",
+ "}\n",
+ "</style>"
+ ],
+ "text/plain": [
+ "<IPython.core.display.HTML at 0x2ceccd0>"
+ ]
+ },
+ "execution_count": 16,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "custom_style()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 17,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "<script>// https://github.com/CamDavidsonPilon/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers\n",
+ "MathJax.Hub.Config({\n",
+ " TeX: {\n",
+ " extensions: [\"AMSmath.js\"]\n",
+ " },\n",
+ " tex2jax: {\n",
+ " inlineMath: [ ['$','$'], [\"\\\\(\",\"\\\\)\"] ],\n",
+ " displayMath: [ ['$$','$$'], [\"\\\\[\",\"\\\\]\"] ]\n",
+ " },\n",
+ " displayAlign: 'center', // Change this to 'center' to center equations.\n",
+ " \"HTML-CSS\": {\n",
+ " styles: {'.MathJax_Display': {\"margin\": 4}}\n",
+ " }\n",
+ " });\n",
+ "</script>"
+ ],
+ "text/plain": [
+ "<IPython.core.display.HTML at 0x2cecdd0>"
+ ]
+ },
+ "execution_count": 17,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "custom_script()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "slideshow": {
+ "slide_type": "skip"
+ }
+ },
+ "source": [
+ "Acknowledgements\n",
+ "========\n",
+ "\n",
+ "[Wibowo Arindrarto](mailto:w.arindrarto@lumc.nl)\n",
+ "\n",
+ "Martijn Vermaat\n",
+ "\n",
+ "[Jeroen Laros](mailto:j.f.j.laros@lumc.nl)\n",
+ "\n",
+ "Based on\n",
+ "---------\n",
+ "[Python Scientific Lecture Notes](http://scipy-lectures.github.io/)\n",
+ "\n",
+ "License\n",
+ "--------\n",
+ "[Creative Commons Attribution 3.0 License (CC-by)](http://creativecommons.org/licenses/by/3.0)"
+ ]
+ }
+ ],
+ "metadata": {
+ "celltoolbar": "Slideshow",
+ "kernelspec": {
+ "display_name": "Python 2",
+ "language": "python",
+ "name": "python2"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 2
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython2",
+ "version": "2.7.12"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 1
+}
diff --git a/more_python/03_more_python_goodness_2.ipynb b/more_python/03_more_python_goodness_2.ipynb
new file mode 100644
index 0000000000000000000000000000000000000000..0cf23e705b4d1c5d3e1b0372f22d6ddc19b88564
--- /dev/null
+++ b/more_python/03_more_python_goodness_2.ipynb
@@ -0,0 +1,1846 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# More Python Goodness (2)\n",
+ "***"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Table of contents\n",
+ "\n",
+ "1. Working with scripts\n",
+ "2. The standard library\n",
+ "3. String methods\n",
+ "4. Comments and docstrings\n",
+ "5. Detour: PEP8 and other PEPs\n",
+ "6. Errors and exceptions\n",
+ "7. [Working with modules](#modules)\n",
+ "8. [Examples from the standard library](#stdlib-examples)\n",
+ "9. [Reading and writing files](#io)\n",
+ "10. [Assignment: Finding the most common 7-mer in a FASTA file](#assignment)\n",
+ "11. [Further reading](#further)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "<a id=\"modules\"></a>\n",
+ "## Working with modules\n",
+ "\n",
+ "Sometimes it is useful to group functions and other objects in different files. Sometimes you need to use that fancy function you've written 2 years ago. This is where modules in Python come in handy.\n",
+ "\n",
+ "More officially, a **module** allows you to share code in the form of libraries. You've seen one example: the `sys` module in the standard library. There are many other modules in the standard library, as we'll see soon."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### What modules look like\n",
+ "\n",
+ "Any Python script can in principle be imported as a module. We can import whenever we can write a valid Python statement, in a script or in an interpreter session.\n",
+ "\n",
+ "If a script is called `script.py`, then we use `import script`. This gives us access to the objects defined in `script.py` by prefixing them with `script` and a dot.\n",
+ "\n",
+ "Keep in mind that this is not the only way to import Python modules. Refer to the Python documentation to find out more ways to do imports."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Using `seq_toolbox.py` as a module\n",
+ "\n",
+ "Open an interpreter and try importing your module:\n",
+ "\n",
+ "```python\n",
+ "import seq_toolbox\n",
+ "```\n",
+ "Does this work? Why?"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Improving our script for importing\n",
+ "\n",
+ "During a module import, Python executes all the statements inside the module.\n",
+ "\n",
+ "To make our script work as a module (in the intended way), we need to add a check whether the module is imported or not:\n",
+ "```python\n",
+ " #!/usr/bin/env python\n",
+ " import sys\n",
+ "\n",
+ " def calc_gc_percent(seq):\n",
+ " \"\"\"\n",
+ " Calculates the GC percentage of the given sequence.\n",
+ "\n",
+ " Arguments:\n",
+ " - seq - the input sequence (string).\n",
+ "\n",
+ " Returns:\n",
+ " - GC percentage (float).\n",
+ "\n",
+ " The returned value is always <= 100.0\n",
+ " \"\"\"\n",
+ " at_count, gc_count = 0, 0\n",
+ " # Change input to all caps to allow for non-capital\n",
+ " # input sequence.\n",
+ " for char in seq.upper():\n",
+ " if char in ('A', 'T'):\n",
+ " at_count += 1\n",
+ " elif char in ('G', 'C'):\n",
+ " gc_count += 1\n",
+ " else:\n",
+ " raise ValueError(\n",
+ " \"Unexpeced character found: {}. Only \"\n",
+ " \"ACTGs are allowed.\".format(char))\n",
+ "\n",
+ " # Corner case handling: empty input sequence.\n",
+ " try:\n",
+ " return gc_count * 100.0 / (gc_count + at_count)\n",
+ " except ZeroDivisionError:\n",
+ " return 0.0\n",
+ "\n",
+ " if __name__ == '__main__':\n",
+ " input_seq = sys.argv[1]\n",
+ " print \"The sequence '{}' has %GC of {:.2f}\".format(\n",
+ " input_seq, calc_gc_percent(input_seq))\n",
+ "```\n",
+ "Now try importing the module again. What happens? Can you still use the module as a script?"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Using modules\n",
+ "\n",
+ "When a module is imported, we can access the objects defined in it:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 1,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import seq_toolbox"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "<function seq_toolbox.calc_gc_percent>"
+ ]
+ },
+ "execution_count": 2,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "seq_toolbox.calc_gc_percent"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "By the way, remember we added docstring to the `calc_gc_percent` function? After importing our module, we can read up on how to use the function in its docstring:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 3,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "seq_toolbox.calc_gc_percent?"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 4,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "50.0"
+ ]
+ },
+ "execution_count": 4,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "seq_toolbox.calc_gc_percent('ACTG')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "We can also expose an object inside the module directly into our current namespace using the `from ... import ...` statement:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 5,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from seq_toolbox import calc_gc_percent"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 6,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "25.0"
+ ]
+ },
+ "execution_count": 6,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "calc_gc_percent('AAAG')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Sometimes, we want to alias the imported object to reduce the chance of it overwriting any already-defined objects with the same name. This is accomplished using the `from ... import ... as ...` statement:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 7,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from seq_toolbox import calc_gc_percent as gc_calc"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 8,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "25.0"
+ ]
+ },
+ "execution_count": 8,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "gc_calc('AAAG')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### (A simple guide on) How modules are discovered\n",
+ "\n",
+ "In our case, Python imports by checking whether the module exists in the current directory. This is not the only place Python looks, however.\n",
+ "\n",
+ "A complete list of paths where Python looks for modules is available via the `sys` module as `sys.path`. It is composed of (in order):\n",
+ "\n",
+ "1. The current directory.\n",
+ "2. The `PYTHONPATH` environment variable.\n",
+ "3. Installation-dependent defaults."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "<a id=\"stdlib-examples\"></a>\n",
+ "## Examples from the standard library\n",
+ "\n",
+ "> Official Python documentation: [The Python Standard Library](http://docs.python.org/2/library/index.html)\n",
+ "\n",
+ "Just to improve our knowledge, let's go through some of the most often used standard library modules."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### The standard library: `os` module\n",
+ "\n",
+ "> The Python Standard Library: [15.1. os — Miscellaneous operating system interfaces](http://docs.python.org/2/library/os.html)\n",
+ "\n",
+ "The `os` module provides a portable way of using various operating system-specific functionality. It is a large module, but the one of the most frequently used bits is the file-related functions."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 9,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import os"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 10,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "'/home/martijn/projects/programming-course'"
+ ]
+ },
+ "execution_count": 10,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "os.getcwd() # Get current directory."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 11,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "'/home/martijn/.virtualenvs/programming-course/bin:/home/martijn/projects/vcftools_0.1.11/bin:/home/martijn/projects/vcftools_0.1.11/cpp:/home/martijn/projects/muscle/muscle3.8.31/src:/home/martijn/projects/bedtools/bin:/home/martijn/projects/bamtools/bamtools/bin:/home/martijn/projects/gvnl/concordance/tabix:/home/martijn/projects/samtools-trunk:/home/martijn/projects/samtools-trunk/bcftools:/home/martijn/.venvburrito/bin:/home/martijn/coq-8.3-rc1/bin:/home/martijn/projects/kiek/trunk:/home/martijn/bin:/home/martijn/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games'"
+ ]
+ },
+ "execution_count": 11,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "os.environ['PATH'] # Get the value of the environment variable PATH."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 12,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "my_filename = 'input.fastq'"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 13,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "('input', '.fastq')"
+ ]
+ },
+ "execution_count": 13,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "os.path.splitext(my_filename) # Split the extension and filename."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 14,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "'/home/martijn/projects/programming-course/input.fastq'"
+ ]
+ },
+ "execution_count": 14,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "# Join the current directory and `my_filename` to create a file path.\n",
+ "os.path.join(os.getcwd(), my_filename)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 15,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "False"
+ ]
+ },
+ "execution_count": 15,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "os.path.exists(my_filename) # Check whether `my_filename` exists or not."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 16,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "True"
+ ]
+ },
+ "execution_count": 16,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "os.path.isdir('/home') # Checks whether '/home' is a directory."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 17,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "False"
+ ]
+ },
+ "execution_count": 17,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "os.path.isfile('/home') # Checks whether '/home' is a file."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### The standard library: `sys` module\n",
+ "\n",
+ "> The Python Standard Library: [27.1. sys — System-specific parameters and functions](http://docs.python.org/2/library/sys.html)\n",
+ "\n",
+ "This module has various runtime-related and interpreter-related functions. We've seen two of the most commonly used: `sys.argv` and `sys.path`."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 18,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import sys"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 19,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "['',\n",
+ " '/home/martijn/.venvburrito/lib/python/distribute-0.6.49-py2.7.egg',\n",
+ " '/home/martijn/.venvburrito/lib/python/pip-1.4.1-py2.7.egg',\n",
+ " '/home/martijn/.venvburrito/lib/python2.7/site-packages',\n",
+ " '/home/martijn/.venvburrito/lib/python',\n",
+ " '/usr/local/samba/lib/python2.6/site-packages',\n",
+ " '/home/martijn/.virtualenvs/programming-course/lib/python2.7',\n",
+ " '/home/martijn/.virtualenvs/programming-course/lib/python2.7/plat-linux2',\n",
+ " '/home/martijn/.virtualenvs/programming-course/lib/python2.7/lib-tk',\n",
+ " '/home/martijn/.virtualenvs/programming-course/lib/python2.7/lib-old',\n",
+ " '/home/martijn/.virtualenvs/programming-course/lib/python2.7/lib-dynload',\n",
+ " '/usr/lib/python2.7',\n",
+ " '/usr/lib/python2.7/plat-linux2',\n",
+ " '/usr/lib/python2.7/lib-tk',\n",
+ " '/home/martijn/.virtualenvs/programming-course/local/lib/python2.7/site-packages',\n",
+ " '/home/martijn/.virtualenvs/programming-course/local/lib/python2.7/site-packages/gtk-2.0',\n",
+ " '/home/martijn/.virtualenvs/programming-course/lib/python2.7/site-packages',\n",
+ " '/home/martijn/.virtualenvs/programming-course/lib/python2.7/site-packages/gtk-2.0',\n",
+ " '/usr/local/samba/lib/python2.6/site-packages',\n",
+ " '/usr/local/samba/lib/python2.6/site-packages',\n",
+ " '/home/martijn/.virtualenvs/programming-course/local/lib/python2.7/site-packages/IPython/extensions']"
+ ]
+ },
+ "execution_count": 19,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "sys.path # List of places where Python looks for modules when importing."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 20,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "'/home/martijn/.virtualenvs/programming-course/bin/python'"
+ ]
+ },
+ "execution_count": 20,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "sys.executable # Path to the current interpreter's executable."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 21,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "sys.version_info(major=2, minor=7, micro=3, releaselevel='final', serial=0)"
+ ]
+ },
+ "execution_count": 21,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "sys.version_info # Information about our Python version."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 22,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "2"
+ ]
+ },
+ "execution_count": 22,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "sys.version_info.major # It also provide a more granular access."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### The standard library: `math` module\n",
+ "\n",
+ "> The Python Standard Library: [9.2. math — Mathematical functions](http://docs.python.org/2/library/math.html)\n",
+ "\n",
+ "Useful math-related functions can be found here. Other more comprehensive modules exist (`numpy`, your lesson tomorrow), but nevertheless `math` is still useful."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 23,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import math"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 24,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "2.302585092994046"
+ ]
+ },
+ "execution_count": 24,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "math.log(10) # Natural log of 10."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 25,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "2.0"
+ ]
+ },
+ "execution_count": 25,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "math.log(100, 10) # Log base 10 of 100."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 26,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "81.0"
+ ]
+ },
+ "execution_count": 26,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "math.pow(3, 4) # 3 raised to the 4th power."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 27,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "1.4142135623730951"
+ ]
+ },
+ "execution_count": 27,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "math.sqrt(2) # Square root of 2."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 28,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "3.141592653589793"
+ ]
+ },
+ "execution_count": 28,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "math.pi # The value of pi."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### The standard library: `random` module\n",
+ "\n",
+ "> The Python Standard Library: [9.6. random — Generate pseudo-random numbers](http://docs.python.org/2/library/random.html)\n",
+ "\n",
+ "The `random` module contains useful functions for generating pseudo-random numbers."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 29,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import random"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 30,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "0.05941901356497081"
+ ]
+ },
+ "execution_count": 30,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "random.random() # Random float x, such that 0.0 <= x < 1.0."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 31,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "13"
+ ]
+ },
+ "execution_count": 31,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "random.randint(2, 17) # Random integer between 2 and 17, inclusive."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 32,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "'grape'"
+ ]
+ },
+ "execution_count": 32,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "# Random choice of any items in the given list.\n",
+ "random.choice(['apple', 'banana', 'grape', 'kiwi', 'orange'])"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 33,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "['orange', 'apple', 'banana']"
+ ]
+ },
+ "execution_count": 33,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "# Random sampling of 3 items from the given list.\n",
+ "random.sample(['apple', 'banana', 'grape', 'kiwi', 'orange'], 3)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### The standard library: `re` module\n",
+ "\n",
+ "> The Python Standard Library: [7.2. re — Regular expression operations](http://docs.python.org/2/library/re.html)\n",
+ "\n",
+ "Regular expression-related functions are in the `re` module."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 34,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import re"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 35,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "my_seq = 'CAGTCAGT'"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 36,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "results1 = re.search(r'CA.+CA', my_seq)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 37,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "'CAGTCA'"
+ ]
+ },
+ "execution_count": 37,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "results1.group(0)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 38,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "results2 = re.search(r'CCC..', my_seq)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 39,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "None\n"
+ ]
+ }
+ ],
+ "source": [
+ "print results2"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### The standard library: `argparse` module\n",
+ "\n",
+ "> The Python Standard Library: [15.4. argparse — Parser for command-line options, arguments and sub-commands](http://docs.python.org/2/library/argparse.html)\n",
+ "\n",
+ "Using `sys.argv` is neat for small scripts, but as our script gets larger and more complex, we want to be able to handle complex arguments too. The `argparse` module has handy functionalities for creating command-line scripts."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "#### Improving our script with `argparse`\n",
+ "\n",
+ "Open your script/module in a text editor and replace `import sys` with `import argparse`. Remove all lines / blocks referencing `sys.argv`\n",
+ "\n",
+ "Change the `if __name__ == '__main__'` block to be the following:\n",
+ "\n",
+ "```python\n",
+ "if __name__ == '__main__':\n",
+ " # Create our argument parser object.\n",
+ " parser = argparse.ArgumentParser()\n",
+ " # Add the expected argument.\n",
+ " parser.add_argument('input_seq', type=str,\n",
+ " help=\"Input sequence\")\n",
+ " # Do the actual parsing.\n",
+ " args = parser.parse_args()\n",
+ " # And show the output.\n",
+ " print \"The sequence '{}' has %GC of {:.2f}\".format(\n",
+ " args.input_seq,\n",
+ " calc_gc_percent(args.input_seq))\n",
+ "```\n",
+ "The code does look a little more verbose, but we get something better in return.\n",
+ "\n",
+ "Go back to the shell and execute your script without any arguments. What happens?\n",
+ "\n",
+ "Try executing the following command in the shell. What happens?\n",
+ "\n",
+ " $ python seq_toolbox.py --help\n",
+ " \n",
+ "We're just getting started on `argparse`. There are other useful bits that we'll see shortly after a small intro on file I/O."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "<a id=\"io\"></a>\n",
+ "## Reading and writing files\n",
+ "\n",
+ "Opening files for reading or writing is done using the `open` function. It is commonly used with two arguments, *name* and *mode*:\n",
+ "\n",
+ "* *name* is the name of the file to open.\n",
+ "* *mode* specifies how the file should be handled.\n",
+ "\n",
+ "These are some of the common file modes:\n",
+ "\n",
+ "* `r`: open file for reading (default).\n",
+ "* `w`: open file for writing.\n",
+ "* `a`: open file for appending content."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 40,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "open?"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Reading files\n",
+ "\n",
+ "Let's go through some ways of reading from a file."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 41,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "fh = open('data/short_file.txt')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "`fh` is a file handle object which we can use to retrieve the file contents. One simple way would be to read the whole file contents:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 42,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "'this short file has two lines\\nit is used in the example code\\n'"
+ ]
+ },
+ "execution_count": 42,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "fh.read()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Executing `fh.read()` a second time gives an empty string. This is because we have \"walked\" through the file to its end."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 43,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "''"
+ ]
+ },
+ "execution_count": 43,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "fh.read()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "We can reset the handle to the beginning of the file again using the `seek()` function. Here, we use 0 as the argument since we want to move the handle to position 0 (beginning of the file):"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 44,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "fh.seek(0)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 45,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "'this short file has two lines\\nit is used in the example code\\n'"
+ ]
+ },
+ "execution_count": 45,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "fh.read()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "In practice, reading the whole file into memory is not always a good idea. It is practical for small files, but not if our file is big (e.g., bigger than our memory). In this case, the alternative is to use the `readline()` function."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 46,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "fh.seek(0)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 47,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "'this short file has two lines\\n'"
+ ]
+ },
+ "execution_count": 47,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "fh.readline()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 48,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "'it is used in the example code\\n'"
+ ]
+ },
+ "execution_count": 48,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "fh.readline()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 49,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "''"
+ ]
+ },
+ "execution_count": 49,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "fh.readline()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "More common in Python is to use the `for` loop with the file handle itself. Python will automatically iterate over each line."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 50,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "fh.seek(0)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 51,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "this short file has two lines\n",
+ "\n",
+ "it is used in the example code\n",
+ "\n"
+ ]
+ }
+ ],
+ "source": [
+ "for line in fh:\n",
+ " print line"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "We can see that iteration exhausts the handle since we are at the end of the file after the loop."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 52,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "''"
+ ]
+ },
+ "execution_count": 52,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "fh.readline()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "We can also check the file handle position using the `tell()` function. If `tell()` returns a nonzero number, then we are not at the beginning of the file."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 53,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/plain": [
+ "61"
+ ]
+ },
+ "execution_count": 53,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "fh.tell()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Now that we're done with the file handle, we can call the `close()` method to free up any system resources still being used to keep the file open. After we closed the file, we can not use the file object anymore."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 54,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "fh.close()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 55,
+ "metadata": {},
+ "outputs": [
+ {
+ "ename": "ValueError",
+ "evalue": "I/O operation on closed file",
+ "output_type": "error",
+ "traceback": [
+ "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m\n\u001b[1;31mValueError\u001b[0m Traceback (most recent call last)",
+ "\u001b[1;32m<ipython-input-55-4e86183cf03e>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m()\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[0mfh\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mreadline\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m",
+ "\u001b[1;31mValueError\u001b[0m: I/O operation on closed file"
+ ]
+ }
+ ],
+ "source": [
+ "fh.readline()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Writing files\n",
+ "\n",
+ "When writing files, we supply the `w` file mode explicitely:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 58,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "fw = open('data/my_file.txt', 'w')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "`fw` is a file handle similar to the `fh` that we've seen previously. It is used only for writing and not reading, however."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 59,
+ "metadata": {},
+ "outputs": [
+ {
+ "ename": "IOError",
+ "evalue": "File not open for reading",
+ "output_type": "error",
+ "traceback": [
+ "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m\n\u001b[1;31mIOError\u001b[0m Traceback (most recent call last)",
+ "\u001b[1;32m<ipython-input-59-73497a15302b>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m()\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[0mfw\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mread\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m",
+ "\u001b[1;31mIOError\u001b[0m: File not open for reading"
+ ]
+ }
+ ],
+ "source": [
+ "fw.read()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "To write to the file, we use its `write()` method. Remember that Python *does not* add newline characters here (as opposed to when you use the `print` statement), so to move to a new line we have to add `\\n` ourselves."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 60,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "fw.write('This is my first line ')"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 61,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "fw.write('Still on my first line\\n')"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 62,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "fw.write('Now on my second line')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "As with the `r` mode, we can close the handle when we're done with it. The file can then be reopened with the `r` mode and we can check its contents."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 63,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "fw.close()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 64,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "This is my first line Still on my first line\n",
+ "\n",
+ "Now on my second line\n"
+ ]
+ }
+ ],
+ "source": [
+ "fr = open('data/my_file.txt') # Remember to use the same file we wrote to.\n",
+ "for line in fr:\n",
+ " print line\n",
+ "fr.close()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "And finally, to remove the file, we can use the `remove()` function from the `os` module."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 65,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "os.remove('data/my_file.txt')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Be cautious when using file handles\n",
+ "\n",
+ "When reading / writing files, we are interacting with external resources that may or may not behave as expected. For example, we don't always have permission to read / write a file, the file itself may not exist, or we have a completely wrong idea of what's in the file. In situations like these, you are encouraged to use the `try ... finally` block.\n",
+ " \n",
+ "The syntax is similar to `try ... except` that we've seen earlier (in fact they are part of the same block, as we'll see later). Unlike `try ... except`, the `finally` block in `try ... finally` is always executed regardless of any raised exceptions.\n",
+ "\n",
+ "Let's take a look at some examples. First, the not recommended one:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 66,
+ "metadata": {},
+ "outputs": [
+ {
+ "ename": "ValueError",
+ "evalue": "invalid literal for int() with base 10: 'this short file has two lines\\n'",
+ "output_type": "error",
+ "traceback": [
+ "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m\n\u001b[1;31mValueError\u001b[0m Traceback (most recent call last)",
+ "\u001b[1;32m<ipython-input-66-f293b9e3578f>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m()\u001b[0m\n\u001b[0;32m 1\u001b[0m \u001b[0mf\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mopen\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m'data/short_file.txt'\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 2\u001b[0m \u001b[1;32mfor\u001b[0m \u001b[0mline\u001b[0m \u001b[1;32min\u001b[0m \u001b[0mf\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m----> 3\u001b[1;33m \u001b[1;32mprint\u001b[0m \u001b[0mint\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mline\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 4\u001b[0m \u001b[0mf\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mclose\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 5\u001b[0m \u001b[1;32mprint\u001b[0m \u001b[1;34m'We closed our filehandle'\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
+ "\u001b[1;31mValueError\u001b[0m: invalid literal for int() with base 10: 'this short file has two lines\\n'"
+ ]
+ }
+ ],
+ "source": [
+ "f = open('data/short_file.txt')\n",
+ "for line in f:\n",
+ " print int(line)\n",
+ "f.close()\n",
+ "print 'We closed our filehandle'"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Apart from our erroneous conversion of a line of text to an integer, the exception raised because of that causes the `f.close()` statement to be not executed. At this point we have a stale open file handle.\n",
+ "\n",
+ "Stubbornly trying to do the same thing again, this time we use a `finally` clause:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 67,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "We closed our file handle\n"
+ ]
+ },
+ {
+ "ename": "ValueError",
+ "evalue": "invalid literal for int() with base 10: 'this short file has two lines\\n'",
+ "output_type": "error",
+ "traceback": [
+ "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m\n\u001b[1;31mValueError\u001b[0m Traceback (most recent call last)",
+ "\u001b[1;32m<ipython-input-67-71128226da53>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m()\u001b[0m\n\u001b[0;32m 2\u001b[0m \u001b[0mf\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mopen\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m'data/short_file.txt'\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 3\u001b[0m \u001b[1;32mfor\u001b[0m \u001b[0mline\u001b[0m \u001b[1;32min\u001b[0m \u001b[0mf\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m----> 4\u001b[1;33m \u001b[1;32mprint\u001b[0m \u001b[0mint\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mline\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 5\u001b[0m \u001b[1;32mfinally\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 6\u001b[0m \u001b[0mf\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mclose\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
+ "\u001b[1;31mValueError\u001b[0m: invalid literal for int() with base 10: 'this short file has two lines\\n'"
+ ]
+ }
+ ],
+ "source": [
+ "try:\n",
+ " f = open('data/short_file.txt')\n",
+ " for line in f:\n",
+ " print int(line)\n",
+ "finally:\n",
+ " f.close()\n",
+ " print 'We closed our file handle'"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "As you can see, this way the file handle still got closed.\n",
+ "\n",
+ "Now, an even better way would be to also use the `catch` block, to handle the exception we might get if we try it a third time."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 68,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Seems there was a line we could not handle\n",
+ "We closed our file handle\n"
+ ]
+ }
+ ],
+ "source": [
+ "try:\n",
+ " f = open('data/short_file.txt')\n",
+ " for line in f:\n",
+ " print int(line)\n",
+ "except ValueError:\n",
+ " print 'Seems there was a line we could not handle'\n",
+ "finally:\n",
+ " f.close()\n",
+ " print 'We closed our file handle'"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Intermezzo: `sys.stdout`, `sys.stderr`, and `sys.stdin`\n",
+ " \n",
+ "We've seen that the `sys` module provides some useful runtime functions. Now that we know about file handles, we can use three `sys` objects that are essentially file handles: `sys.stdout`, `sys.stderr`, and `sys.stdin`.\n",
+ "\n",
+ "Together, they provide access to the standard output, standard error, and standard input streams. We can use them appropriately by writing to `sys.stdout` and `sys.stderr`, and reading from `sys.stdin`.\n",
+ "\n",
+ "Unlike regular file handles, you don't need to close them after using (in fact you should not). The assumption is that these handles are always open to write to or to read from."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 69,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "I'm writing to stdout!\n"
+ ]
+ }
+ ],
+ "source": [
+ "sys.stdout.write(\"I'm writing to stdout!\\n\")"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 70,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stderr",
+ "output_type": "stream",
+ "text": [
+ "Now to stderr.\n"
+ ]
+ }
+ ],
+ "source": [
+ "sys.stderr.write(\"Now to stderr.\\n\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Improving our script to allow input from a file\n",
+ "\n",
+ "Before we go on to the exercise, let's do a final improvement on our script/module.\n",
+ "\n",
+ "We want to add some extra functionality: the script should accept as its argument a path to a file containing sequences. It will then compute the GC percentage for each sequence in this file.\n",
+ "\n",
+ "There are at least two things we need to do:\n",
+ "\n",
+ "1. Change the argument parser so that it deals with a new execution mode.\n",
+ "2. Add some statements to read from a file.\n",
+ "\n",
+ "Open the script in your text editor, and change the `if __name__ == '__main__'` block to the following:\n",
+ "```python\n",
+ "if __name__ == '__main__':\n",
+ " # Create our argument parser object.\n",
+ " parser = argparse.ArgumentParser()\n",
+ " # Add argument for the input type.\n",
+ " parser.add_argument(\n",
+ " 'mode', type=str, choices=['file', 'text'],\n",
+ " help='Input type of the script')\n",
+ " # Add argument for the input value.\n",
+ " parser.add_argument(\n",
+ " 'value', type=str,\n",
+ " help='Input value of the script')\n",
+ " # Do the actual parsing.\n",
+ " args = parser.parse_args()\n",
+ "\n",
+ " message = \"The sequence '{}' has a %GC of {:.2f}\"\n",
+ "\n",
+ " if args.mode == 'file':\n",
+ " try:\n",
+ " f = open(args.value, 'r')\n",
+ " for line in f:\n",
+ " seq = line.strip()\n",
+ " gc = calc_gc_percent(seq)\n",
+ " print message.format(seq, gc)\n",
+ " finally:\n",
+ " f.close()\n",
+ " else:\n",
+ " seq = args.value\n",
+ " gc = calc_gc_percent(seq)\n",
+ " print message.format(seq, gc)\n",
+ "``` \n",
+ "Note the things we've done here:\n",
+ "\n",
+ "1. We've added a new argument to our parser to specify the input type.\n",
+ "2. Correspondingly, we've expanded the our function call to handle both input types.\n",
+ "\n",
+ "Save the script, and try running it. What do you see? Is running\n",
+ "\n",
+ " $ python seq_toolbox.py --help\n",
+ "\n",
+ "helpful to resolve this?\n",
+ "\n",
+ "Try running the script with the following command. What do you see?\n",
+ "\n",
+ " $ python seq_toolbox.py file data/seq.txt\n",
+ "\n",
+ "Feel free to look into `data/seq.txt`."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "<a id=\"assignment\"></a>\n",
+ "## Assignment: Finding the most common 7-mer in a FASTA file\n",
+ "\n",
+ "### Your task\n",
+ "\n",
+ "Write a script to print out the most common 7-mer and its GC percentage from all the sequences in `data/records.fa`. You are free to reuse your existing toolbox.\n",
+ "\n",
+ "> The example FASTA file was adapted from: [Genome Biology DNA60 Bioinformatics Challenge](http://genomebiology.com/about/update/DNA60_STEPONE)\n",
+ "\n",
+ "### Hints\n",
+ "\n",
+ "1. FASTA files have two types of lines: header lines starting with a `>` character and sequence lines. We are only concerned with the sequence line.\n",
+ "2. Read the string functions documentation.\n",
+ "3. Read the documentation for built in functions.\n",
+ "\n",
+ "### Challenges\n",
+ "\n",
+ "1. Find out how to change your script so that it can read from `data/challenge.fa.gz` without unzipping the file first (hint: standard library).\n",
+ "\n",
+ "2. Can you change the parser so that there is an option flag to tell the program whether the input file is gzipped or not?\n",
+ "\n",
+ "3. Can you change your script so that it works for any N-mers instead of for just 7-mers?"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "<a id=\"further\"></a>\n",
+ "## Further reading\n",
+ "\n",
+ "> Python standard library by examples: [Python Module of the Week](http://pymotw.com/2/contents.html)\n",
+ "\n",
+ "> [PEP8: Style Guide for Python Code](http://www.python.org/dev/peps/pep-0008/)\n",
+ "\n",
+ "> [PEP20: The Zen of Python](http://www.python.org/dev/peps/pep-0020/)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 71,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from IPython.core.display import HTML\n",
+ "def custom_style():\n",
+ " style = open('styles/notebook.css', 'r').read()\n",
+ " return HTML('<style>' + style + '</style>')\n",
+ "def custom_script():\n",
+ " script = open('styles/notebook.js', 'r').read()\n",
+ " return HTML('<script>' + script + '</script>')"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 72,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "<style>/*\n",
+ " https://github.com/CamDavidsonPilon/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers\n",
+ "*/\n",
+ "@font-face {\n",
+ " font-family: \"Computer Modern\";\n",
+ " src: url('http://mirrors.ctan.org/fonts/cm-unicode/fonts/otf/cmunss.otf');\n",
+ "}\n",
+ "div.cell{\n",
+ " width:800px;\n",
+ " margin-left:16% !important;\n",
+ " margin-right:auto;\n",
+ "}\n",
+ "h1 {\n",
+ " font-family: Helvetica, serif;\n",
+ "}\n",
+ "h4{\n",
+ " margin-top:12px;\n",
+ " margin-bottom: 3px;\n",
+ " }\n",
+ "div.text_cell_render{\n",
+ " font-family: Computer Modern, \"Helvetica Neue\", Arial, Helvetica, Geneva, sans-serif;\n",
+ " line-height: 145%;\n",
+ " font-size: 130%;\n",
+ " width:800px;\n",
+ " margin-left:auto;\n",
+ " margin-right:auto;\n",
+ "}\n",
+ ".CodeMirror{\n",
+ " font-family: \"Source Code Pro\", source-code-pro,Consolas, monospace;\n",
+ "}\n",
+ ".prompt{\n",
+ " display: None;\n",
+ "}\n",
+ ".text_cell_render .exercise {\n",
+ " font-weight: 300;\n",
+ " /*font-size: 22pt;*/\n",
+ " color: #4057A1;\n",
+ " font-style: italic;\n",
+ " /*margin-bottom: .5em;\n",
+ " margin-top: 0.5em;\n",
+ " display: block;*/\n",
+ "}\n",
+ ".text_cell_render .example {\n",
+ " font-weight: 300;\n",
+ " color: #40A157;\n",
+ " font-style: italic;\n",
+ "}\n",
+ "\n",
+ ".warning{\n",
+ " color: rgb( 240, 20, 20 )\n",
+ "}\n",
+ "</style>"
+ ],
+ "text/plain": [
+ "<IPython.core.display.HTML at 0x1caac50>"
+ ]
+ },
+ "execution_count": 72,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "custom_style()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 73,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "<script>// https://github.com/CamDavidsonPilon/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers\n",
+ "MathJax.Hub.Config({\n",
+ " TeX: {\n",
+ " extensions: [\"AMSmath.js\"]\n",
+ " },\n",
+ " tex2jax: {\n",
+ " inlineMath: [ ['$','$'], [\"\\\\(\",\"\\\\)\"] ],\n",
+ " displayMath: [ ['$$','$$'], [\"\\\\[\",\"\\\\]\"] ]\n",
+ " },\n",
+ " displayAlign: 'center', // Change this to 'center' to center equations.\n",
+ " \"HTML-CSS\": {\n",
+ " styles: {'.MathJax_Display': {\"margin\": 4}}\n",
+ " }\n",
+ " });\n",
+ "</script>"
+ ],
+ "text/plain": [
+ "<IPython.core.display.HTML at 0x1d3d950>"
+ ]
+ },
+ "execution_count": 73,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "custom_script()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Acknowledgements\n",
+ "========\n",
+ "\n",
+ "[Wibowo Arindrarto](mailto:w.arindrarto@lumc.nl)\n",
+ "\n",
+ "Martijn Vermaat\n",
+ "\n",
+ "[Jeroen Laros](mailto:j.f.j.laros@lumc.nl)\n",
+ "\n",
+ "Based on\n",
+ "---------\n",
+ "[Python Scientific Lecture Notes](http://scipy-lectures.github.io/)\n",
+ "\n",
+ "License\n",
+ "--------\n",
+ "[Creative Commons Attribution 3.0 License (CC-by)](http://creativecommons.org/licenses/by/3.0)"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 2",
+ "language": "python",
+ "name": "python2"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 2
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython2",
+ "version": "2.7.12"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 1
+}