Monochrome (Black & white) plots in matplotlib

Posted on Wed 10 August 2016 in Notebooks

Monochrome (Black & white) plots in matplotlib

While writing my thesis, I was annoyed that there wasn't any good default options for outputting monochrome plots, as I didn't count on being able to expect that all prints would be in color. I therefore wanted plots that could work without any greyscales.

Right now this notebook describes how to setup and use line plots and bar plots. If you need other types of plots, do not hessitate to contact me, and I'll see what I can do.

In [1]:
from sklearn import datasets
from matplotlib import pyplot as plt
%pylab
%matplotlib inline
import numpy as np

Using matplotlib backend: MacOSX
Populating the interactive namespace from numpy and matplotlib

Line and marker styles

To get an idea of which line styles and markers are available we can inspect the lines and markers object

In [2]:
from matplotlib import lines, markers
In [3]:
lines.lineStyles.keys()
Out[3]:
dict_keys(['', ' ', '--', ':', 'None', '-', '-.'])
In [4]:
markers.MarkerStyle.markers.keys()
Out[4]:
dict_keys([0, 1, '*', 3, 4, 5, 6, 7, '8', 'None', 'd', 'h', 'D', 'v', None, '^', ',', '>', 'x', '<', 's', 'p', '', '2', '4', ' ', '_', 'o', '+', 'H', '|', 2, '1', '3', '.'])

Cycle through line and marker styles

First we are going to create a cycler object, that we will use to cycle through different styles. Using this object we can have a new line-style every time we plot a new line, and don't have to manually ensure that our lines are monochrome and different.

Cycler objects can be composed of several cycler objects and will iterate over all permutations of its components forever. Let us create a cycler object that cycles through several line and marker styles all with the color black.

In [5]:
from cycler import cycler

# Create cycler object. Use any styling from above you please
monochrome = (cycler('color', ['k']) * cycler('linestyle', ['-', '--', ':', '=.']) * cycler('marker', ['^',',', '.']))

# Print examples of output from cycler object. 
# A cycler object, when called, returns a `iter.cycle` object that iterates over items indefinitely
print("number of items in monochrome:", len(monochrome))
for i, item in zip(range(15), monochrome()):
    print(i, item)

number of items in monochrome: 12
0 {'color': 'k', 'linestyle': '-', 'marker': '^'}
1 {'color': 'k', 'linestyle': '-', 'marker': ','}
2 {'color': 'k', 'linestyle': '-', 'marker': '.'}
3 {'color': 'k', 'linestyle': '--', 'marker': '^'}
4 {'color': 'k', 'linestyle': '--', 'marker': ','}
5 {'color': 'k', 'linestyle': '--', 'marker': '.'}
6 {'color': 'k', 'linestyle': ':', 'marker': '^'}
7 {'color': 'k', 'linestyle': ':', 'marker': ','}
8 {'color': 'k', 'linestyle': ':', 'marker': '.'}
9 {'color': 'k', 'linestyle': '=.', 'marker': '^'}
10 {'color': 'k', 'linestyle': '=.', 'marker': ','}
11 {'color': 'k', 'linestyle': '=.', 'marker': '.'}
12 {'color': 'k', 'linestyle': '-', 'marker': '^'}
13 {'color': 'k', 'linestyle': '-', 'marker': ','}
14 {'color': 'k', 'linestyle': '-', 'marker': '.'}
In [6]:
# ipython can also pretty pring our cycler object
monochrome
Out[6]:
'color''linestyle''marker'
'k''-''^'
'k''-'','
'k''-''.'
'k''--''^'
'k''--'','
'k''--''.'
'k'':''^'
'k'':'','
'k'':''.'
'k''=.''^'
'k''=.'','
'k''=.''.'

Create monochrome figure and axes object

Most people learn matplotlib through pyplot, the command style functions that make matplotlib work like MATLAB. Meanwhile there is also the more direct approach, manupulating matplotlib objects directly. It is my experience that this is more powerful and as far as I can tell, we can't make monochrome plots without using the object-oriented interface, so in this tutorial I will try and use it as much as possible.

It is however a cumbersome interface, so it appears that the people behind matplotlib recommends mixing both, so will I.

First, let us take a look at an empty plot:

In [7]:
plt.plot();

If we draw a number of lines, we can see that the default behavior of matplotlib is to give them different colors:

In [8]:
for i in range(1,5):
    plt.plot(np.arange(10), np.arange(10)*i)

Add a custom cycler

Let us add the monochrome cycler as the default prop_cycle for the next plot. This plot we will generate using the object approach. The subplots function (notice the s) returns a figure object and any number of axes objects we ask it to. I find this the easiest way to get both of these objects, even for plots with only 1 ax.

In [9]:
fig, ax = plt.subplots(1,1)
ax.set_prop_cycle(monochrome)
for i in range(1,5):
    ax.plot(np.arange(10), np.arange(10)*i)

Set a grid and clear the axis for a prettier plot

In [10]:
fig, ax = plt.subplots(1,1)
ax.set_prop_cycle(monochrome)
ax.grid()
ax.spines['top'].set_visible(False)
ax.spines['right'].set_visible(False)
ax.spines['bottom'].set_visible(False)
ax.spines['left'].set_visible(False)
for i in range(1,5):
    ax.plot(np.arange(10), np.arange(10)*i)

Override styles for current script

Writing all the ax.set_grid ... code for every figure is tedious. We can tell matplotlib to set all new figures with a particular style.

All styles are saved in a dictionary in plt.rcParams. We can override its values manually for a single script and will do this now. You can also save your styles manually to a .mplstyle-file and load them at will. See Customizing plots with stylesheets.

You can load custom and builtin styles at will using plt.style.use() function. You can even load and combine several styles.

Below we will just override entries in the rcParams dictionary manually, so that this notebook is not dependent on external files.

In [11]:
# Overriding styles for current script
plt.rcParams['axes.grid'] = True
plt.rcParams['axes.prop_cycle'] = monochrome
plt.rcParams['axes.spines.top'] = False
plt.rcParams['axes.spines.right'] = False
plt.rcParams['axes.spines.bottom'] = False
plt.rcParams['axes.spines.left'] = False

Bar plots

In [12]:
fig, ax = plt.subplots(1,1)

for x in range(1,5):
    ax.bar(x, np.random.randint(2,10))

Now there are 3 problems with this barplot:

  1. The bars are colored
  2. The bars cannot be distinguished
  3. The grid is above the bars (will become a big problem when 1 is solved)

We will color all the bars white and leave the black border. To distinguish the bars using only monochrome colors, we will paint them with hatches - repeating patterns. To place the bars in front of the grid, we will set their zorder to something high.

More on hatches:

In [13]:
fig, ax = plt.subplots(1,1)

bar_cycle = (cycler('hatch', ['///', '--', '...','\///', 'xxx', '\\\\']) * cycler('color', 'w')*cycler('zorder', [10]))
styles = bar_cycle()

for x in range(1,5):
    ax.bar(x, np.random.randint(2,10), **next(styles))

Further reading / sources

In [ ]:

Notes about Keras deeplearning framework

Posted on Mon 27 June 2016 in Notes

Short introduction to Keras

Keras can be thought of as 4 steps

  • Prepare input and output tensors
  • Create first layer (input)
  • Create output layer
  • Build any model inbetween

Everything is a layer!

Layers are minimally defined as output dimensions (input dimensions are optional, typically only required for first layer)

return_sequence, if true output can be feeded to another RNN

map to a sequence

if false, feed to fully connected layers

even dropout is a layer. Makes sense, as dropout can be seen as a

random matrix that will multiply inputs with 1 or 0

models

instantiate a model:

model = Sequential()

Expand a model

 model.add([layer type])

Check a model

model.summary()

Neural Net is implemented as a model

  • Layers are all contained within a model

Sequential model

  • Regular run-of-the-mill NN
  • setup input and output layer
  • one layer feeds into the next

Graph

  • One layer can split into several layers

model.Compile() sets up your model, which loss-function and optimizer that will be used. This will compile the model into machine code via Theano or TensorFlow.

model.fit() is the training function.

Python static typing

Posted on Sun 26 June 2016 in misc

Normally in Python if you create a variable i as an integer, you can later assign a a string value. i just points to some data of some type in memory and can be re-assigned at will.

This can make it hard to reason about code. Is the input to this function a float or an integer? A list or an array?

With static typing a variable can't change its type and it can make reasoning about some code easier. If you know that i is an integer and you know that no-matter what, it will never become a float, then you can always use i as an index. With true static typing, you will even get an error if you try and change i.

In Python you can annotate type and you can have a script check if these annotations are violated at any point in code, as a sort of pre-execution sanity check.

Here is how it works:

from typing import List

def myfunction(list_of_int:List[int], floatingpoint:float) -> bool:
    """ This function takes a list of integers and a
    floating point number and returns a boolean """
    return True

links:

'Hiragana <-> katakana transliteration in 4 lines of Python'

Posted on Thu 26 May 2016 in Notes

This is a quick script to make good hiragana <-> katakana conversion in just 4 lines of Python.

This code will make it easy to convert かたかな to カタカナ and ヒラガナ to ひらがな without any dependencies. It even handles mixed script correctly.

If you don't need romaji translitteration and want to lower your scripts dependencies you can forgo pip installing some surprisingly large libraries just to convert from hiraganan to katakana and simply copy paste the below 4 lines (and preferrably a link to my homepage or github) and you are good to go.

Tested in Python 3.x, doesn't seem to work in Python 2.7

Download it off my github here

How it works

I use the builtin string function translate which converts characters to corrosponding characters in a translations table, easily created with another string function, maketrans. See documentation here

We simply create our hiragana and katakana translation tables and use the str.translate() function to do the heavy lifting.

I've used Mark Rogoyski list of hiragana and katakana unicode codepoints and removed characters I don't want transliterated. For example, I want to be able to convert コーヒ to hiragana and back. If I had naively used the table, then ー would be converted into ゜, which wouldn't make any sense.

The magic happens in these 4 lines of code:

katakana_chart = "ァアィイゥウェエォオカガキギクグケゲコゴサザシジスズセゼソゾタダチヂッツヅテデトドナニヌネノハバパヒビピフブプヘベペホボポマミムメモャヤュユョヨラリルレロヮワヰヱヲンヴヵヶヽヾ"
hiragana_chart = "ぁあぃいぅうぇえぉおかがきぎくぐけげこごさざしじすずせぜそぞただちぢっつづてでとどなにぬねのはばぱひびぴふぶぷへべぺほぼぽまみむめもゃやゅゆょよらりるれろゎわゐゑをんゔゕゖゝゞ" 
hir2kat = str.maketrans(hiragana_chart, katakana_chart)
kat2hir  =str.maketrans(katakana_chart, hiragana_chart)

And it is used like so:

mixed = 'きゃりーぱみゅぱみゅは日本の歌手です。'
print(mixed.translate(hir2kat))
# out: キャリーパミュパミュハ日本ノ歌手デス。

# transliterate back and forth
print(mixed.translate(hir2kat).translate(kat2hir))
# out: きゃりーぱみゅぱみゅは日本の歌手です。

Notice how kanji and special characters are left alone.

PyCNN install notes (for OSX, but also in general)

Posted on Tue 19 April 2016 in Notes

The guide available at https://github.com/clab/cnn/blob/master/INSTALL.md will mostly get you through installing PyCNN with CNN and Eigen (necessary for CNN)

Here's a few notes

  • The PyCNN install guide recommends latest stable version of Eigen, the README file for CNN recommends latest development build. You might have to try both. Either can be installed from http://eigen.tuxfamily.org/ (just unzip and move to cnn folder and rename to eigen and you don't have to mess with mercury/hg)
  • The PyCNN install guide hints that you should use python 2.x. This is not a hint Python 2.7 is required for PyCNN

  • You can add extra compiler flags in setup.py. On OSX you may need to include the following extra_compile_args(remember the trailing comma. These are function arguments!): [‘-mmacosx-version-min=10.7′,'-std=c++11′,'-stdlib=libc++'],

  • You might need to do things several times. Be patient and be stubborn!