Sidecar

& Other Jupyter Frontend Diversions

Kyle Kelley - Dev @ Rackspace

Sidecar

  • jupyter console
  • Rich output off to the side
  • github.com/rgbkrk/jupyter-sidecar
$ jupyter console
Jupyter Console 4.0.0-dev

[ZMQTerminalIPythonApp] Loading IPython extension: storemagic

In [1]: %matplotlib inline

In [2]: import pandas as pd

In [3]: import numpy as np

In [4]: from matplotlib import pyplot as plt

In [5]: import seaborn as sns

In [6]: ts = pd.Series(np.random.randn(1000), index=pd.date_range('1/1/2000', periods=1000))

In [7]: ts = ts.cumsum()

In [8]: df = pd.DataFrame(np.random.randn(1000, 4), index=ts.index,
   ...:                   columns=['A', 'B', 'C', 'D'])

In [9]: df = df.cumsum()

In [10]: df.head()
Out[10]:
                   A         B         C         D
2000-01-01  1.153881 -0.462802 -1.304602 -0.112967
2000-01-02  2.041548  0.193145 -2.154538 -0.026075
2000-01-03  2.277393 -0.990363 -1.676727  0.463335
2000-01-04  2.472088  0.376239 -2.691538 -0.329923
2000-01-05  3.426450 -0.052412 -3.275203 -1.422521

In [11]: plt.figure(); df.plot(); plt.legend(loc='best')
Out[11]: <matplotlib.legend.Legend at 0x107ddffd0>
<matplotlib.figure.Figure at 0x107ddf358>

j.mp/jscar

Now
For
Diversions

How Jupyter Output Works

reprs
In [16]: df._repr_html_?
Signature: df._repr_html_()
Docstring:
Return a html representation for a particular DataFrame.
Mainly for IPython notebook.
File:      /usr/local/lib/python3.4/site-packages/pandas/core/frame.py
Type:      method
    def _repr_html_(self):
        """
        Return a html representation for a particular DataFrame.
        Mainly for IPython notebook.
        """
        # qtconsole doesn't report its line width, and also
        # behaves badly when outputting an HTML table
        # that doesn't fit the window, so disable it.
        # XXX: In IPython 3.x and above, the Qt console will not attempt to
        # display HTML, so this check can be removed when support for IPython 2.x
        # is no longer needed.
        if com.in_qtconsole():
            # 'HTML output is disabled in QtConsole'
            return None

        if self._info_repr():
            buf = StringIO(u(""))
            self.info(buf=buf)
            # need to escape the <class>, should be the first line.
            val = buf.getvalue().replace('<', r'<', 1).replace('>',
                                                                  r'>', 1)
            return '<pre>' + val + '</pre>'

        if get_option("display.notebook_repr_html"):
            max_rows = get_option("display.max_rows")
            max_cols = get_option("display.max_columns")
            show_dimensions = get_option("display.show_dimensions")

            return ('<div style="max-height:1000px;'
                    'max-width:1500px;overflow:auto;">\n' +
                    self.to_html(max_rows=max_rows, max_cols=max_cols,
                                 show_dimensions=show_dimensions) + '\n</div>')
Jupyter Message Protocol
{ 
  content:
   { data:
      { 'text/plain': '                   A         B         C         D\n2000-01-01  1.153881 -0.462802 -1.304602 -0.112967\n2000-01-02  2.041548  0.193145 -2.154538 -0.026075\n2000-01-03  2.277393 -0.990363 -1.676727  0.463335\n2000-01-04  2.472088  0.376239 -2.691538 -0.329923\n2000-01-05  3.426450 -0.052412 -3.275203 -1.422521',
        'text/html': '<div style="max-height:1000px;max-width:1500px;overflow:auto;">\n<table border="1" class="dataframe">\n  <thead>\n    <tr style="text-align: right;">\n      <th></th>\n      <th>A</th>\n      <th>B</th>\n      <th>C</th>\n      <th>D</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>2000-01-01</th>\n      <td> 1.153881</td>\n      <td>-0.462802</td>\n      <td>-1.304602</td>\n      <td>-0.112967</td>\n    </tr>\n    <tr>\n      <th>2000-01-02</th>\n      <td> 2.041548</td>\n      <td> 0.193145</td>\n      <td>-2.154538</td>\n      <td>-0.026075</td>\n    </tr>\n    <tr>\n      <th>2000-01-03</th>\n      <td> 2.277393</td>\n      <td>-0.990363</td>\n      <td>-1.676727</td>\n      <td> 0.463335</td>\n    </tr>\n    <tr>\n      <th>2000-01-04</th>\n      <td> 2.472088</td>\n      <td> 0.376239</td>\n      <td>-2.691538</td>\n      <td>-0.329923</td>\n    </tr>\n    <tr>\n      <th>2000-01-05</th>\n      <td> 3.426450</td>\n      <td>-0.052412</td>\n      <td>-3.275203</td>\n      <td>-1.422521</td>\n    </tr>\n  </tbody>\n</table>\n</div>' },
     execution_count: 21,
     metadata: {} },
  header:
   { username: 'kyle6475',
     msg_id: 'e697b50a-9aa0-4d7a-a825-27ac99479679',
     msg_type: 'execute_result',
     version: '5.0',
     date: '2015-07-26T15:52:59.473133',
     session: 'c297c1c0-89bb-4ded-9720-d298c33b2595' },
  parentHeader:
   { username: 'kyle6475',
     msg_id: '841e2314-0af6-46c8-8083-d4b544f91e0e',
     version: '5.0',
     session: 'b4e75019-83e8-4bd4-99f5-e462d5203886',
     date: '2015-07-26T15:52:59.468333',
     msg_type: 'execute_request' },
  metadata: {},
  signature: '6c41e0ecbbb50010c26d8e04e8ff2c634816761f610b30ab1765539c216be591',
  blobs: []
}

http://jupyter-client.readthedocs.org/en/latest/messaging.html

Jupyter Message Protocol
{ 
  content:
   { data:
      { 'text/plain': '                   A         B         C         D\n2000-01-01  1.153881 -0.462802 -1.304602 -0.112967\n2000-01-02  2.041548  0.193145 -2.154538 -0.026075\n2000-01-03  2.277393 -0.990363 -1.676727  0.463335\n2000-01-04  2.472088  0.376239 -2.691538 -0.329923\n2000-01-05  3.426450 -0.052412 -3.275203 -1.422521',
        'text/html': '<div style="max-height:1000px;max-width:1500px;overflow:auto;">\n<table border="1" class="dataframe">\n  <thead>\n    <tr style="text-align: right;">\n      <th></th>\n      <th>A</th>\n      <th>B</th>\n      <th>C</th>\n      <th>D</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>2000-01-01</th>\n      <td> 1.153881</td>\n      <td>-0.462802</td>\n      <td>-1.304602</td>\n      <td>-0.112967</td>\n    </tr>\n    <tr>\n      <th>2000-01-02</th>\n      <td> 2.041548</td>\n      <td> 0.193145</td>\n      <td>-2.154538</td>\n      <td>-0.026075</td>\n    </tr>\n    <tr>\n      <th>2000-01-03</th>\n      <td> 2.277393</td>\n      <td>-0.990363</td>\n      <td>-1.676727</td>\n      <td> 0.463335</td>\n    </tr>\n    <tr>\n      <th>2000-01-04</th>\n      <td> 2.472088</td>\n      <td> 0.376239</td>\n      <td>-2.691538</td>\n      <td>-0.329923</td>\n    </tr>\n    <tr>\n      <th>2000-01-05</th>\n      <td> 3.426450</td>\n      <td>-0.052412</td>\n      <td>-3.275203</td>\n      <td>-1.422521</td>\n    </tr>\n  </tbody>\n</table>\n</div>' },
     execution_count: 21,
     metadata: {} },
  header:
   { username: 'kyle6475',
     msg_id: 'e697b50a-9aa0-4d7a-a825-27ac99479679',
     msg_type: 'execute_result',
     version: '5.0',
     date: '2015-07-26T15:52:59.473133',
     session: 'c297c1c0-89bb-4ded-9720-d298c33b2595' },
  parentHeader:
   { username: 'kyle6475',
     msg_id: '841e2314-0af6-46c8-8083-d4b544f91e0e',
     version: '5.0',
     session: 'b4e75019-83e8-4bd4-99f5-e462d5203886',
     date: '2015-07-26T15:52:59.468333',
     msg_type: 'execute_request' },
  metadata: {},
  signature: '6c41e0ecbbb50010c26d8e04e8ff2c634816761f610b30ab1765539c216be591',
  blobs: []
}

http://jupyter-client.readthedocs.org/en/latest/messaging.html

mime bundle
{
  'text/plain': 
    '                   A         B         C         D\n
     2000-01-01  1.153881 -0.462802 -1.304602 -0.112967\n
     2000-01-02  2.041548  0.193145 -2.154538 -0.026075\n
     2000-01-03  2.277393 -0.990363 -1.676727  0.463335\n
     2000-01-04  2.472088  0.376239 -2.691538 -0.329923\n
     2000-01-05  3.426450 -0.052412 -3.275203 -1.422521',

  'text/html': 
    '<div style="max-height:1000px;max-width:1500px;overflow:auto;">\n
       <table border="1" class="dataframe">\n  <thead>\n
         <tr style="text-align: right;">\n
         <th></th>\n      <th>A</th>\n      <th>B</th>\n      <th>C</th>\n      <th>D</th>\n    </tr>\n  </thead>\n
         <tbody>\n
           <tr>\n
             <th>2000-01-01</th>\n      <td> 1.153881</td>\n      <td>-0.462802</td>\n      <td>-1.304602</td>\n      <td>-0.112967</td>\n</tr>\n
           <tr>\n
             <th>2000-01-02</th>\n      <td> 2.041548</td>\n      <td> 0.193145</td>\n      <td>-2.154538</td>\n      <td>-0.026075</td>\n</tr>\n
           <tr>\n
             <th>2000-01-03</th>\n      <td> 2.277393</td>\n      <td>-0.990363</td>\n      <td>-1.676727</td>\n      <td> 0.463335</td>\n</tr>\n
           <tr>\n
             <th>2000-01-04</th>\n      <td> 2.472088</td>\n      <td> 0.376239</td>\n      <td>-2.691538</td>\n      <td>-0.329923</td>\n</tr>\n
           <tr>\n
             <th>2000-01-05</th>\n      <td> 3.426450</td>\n      <td>-0.052412</td>\n      <td>-3.275203</td>\n      <td>-1.422521</td>\n</tr>\n
         </tbody>\n
       </table>\n
     </div>'
}
💥 converted 💥
{
  'text/plain': 
    '                   A         B         C         D\n
     2000-01-01  1.153881 -0.462802 -1.304602 -0.112967\n
     2000-01-02  2.041548  0.193145 -2.154538 -0.026075\n
     2000-01-03  2.277393 -0.990363 -1.676727  0.463335\n
     2000-01-04  2.472088  0.376239 -2.691538 -0.329923\n
     2000-01-05  3.426450 -0.052412 -3.275203 -1.422521',

  'text/html': 
    '<div style="max-height:1000px;max-width:1500px;overflow:auto;">\n
       <table border="1" class="dataframe">\n  <thead>\n
         <tr style="text-align: right;">\n
         <th></th>\n      <th>A</th>\n      <th>B</th>\n      <th>C</th>\n      <th>D</th>\n    </tr>\n  </thead>\n
         <tbody>\n
           <tr>\n
             <th>2000-01-01</th>\n      <td> 1.153881</td>\n      <td>-0.462802</td>\n      <td>-1.304602</td>\n      <td>-0.112967</td>\n</tr>\n
           <tr>\n
             <th>2000-01-02</th>\n      <td> 2.041548</td>\n      <td> 0.193145</td>\n      <td>-2.154538</td>\n      <td>-0.026075</td>\n</tr>\n
           <tr>\n
             <th>2000-01-03</th>\n      <td> 2.277393</td>\n      <td>-0.990363</td>\n      <td>-1.676727</td>\n      <td> 0.463335</td>\n</tr>\n
           <tr>\n
             <th>2000-01-04</th>\n      <td> 2.472088</td>\n      <td> 0.376239</td>\n      <td>-2.691538</td>\n      <td>-0.329923</td>\n</tr>\n
           <tr>\n
             <th>2000-01-05</th>\n      <td> 3.426450</td>\n      <td>-0.052412</td>\n      <td>-3.275203</td>\n      <td>-1.422521</td>\n</tr>\n
         </tbody>\n
       </table>\n
     </div>'
}

Working
Outside
the Notebook

O'Reilly Media's Thebe

https://github.com/oreillymedia/thebe

Extract Packages

Transformime

https://github.com/nteract/transformime

Transformime

Mimetype + data / Mime bundle
  ⬇️
 HTMLElement

npm install transformime
> var mimeBundle = {'text/html': "<h1>Woo</h1>", 'text/plain': "Woo"}
> p1 = transformime.transform({'text/html': '<h1>Woo</h1>'}, document)
> p1.then(function(result){
...   console.log(result.el.innerHTML);
...   console.log(result.el.textContent);
... });

> var p4 = transformer.transformAll(mimes, document);
> els = p4.valueOf()
> 
> els.forEach(function(bundle) {
...   console.log(bundle.mimetype + ": " + bundle.el.innerHTML);
... });
<h1>Woo</h1>

Thank you!

Reach me on Twitter & GitHub

@rgbkrk

developer.rackspace.com

jupyter.org

Sidecar and Frontend Diversions

By Kyle Kelley

Made with Slides.com

Sidecar and Frontend Diversions

  • 2,562

More from Kyle Kelley