Migration to sphinx-gallery #8247

Closed
wants to merge 21 commits into
from
Commits
Jump to file
+225 −305
Split
View
@@ -32,6 +32,7 @@
build
# sphinx build directory
doc/_build
+doc/auto_examples
# setup.py dist directory
dist
# Egg metadata
@@ -58,7 +59,6 @@ lib/matplotlib/mpl-data/matplotlibrc
# Documentation generated files #
#################################
doc/examples
-doc/_templates/gallery.html
doc/users/installing.rst
doc/_static/matplotlibrc
doc/pyplots/tex_demo.png
View
@@ -142,6 +142,7 @@ install:
pip install -ve .
script: source ci/travis/test_script.sh
+
before_cache:
- rm -rf $HOME/.cache/matplotlib/tex.cache
- rm -rf $HOME/.cache/matplotlib/test_cache
View
@@ -30,7 +30,7 @@ if [[ $BUILD_DOCS == false ]]; then
fi
else
cd doc
- python make.py html -n 2
+ python make.py html
@QuLogic

QuLogic Mar 24, 2017

Member

Does parallel build not work?

@NelleV

NelleV Mar 25, 2017

Contributor

No, but ironically, the parallel build doesn't change the run time :D
We can easily activate the flags in sphinx-gallery to make it parallel-safe, but considering there is no gain at all, I haven't created the PR yet.
Actually making the code parallel requires more work, but should be doable.

@tacaswell

tacaswell Apr 9, 2017

Owner

I I have definitely seen good speed up on building the docs (as they are now) but have found you need to use ~2x the number of workers compared to the number of core.

# We don't build the LaTeX docs here, so linkchecker will complain
touch build/html/Matplotlib.pdf
# Linkchecker only works with python 2.7 for the time being
View
@@ -12,3 +12,4 @@ ipython
mock
colorspacious
pillow
+sphinx-gallery
@phobson

phobson Mar 24, 2017

Member

we need to add graphviz to this list

@NelleV

NelleV Mar 24, 2017

Contributor

unfortunately, we can't as this file is meant to be used by pip, and graphviz is not pip installable.

View
@@ -21,7 +21,8 @@ All of these dependencies *except graphviz* can be installed through pip::
or all of them via conda and pip::
- conda install sphinx numpydoc ipython mock graphviz pillow
+ conda install sphinx numpydoc ipython mock graphviz pillow \
+ sphinx-gallery
pip install colorspacious
To build the HTML documentation, type ``python make.py html`` in this
@@ -75,7 +75,8 @@
<p>Matplotlib tries to make easy things easy and hard things possible.
You can generate plots, histograms, power spectra, bar charts,
errorcharts, scatterplots, etc., with just a few lines of code.
- For a sampling, see the <a href="{{ pathto('users/screenshots') }}">screenshots</a>, <a href="{{ pathto('gallery') }}">thumbnail</a> gallery, and
+ For a sampling, see the <a href="{{ pathto('users/screenshots')
+ }}">screenshots</a>, <a href="{{ pathto('auto_examples/index') }}">thumbnail</a> gallery, and
@tacaswell

tacaswell Apr 9, 2017

Owner

I assume the linebreak does not affect the templating?

@NelleV

NelleV Apr 9, 2017

Contributor

No, it works fine

<a href="{{ pathto('examples/index') }}">examples</a> directory</p>
<p>For simple plotting the <tt>pyplot</tt> module provides a
@@ -99,7 +100,7 @@
</script>
<p>Trying to learn how to do a particular kind of plot? Check out
- the <a href="{{ pathto('gallery') }}">gallery</a>, <a href="{{ pathto('examples/index') }}">examples</a>,
+ the <a href="{{ pathto('auto_examples/index') }}">gallery</a>, <a href="{{ pathto('examples/index') }}">examples</a>,
or the <a href="{{ pathto('api/pyplot_summary') }}">list of plotting
commands</a>.</p>
@@ -39,7 +39,7 @@
<li><a href="{{ pathto('index') }}">home</a>|&nbsp;</li>
<li><a href="{{ pathto('examples/index') }}">examples</a>|&nbsp;</li>
- <li><a href="{{ pathto('gallery') }}">gallery</a>|&nbsp;</li>
+ <li><a href="{{ pathto('auto_examples/index') }}">gallery</a>|&nbsp;</li>
<li><a href="{{ pathto('api/pyplot_summary') }}">pyplot</a>|&nbsp;</li>
<li><a href="{{ pathto('contents') }}">docs</a> &raquo;</li>
View
@@ -29,9 +29,10 @@
extensions = ['matplotlib.sphinxext.mathmpl', 'sphinxext.math_symbol_table',
'sphinx.ext.autodoc', 'matplotlib.sphinxext.only_directives',
'sphinx.ext.doctest', 'sphinx.ext.autosummary',
- 'matplotlib.sphinxext.plot_directive',
'sphinx.ext.inheritance_diagram',
- 'sphinxext.gen_gallery', 'sphinxext.gen_rst',
+ 'sphinx_gallery.gen_gallery',
+ 'sphinxext.gen_rst',
+ 'matplotlib.sphinxext.plot_directive',
'sphinxext.github',
'numpydoc']
@@ -54,6 +55,12 @@
"numpydoc to build the documentation.")
try:
+ import sphinx_gallery
+except ImportError:
+ raise ImportError("No module named sphinx_gallery - you need to install "
+ "sphinx_gallery to build the documentation.")
+
+try:
import colorspacious
except ImportError:
raise ImportError("No module named colorspacious - you need to install "
@@ -96,6 +103,16 @@
autodoc_docstring_signature = True
+
+# Sphinx gallery configuration
+sphinx_gallery_conf = {
+ # path to your examples scripts
+ 'examples_dirs' : '../examples',
@QuLogic

QuLogic Mar 24, 2017

Member

Does this mean we can delete the mpl_examples symlink?

@NelleV

NelleV Mar 25, 2017

Contributor

Not yet. Our plot directive writes into mpl_examples, and unfortunately we have a bit more work to do before getting rid of this (including more examples to migrate). This will be my next step.

+ # path where to save gallery generated examples
+ 'gallery_dirs' : 'auto_examples'}
@QuLogic

QuLogic Mar 24, 2017

Member

Can we not put it in gallery instead? Or if it were mpl_examples, then some of the lines that were changed would not have been necessary.

Whatever it becomes, I don't like the auto_ prefix; it's an internal concern and doesn't really make any sense to present that in the URL to a user from a semantic point of view.

@NelleV

NelleV Mar 25, 2017

Contributor

It's the default and is a convention followed by many (if not all) the projects. I'd rather stick with it for consistency.
We can't rename this to mpl_examples as mpl_examples still exists.

@QuLogic

QuLogic Mar 26, 2017

Member

I'm not sure those are compelling reasons. I went back through the blame 7 years and couldn't find any reasoning behind it (that's not to say that there wasn't one.) My reasoning here is that URLs should be discoverable, simple, and at least try to give some semantic meaning.

A user might go to matplotlib.org/gallery or matplotlib.org/examples directly (yes, there's Google, but let's assume they're not using it for some reason), but there's a 99% chance that no-one will spontaneously go to matplotlib.org/auto_examples.

And what does the "auto_" prefix mean? It means that the rst was automatically generated from the example Python code. This is a meaningless distinction to a user. When they see "auto", they see "automatically generated" aka "not hand-written" aka "not very good". I'm not going to claim that all of our examples are 100% clear, but they've all been hand-written in some fashion, and some of them are even quite amazing.

In fact, for that same reasoning, I've just talked myself into retracting the "mpl_examples" suggestion, since the "mpl_" prefix is redundant and meaningless in this context. Also, our existing site uses matplotlib.org/examples/ and matplotlib.org/gallery.html; I don't think we should break that.

@NelleV

NelleV Mar 26, 2017

Contributor

For the time being, the folder cannot be neither "examples" nor "mpl_examples", and this will not change in the foreseeable future considering the state of our examples (unless we remove all examples that are not SG compatible).

@dopplershift

dopplershift Mar 27, 2017

Contributor

It's not possible to rename the current examples/mpl_examples to make room for the gallery-based examples? I strongly dislike the auto_examples name as well--I've overridden that name for sphinx-gallery in my own projects.

@NelleV

NelleV Mar 28, 2017

Contributor

It is possible, but (1) so far there has been no compelling reason to do it; (2) the amount of work required to change this is sufficiently large that I am not doing it.

To extend on point 1, our naming patterns for example plots and links is obscure, inconsistent and so poorly descriptive of what is in the examples that I really don't think the name of the folder is ever going to help or prevent our users from finding anything (we have about 5 examples named "demo bar_X", with X an incremental number). We also need to be realistic: no one is ever going to guess our url or even attempt to guess our url, and possibly not even look at the url.
Second, it is a convention that is followed by most projects and following conventions have many benefits for easing contribution, maintenance, etc.
Third, taking personal preferences into account is not for me a compelling reason (I personally prefer auto_examples over both examples and mpl_examples, as it makes it to a certain extent more clear that this folder is auto-generated and thus should not be modified by our contributors, and this is typically the kind of questions where if you have N developers in the room, you'll have N opinions).

@tacaswell

tacaswell Apr 9, 2017

Owner

Will this end up exposed in the URL?

If so, I think we should pick something either a) shorter (ex sg or ae) or b) something more meaningful.

@choldgraf

choldgraf Apr 9, 2017

Contributor

I does show up in the URL, for example this is our gallery at MNE-python. It does have auto-examples in the page, though that's sorta become the convention with sphinx-gallery (same goes for scikit-learn: http://scikit-learn.org/stable/auto_examples/index.html).

I seem to remember conversations with folks that this was a non-trivial thing to change but I could be wrong...it's been working out well enough for us. If you want you could use an .rst file that contains the gallery itself, then the objects that the gallery links to would be in the auto_examples folder.

@dopplershift

dopplershift Apr 9, 2017

Contributor

Nope, it's completely trivial to change: https://unidata.github.io/MetPy/examples/index.html

I'm going to argue one more time NOT to use auto_examples. Whether they're generated automatically or not is immaterial to the user, so we should NOT be bleeding that implementation detail of our documentation build procedure to the outside world.

@NelleV

NelleV Apr 10, 2017

Contributor

It's not that trivial, because matplotlib somehow does I don't know what in the plot directive that makes this a pain in the butt to change. Which is why I asked for feedback a month ago.

@choldgraf

choldgraf Apr 10, 2017

Contributor

FWIW I'll give this a shot to change the name to examples_gallery. It doesn't look completely trivial in the sense that there are links hard-coded all over the codebase. There are also some weird errors happening inside the plot directive that I can try to debug. I'll open a separate PR then if I can figure it out.

+
+plot_gallery = True
@QuLogic

QuLogic Mar 24, 2017

Member

Blank line after.

+
# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']
@@ -234,7 +251,6 @@
# Additional templates that should be rendered to pages, maps page names to
# template names.
html_additional_pages = {'index': 'index.html',
- 'gallery':'gallery.html',
'citing': 'citing.html'}
# If false, no module index is generated.
@@ -442,7 +442,7 @@ We have hundreds of examples in subdirectories of
:file:`matplotlib/examples`, and these are automatically generated
when the website is built to show up both in the `examples
<../examples/index.html>`_ and `gallery
-<../gallery.html>`_ sections of the website.
+<../auto_examples/index.html>`_ sections of the website.
Any sample data that the example uses should be kept small and
distributed with Matplotlib in the
@@ -30,7 +30,7 @@ plotting (scatter, surf, line, mesh) tools. Not the fastest or feature
complete 3D library out there, but ships with Matplotlib and thus may be a
lighter weight solution for some use cases.
-.. plot:: mpl_examples/mplot3d/contourf3d_demo2.py
+.. plot:: auto_examples/mplot3d/plot_contourf3d_2.py
.. _toolkit_axes_grid1:
@@ -36,41 +36,41 @@ Line plots
====================
.. automethod:: Axes3D.plot
-.. plot:: mpl_examples/mplot3d/lines3d_demo.py
+.. plot:: auto_examples/mplot3d/plot_lines3d.py
.. _scatter3d:
Scatter plots
=============
.. automethod:: Axes3D.scatter
-.. plot:: mpl_examples/mplot3d/scatter3d_demo.py
+.. plot:: auto_examples/mplot3d/plot_scatter3d.py
.. _wireframe:
Wireframe plots
===============
.. automethod:: Axes3D.plot_wireframe
-.. plot:: mpl_examples/mplot3d/wire3d_demo.py
+.. plot:: auto_examples/mplot3d/plot_wire3d.py
.. _surface:
Surface plots
=============
.. automethod:: Axes3D.plot_surface
-.. plot:: mpl_examples/mplot3d/surface3d_demo.py
-.. plot:: mpl_examples/mplot3d/surface3d_demo2.py
-.. plot:: mpl_examples/mplot3d/surface3d_demo3.py
+.. plot:: auto_examples/mplot3d/plot_surface3d.py
+.. plot:: auto_examples/mplot3d/plot_surface3d_2.py
+.. plot:: auto_examples/mplot3d/plot_surface3d_3.py
.. _trisurface:
Tri-Surface plots
=================
.. automethod:: Axes3D.plot_trisurf
-.. plot:: mpl_examples/mplot3d/trisurf3d_demo.py
+.. plot:: auto_examples/mplot3d/plot_trisurf3d.py
.. _contour3d:
@@ -79,18 +79,18 @@ Contour plots
=============
.. automethod:: Axes3D.contour
-.. plot:: mpl_examples/mplot3d/contour3d_demo.py
-.. plot:: mpl_examples/mplot3d/contour3d_demo2.py
-.. plot:: mpl_examples/mplot3d/contour3d_demo3.py
+.. plot:: auto_examples/mplot3d/plot_contour3d.py
+.. plot:: auto_examples/mplot3d/plot_contour3d_2.py
+.. plot:: auto_examples/mplot3d/plot_contour3d_3.py
.. _contourf3d:
Filled contour plots
====================
.. automethod:: Axes3D.contourf
-.. plot:: mpl_examples/mplot3d/contourf3d_demo.py
-.. plot:: mpl_examples/mplot3d/contourf3d_demo2.py
+.. plot:: auto_examples/mplot3d/plot_contourf3d.py
+.. plot:: auto_examples/mplot3d/plot_contourf3d_2.py
.. versionadded:: 1.1.0
The feature demoed in the second contourf3d example was enabled as a
@@ -102,37 +102,37 @@ Polygon plots
====================
.. automethod:: Axes3D.add_collection3d
-.. plot:: mpl_examples/mplot3d/polys3d_demo.py
+.. plot:: auto_examples/mplot3d/plot_polys3d.py
.. _bar3d:
Bar plots
====================
.. automethod:: Axes3D.bar
-.. plot:: mpl_examples/mplot3d/bars3d_demo.py
+.. plot:: auto_examples/mplot3d/plot_bars3d.py
.. _quiver3d:
Quiver
====================
.. automethod:: Axes3D.quiver
-.. plot:: mpl_examples/mplot3d/quiver3d_demo.py
+.. plot:: auto_examples/mplot3d/plot_quiver3d.py
.. _2dcollections3d:
2D plots in 3D
====================
-.. plot:: mpl_examples/mplot3d/2dcollections3d_demo.py
+.. plot:: auto_examples/mplot3d/plot_2dcollections3d.py
.. _text3d:
Text
====================
.. automethod:: Axes3D.text
-.. plot:: mpl_examples/mplot3d/text3d_demo.py
+.. plot:: auto_examples/mplot3d/plot_text3d.py
.. _3dsubplots:
@@ -146,5 +146,5 @@ in the same figure.
Subplotting 3D plots was added in v1.0.0. Earlier version can not
do this.
-.. plot:: mpl_examples/mplot3d/subplot3d_demo.py
-.. plot:: mpl_examples/mplot3d/mixed_subplots_demo.py
+.. plot:: auto_examples/mplot3d/plot_subplot3d.py
+.. plot:: auto_examples/mplot3d/plot_mixed_subplots.py
Oops, something went wrong.