15. Stingray Build¶
The source for stingray
is a Sphinx project that depends on PyLit.
Yes. The documentation spawns the code.
Note
PyLit Feature
The first line of each file should be .. #!/usr/bin/env python3
.
The four spaces between ..
and #!
defines the indent used for
each line of code in the file.
Four spaces.
Stingray depends on the following
xlrd. http://www.lexicon.net/sjmachin/xlrd.htm
Version 0.9.2 is Python 3 compatible. https://pypi.python.org/pypi/xlrd/0.9.2
In addition to Python 3.3, there are two other projects used to build.
- PyLit. https://github.com/slott56/PyLit-3
- Sphinx. http://sphinx.pocoo.org/
The PyLit install is little more than a download and move the pylit.py
file to
the Python site-packages
directory.
Sphinx and XLRD should be installed with easy_install.
easy_install xlrd
easy_install sphinx
In the case of having Python2 and Python3 installed, easy_install-3.3
may be required.
On most systems, sudo
is also required.
The diagrams are done with YUML. See http://yuml.me.
15.1. Build Procedure¶
Bootstrap the
build.py
script by running PyLit.python3 -m pylit -t source/build.rst build.py
This reports that an extract was written to
build.py
.Use the
build.py
script to create thestingray
source, unit tests, demonstration applications. Build the Sphinx documentation. And run the unit test, too.python3 build.py
At the end of this step, the directory tree will include the following.
build
. The documentation. In HTML.demo
. Some demonstration applications that usestingray
. See Stingray Demo Applications.stingray
. The Python library, ready for installation.test
. The unit test script.
This reports, also, that 174 tests were run.
In general (i.e., any OS except Windows), it’s sensible to do this:
chmod +x build.py
This allows us to use the following for a rebuild:
./build.py
15.2. Build Script Design¶
This is a platform-independent build.py
file for the build script.
This can use from sphinx.application import Sphinx
and import pylit
to access these modules from within Python
instead of using command-line shell script techniques.
15.2.1. Overheads¶
We’re going to make use of three “applications”.
- Sphinx top-level application.
- PyLit top-level application.
- Unittest top-level test runner.
"""Platform-independent build script"""
from __future__ import print_function
import os
import sys
import errno
from sphinx.application import Sphinx
import pylit
import unittest
import logging
import shutil
15.2.2. Sphinx Build¶
-
sphinx_build
(srcdir, outdir, buildername='html')¶ Handle the simple use case for the
sphinx-build
script.
def sphinx_build( srcdir, outdir, buildername='html' ):
"""Essentially: ``sphinx-build $* -b html source build/html``"""
confdir= srcdir= os.path.abspath( srcdir )
outdir= os.path.abspath( outdir )
doctreedir = os.path.join(outdir, '.doctrees')
app = Sphinx(srcdir, confdir, outdir, doctreedir, buildername)
app.build(force_all=False, filenames=[])
return app.statuscode
15.2.3. PyLit Build¶
-
pylit_build
(srcdir, outdir)¶ Handle the simple use case for PyLit.
This also handles the necessary rewrite to modify standard paths to Windows paths.
def pylit_build( infile, outfile ):
"""Essentially: ``python3 -m pylit -t source/demo/data_quality.rst demo/test.py``
The issue here is that we need to provide platform-specific paths.
"""
if os.sep != '/':
# Fix windows paths.
infile= os.path.join( *infile.split('/') )
outfile= os.path.join( *outfile.split('/') )
pylit.main( txt2code= True, overwrite="yes", infile= infile, outfile= outfile )
15.2.4. Make Directories¶
-
mkdir
(path)¶ Handles the simple use case for assuring that the directory tree exists.
This also handles a rewrite to modify standard paths to Windows paths.
def mkdir( path ):
if os.sep != '/':
# Fix windows paths.
path= os.path.join( *path.split('/') )
try:
os.makedirs( path )
except OSError as e:
if e.errno == errno.EEXIST:
pass
else:
raise
15.2.5. Copy Data File¶
-
copy_file
(srcdir, outdir)¶ Handles the simple use case for copying a file.
def copy_file( srcdir, outdir ):
"""Essentially: ``cp srcdir outdir``"""
shutil.copy2( srcdir, outdir )
15.2.6. Run the Test Script¶
-
run_test
()¶ In effect, this does
python3 test/main.py
def run_test():
import test.main
result= test.main.main()
if result.failures:
sys.exit(result.failures)
15.2.7. The Build Sequence¶
def build():
mkdir( 'stingray/schema' )
mkdir( 'stingray/cobol' )
mkdir( 'stingray/workbook' )
pylit_build( 'source/stingray_init.rst', 'stingray/__init__.py' )
pylit_build( 'source/cell.rst', 'stingray/cell.py' )
pylit_build( 'source/sheet.rst', 'stingray/sheet.py' )
pylit_build( 'source/workbook/init.rst', 'stingray/workbook/__init__.py' )
pylit_build( 'source/workbook/base.rst', 'stingray/workbook/base.py' )
pylit_build( 'source/workbook/csv.rst', 'stingray/workbook/csv.py' )
pylit_build( 'source/workbook/xls.rst', 'stingray/workbook/xls.py' )
pylit_build( 'source/workbook/xlsx.rst', 'stingray/workbook/xlsx.py' )
pylit_build( 'source/workbook/ods.rst', 'stingray/workbook/ods.py' )
pylit_build( 'source/workbook/numbers_09.rst', 'stingray/workbook/numbers_09.py' )
pylit_build( 'source/workbook/numbers_13.rst', 'stingray/workbook/numbers_13.py' )
pylit_build( 'source/workbook/fixed.rst', 'stingray/workbook/fixed.py' )
pylit_build( 'source/schema.rst', 'stingray/schema/__init__.py' )
pylit_build( 'source/schema_loader.rst', 'stingray/schema/loader.py' )
pylit_build( 'source/cobol_init.rst', 'stingray/cobol/__init__.py' )
pylit_build( 'source/cobol_loader.rst', 'stingray/cobol/loader.py' )
pylit_build( 'source/cobol_defs.rst', 'stingray/cobol/defs.py' )
pylit_build( 'source/snappy.rst', 'stingray/snappy.py' )
pylit_build( 'source/protobuf.rst', 'stingray/protobuf.py' )
pylit_build( 'source/installation.rst', 'setup.py' )
copy_file( 'source/Numbers.json', 'stingray/Numbers.json' )
copy_file( 'source/Common.json', 'stingray/Common.json' )
mkdir( 'test' )
pylit_build( 'source/testing/test_init.rst', 'test/__init__.py' )
pylit_build( 'source/testing/main.rst', 'test/main.py' )
pylit_build( 'source/testing/cell.rst', 'test/cell.py' )
pylit_build( 'source/testing/sheet.rst', 'test/sheet.py' )
pylit_build( 'source/testing/schema.rst', 'test/schema.py' )
pylit_build( 'source/testing/schema_loader.rst', 'test/schema_loader.py' )
pylit_build( 'source/testing/workbook.rst', 'test/workbook.py' )
pylit_build( 'source/testing/cobol.rst', 'test/cobol.py' )
pylit_build( 'source/testing/cobol_loader.rst', 'test/cobol_loader.py' )
pylit_build( 'source/testing/cobol_2.rst', 'test/cobol_2.py' )
pylit_build( 'source/testing/snappy_protobuf.rst', 'test/snappy_protobuf.py' )
mkdir( 'demo' )
pylit_build( 'source/demo/data_quality.rst', 'demo/test.py' )
pylit_build( 'source/demo/validation.rst', 'demo/app.py' )
pylit_build( 'source/demo/profile.rst', 'demo/profile.py' )
pylit_build( 'source/demo/cobol_reader.rst', 'demo/cobol_reader.py' )
run_test()
sphinx_build( 'source', 'build/html', 'html' )
sphinx_build( 'source', 'build/latex', 'latex' )
15.2.8. Main Program Switch¶
When the build.py
script is run from the command line,
it will execute the build()
function. When it is imported,
however, it will do nothing special.
if __name__ == "__main__":
build()
15.3. Additional Builds¶
Sometimes it’s desriable to refresh the documentation.
The HTML pages are built with this command.
sphinx-build $* -b html source build/html
The LaTeX document is built with this command.
sphinx-build $* -b latex source build/latex