<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html> <head> <title>[pytest-overview] </title> <meta content="text/html;charset=ISO-8859-1" name="Content-Type"/> <link href="../style.css" media="screen" rel="stylesheet" type="text/css"/></head> <body> <div><a href="http://codespeak.net"><img alt="py lib" height="114" id="pyimg" src="http://codespeak.net/img/pylib.png" width="154"/></a></div> <div id="metaspace"> <div class="project_title">[pytest-overview] </div> <div id="menubar"><a class="menu" href="../index.html">index</a> <a class="menu" href="../../../apigen/api/index.html">api</a> <a class="menu" href="../../../apigen/source/index.html">source</a> <a class="menu" href="../contact.html">contact</a> <a class="menu" href="../download.html">download</a></div></div> <div id="contentspace"> <div id="docinfoline"> <div style="float: right; font-style: italic;"> </div></div> <div class="document" id="py-test-flexible-and-powerful-automated-testing"> <h1 class="title">py.test - flexible and powerful automated testing</h1> <table class="docinfo" frame="void" rules="none"> <col class="docinfo-name" /> <col class="docinfo-content" /> <tbody valign="top"> <tr><th class="docinfo-name">Authors:</th> <td>Holger Krekel <br />merlinux GmbH</td></tr> <tr><th class="docinfo-name">Date:</th> <td>13th March 2006</td></tr> </tbody> </table> <!-- Definitions of interpreted text roles (classes) for S5/HTML data. --> <!-- This data file has been placed in the public domain. --> <!-- Colours ======= --> <!-- Text Sizes ========== --> <!-- Display in Slides (Presentation Mode) Only ========================================== --> <!-- Display in Outline Mode Only ============================ --> <!-- Display in Print Only ===================== --> <!-- Display in Handout Mode Only ============================ --> <!-- Incremental Display =================== --> <div class="section" id="intro-benefits-of-automated-testing"> <h1>Intro: Benefits of Automated Testing</h1> <ul class="simple"> <li>prove that code changes actually fix a certain issue</li> <li>minimizing Time to Feedback for developers</li> <li>reducing overall Time to Market</li> <li>document usage of plugins</li> <li>tests as a means of communication</li> <li>easing entry for newcomers</li> </ul> </div> <div class="section" id="py-test-purposes-goals"> <h1>py.test Purposes & Goals</h1> <ul class="simple"> <li>automated cross-project open source testing tool</li> <li>flexible per-project customization</li> <li>reusing test methods/reporting across projects</li> <li>various iterative test collection methods</li> <li>support for distributed testing</li> <li>py lib is a development support library</li> </ul> </div> <div class="section" id="what-is-python"> <h1>What is Python?</h1> <ul class="simple"> <li>easy-to-learn flexible OO high level language</li> <li>glue-language for connecting C++, Java and scripting</li> <li>used e.g. by Google for deployment/testing/implementation</li> <li>used by BIND (mainstream DNS internet server) for testing</li> <li>Jython provides Python for JVM</li> <li>IronPython provides Python for .NET</li> <li>CPython is mainstream C-based platform</li> <li>PyPy - Python in Python implementation</li> </ul> </div> <div class="section" id="main-drivers-of-py-test-development"> <h1>Main drivers of py.test development</h1> <ul class="simple"> <li>PyPy project testing needs (part of EU project)</li> <li>needs by individual (freely contributing) projects</li> <li>at least 20 projects using py.test and py lib</li> <li>higher level innovation plans by merlinux & experts</li> <li>commercial needs</li> <li>almost three years of (non-fulltime) development</li> </ul> </div> <div class="section" id="authors-copyrights"> <h1>Authors & copyrights</h1> <ul class="simple"> <li>initial: Holger Krekel, Armin Rigo</li> <li>major contributors: Jan Balster, Brian Dorsey, Grig Gheorghiu</li> <li>many others with small patches</li> <li>MIT license</li> </ul> </div> <div class="section" id="who-is-merlinux"> <h1>who is merlinux?</h1> <ul class="simple"> <li>small company founded in 2004 by Holger Krekel and Laura Creighton</li> <li>purpose: research and development / open source technologies</li> <li>7 employees (no win32 experts!), 6 freelancers</li> <li>three larger projects:<ul> <li>PyPy - next generation Python implementation</li> <li>mailwitness - digital invoicing/signatures</li> <li>provider of development servers</li> </ul> </li> <li>technologies: virtualization, deployment and testing</li> </ul> </div> <div class="section" id="main-features-of-py-test"> <h1>Main Features of py.test</h1> <ul class="simple"> <li>simplest possible <tt class="docutils literal"><span class="pre">assert</span></tt> approach</li> <li>clean setup/teardown semantics</li> <li>stdout/stderr capturing per test</li> <li>per-project/directory cmdline options (many predefined)</li> <li>test selection support</li> <li>customizable auto-collection of tests</li> <li><a class="reference external" href="../test.html#features">more features</a> ...</li> </ul> </div> <div class="section" id="main-user-level-entry-points"> <h1>Main User-Level entry points</h1> <ul class="simple"> <li><tt class="docutils literal"><span class="pre">py.test.raises(Exc,</span> <span class="pre">func,</span> <span class="pre">*args,</span> <span class="pre">**kwargs)</span></tt></li> <li><tt class="docutils literal"><span class="pre">py.test.fail(msg)</span></tt> -> fail a test</li> <li><tt class="docutils literal"><span class="pre">py.test.skip(msg)</span></tt> -> skip a test</li> <li><tt class="docutils literal"><span class="pre">py.test.ensuretemp(prefix)</span></tt> -> per-test session temporary directory</li> <li><tt class="docutils literal"><span class="pre">conftest.py</span></tt> can modify almost arbitrary testing aspects (but it's a bit involved)</li> </ul> </div> <div class="section" id="some-py-lib-components"> <h1>some py lib components</h1> <ul class="simple"> <li><tt class="docutils literal"><span class="pre">py.execnet</span></tt> provides ad-hoc means to distribute programs</li> <li><tt class="docutils literal"><span class="pre">py.path</span></tt> objects abstract local and svn files</li> <li><tt class="docutils literal"><span class="pre">py.log</span></tt> offers (preliminary) logging support</li> <li><tt class="docutils literal"><span class="pre">py.xml.html</span></tt> for programmatic html generation</li> <li>lazy import ...``import py`` is enough</li> </ul> </div> <div class="section" id="py-test-implementation"> <h1>py.test Implementation</h1> <ul class="simple"> <li><a class="reference external" href="../test.html">basic picture</a></li> <li>Session objects (Terminal and Tcl-GUI)</li> <li>reporting hooks are on session objects</li> <li>Collector hierarchy yield iteratively tests</li> <li>uses py lib extensively (py.path/py.execnet)</li> <li>"conftest.py" per-directory configuration mechanism</li> </ul> </div> <div class="section" id="session-objects"> <h1>Session objects</h1> <ul class="simple"> <li>responsible for driving the testing process</li> <li>make use of iterative Collector hierarchies</li> <li>responsible for reporting (XXX)</li> <li>can be split to a Frontend and BackendSession for distributed testing (GUI frontend uses it)</li> </ul> </div> <div class="section" id="collector-objects"> <h1>Collector objects</h1> <ul class="simple"> <li>Collectors / Test Items form a tree</li> <li>the tree is build iteratively (driven from Sessions)</li> <li>collector tree can be viewed with <tt class="docutils literal"><span class="pre">--collectonly</span></tt></li> <li><tt class="docutils literal"><span class="pre">run()</span></tt> returns list of (test) names or runs the test</li> <li><tt class="docutils literal"><span class="pre">join(name)</span></tt> returns a sub collector/item</li> <li>various helper methods to e.g. determine file/location</li> </ul> </div> <div class="section" id="extensions-rest-documentation-checking"> <h1>Extensions: ReST documentation checking</h1> <ul class="simple"> <li><a class="reference external" href="../conftest.py">py/documentation/conftest.py</a> provides test items for checking documentation and link integrity</li> <li>uses its own collector/testitem hierarchy</li> <li>invokes <tt class="docutils literal"><span class="pre">docutils</span></tt> processing, reports errors</li> </ul> </div> <div class="section" id="extensions-distributed-testing"> <h1>Extensions: Distributed Testing</h1> <ul class="simple"> <li>using py.execnet to dispatch on different python versions</li> <li>using py.execnet to dispatch tests on other hosts/platforms</li> <li>currently: Popen, SSH and Socket gateways</li> <li>missing support pushing tests to "the other side"</li> <li>missing for deployment on multiple machines</li> <li>but it's already possible ...</li> </ul> </div> <div class="section" id="example-using-pywinauto-from-linux"> <h1>Example using pywinauto from linux</h1> <ul class="simple"> <li>start socketserver.py on windows</li> <li>connect a SocketGateway e.g. from linux</li> <li>send tests, execute and report tracebacks through the gateway</li> <li>remotely use pywinauto to automate testing of GUI work flow</li> <li>interactive example ...</li> </ul> </div> <div class="section" id="status-of-py-lib"> <h1>Status of py lib</h1> <ul class="simple"> <li>mostly developed on linux/OSX</li> <li>basically all tests pass on win32 as well</li> <li>but missing some win32 convenience</li> <li>some support for generation of html/ReST/PDFs reports</li> <li>py.execnet works rather reliably (pending deeper win32 testing)</li> <li>flexible configuration but sometimes non-obvious/documented (requires understanding of internals)</li> </ul> </div> <div class="section" id="next-steps-py-lib-py-test"> <h1>Next Steps py lib / py.test</h1> <ul class="simple"> <li>refined py.execnet distribution of programs</li> <li>more configurable and customizable reporting</li> <li>implement support for testing distribution</li> <li>explore refined win32 support</li> <li>automated collection of unittest.py based tests</li> <li>make spawning processes/gateways more robust</li> <li>doctest support</li> <li>unify logging approaches (py.log.*)</li> <li>...</li> </ul> </div> </div> <div class="footer"> <hr class="footer" /> Holger Krekel (merlinux) • • 13th March 2006 </div> </div></body></html>