Robot Framework in
Plone CMS Project

Asko Soukka
asko.soukka@iki.fi
IT Services
University of Jyväskylä

How ”robot testing” became the synonym for acceptance testing in Plone community.

Executive summary

  • robot testing has become the norm in community
  • documentation screenshots are being generated
  • multiple robot packages are being maintained

Technical scope

  • robotframework with seleniumlibrary
  • acceptance testing
  • plain text robot format
  • BDD syntax preferred (given, when, then)

Plone CMS

  • ”enterprise Python CMS”, 2001–
  • component architecture
  • hierarchical object database
  • themed content management
  • security, content types, workflows

2011: In the beginning

  • previous javascript strategy had failed
  • only a few recorded Selenium tests existed
  • existing frontend had to be acceptance tested
  • Robot Framework introduced at Plone conference

2012: New hope

  • Python unittest integration
  • common keywords library
  • introductory blog posts with runnable examples
  • robot training at Plone conference sprints

2013: Perfect storm

  • dedicated testing tooling sprint
  • opinionated integration package
  • bundled with add-on scaffolds
  • narrative documentation
  • more robot training at other sprints
  • Sphinx-integration

Code sprints were pioneered by the Zope, who completed in excess of 30 sprints between January 2002 and January 2006.

–Wikipedia

Results

  • core has 85 robot test suites
  • documentation has 123 robot screenshots
  • collective has 221 robot test suites
  • strong adoption in private projects

Recipe of our success

  • excellent existing test framework
  • 1st class integration packages
  • bundled with scaffolding tools
  • narrative blog posts, documentation
  • training change agents at sprints
  • being nice to fellow developers

Common issues

  • timing issues in Selenium tests
  • instability of Selenium ecosystem
  • forgotting term separator in plain text format
  • quality of tests hard to control

robotsuite

  • robot test suite wrapper for unittest
  • inspired by doctest suite wrapper
  • each test is executed separately
  • plays well with zope.testrunner layers

plone.app.robotframework

  • parameterized Open browser keyword for CI
  • extensible remote library for managing test fixture directly from tests
  • test server with listener for resetting fixture between tests
  • community maintained keywords libraries

selenium2screenshots

***Test Cases***

Show feedback form
    Go to ${URL}
    Page should contain  Send feedback
    Capture and crop page screenshot
    ...    form.png
    ...    css=h1
    ...    css=form


***Test Cases***

Show locked original

    ${folder_id} =  Translate  folder_news_id
    ...  default=news
    ${item_id} =  Translate  sample_news_id
    ...  default=website-refresh
    Go to  ${PLONE_URL}/${folder_id}/${item_id}

    Element should be visible  css=#plone-lock-status
    Update element style
    ...  css=a.managePortletsFallback  display  none

    Capture and crop page screenshot
    ...  ${CURDIR}/../../_robot/working-copy_locked.png
    ...  css=#portal-column-content

sphinxcontrib-robotframework

.. figure:: form.png
.. code:: robotframework

   ***Test Cases***

   Show feedback form
       Go to ${URL}
       Page should contain  Send feedback
       Capture and crop page screenshot
       ...    form.png  css=h1 css=form

Robot Sphinx + Travis + S3

youtube

More robot by Plonistas

Questions?

Thank you!

Asko Soukka / asko.soukka@iki.fi
http://datakurre.github.com/robocon2018