[Unicode]   Common Locale Data Repository : Bug Tracking Home | Site Map | Search

CLDR Ticket #11488(accepted)

Opened 3 months ago

Last modified 5 weeks ago

Implement new Survey Tool automated test framework and infrastructure

Reported by: tbishop Owned by: tbishop
Component: surveytool-other Data Locale:
Phase: dsub Review:
Weeks: Data Xpath:



Survey Tool needs automated integration testing covering the front end as well as the back end, to simulate realistic usage, especially under heavy load by multiple simultaneous voters.

Purposes include: (1) integration testing, to discover existing bugs and avoid creation of new bugs or regressions when refactoring or adding new features; (2) performance testing, to discover hot spots where performance improvements can be most helpful to voters.

There are already some back-end unit tests. Currently we lack automated front-end testing. We need both integration tests and front-end unit tests, and while the latter are not the subject of this ticket, they are related, since integration tests will provide some support for refactoring which is needed for implementing front-end unit tests. Some of the current front-end code isn't modular enough to support unit tests without extensive refactoring.

WebDriver is the main standard for web application test automation to simulate user interaction using the actual browsers supported by Survey Tool: Chrome, Edge, Firefox, and Safari. We plan to use Selenium, a popular implementation of WebDriver, as a key component of the new test framework.

We already have some WebDriver test code written while working on tickets 10990, 11211, 11299, and 11238. At the time of creating this ticket that code isn't in a public version control system yet. An important part of this ticket is to set up a version control repository for this new code, which should be distinct from the main repository for Survey Tool, which is currently in cldr-apps, in the svn repository for cldr. It should be a distinct repository, since the test framework is an optional addition to cldr-apps. Also, the WebDriver code is capable of running on separate machines from Survey Tool itself, and doesn't require linking with the existing Survey Tool code, other than "linking" in the sense of being a Survey Tool client. The infrastructure described by this ticket includes the repository and documentation as well as the code itself, and may also eventually include setting up a permanent grid, with nodes to run clients for continually testing a server such as cldr-smoke.unicode.org, and profiling tools.

Performance testing should employ profiling tools such as JMeter, which can be used both in conjunction with WebDriver and separately. WebDriver should enable clarification of hot spots that are important to typical realistic usage. The most accurate measurement of performance of the back end should probably include tests that do not depend on WebDriver, a browser, or the front end at all.


Change History

comment:1 Changed 3 months ago by mark

  • Owner changed from anybody to tbishop
  • Priority changed from major to critical
  • Status changed from new to accepted
  • Milestone changed from UNSCH to 35

comment:2 Changed 3 months ago by tbishop

I made a GitHub repository for the WebDriver code:


comment:3 Changed 2 months ago by tbishop

There's a new ticket: 11571 Avoid rebuilding entire table on Survey Tool update. Fixing that will pave the way for making the WebDriver code more effective. As it is, we're plagued by StaleElementReferenceException when a cell disappears just as we're simulating a click in that cell.

comment:4 Changed 8 weeks ago by mark

  • Milestone changed from 35 to 35-optional

comment:5 Changed 7 weeks ago by tbishop

Added test used in ticket 11270. It includes a long (comprehensive?) list of locales.

comment:6 Changed 5 weeks ago by srl

  • Xref set to 5992
  • See ticket:5992 for restructuring that would support automated testing

Add a comment

Modify Ticket

as accepted

E-mail address and user name can be saved in the Preferences.

Note: See TracTickets for help on using tickets.