Automated Markup Testing with Galen Framework

Overview of automated testing of the HTML + CSS code with a relatively new tool for adaptive design and cross-browser testing, Galen Framework, presented by Logicify QA specialist, Victoria Remnyova. The article presents general notions on the framework, its syntax, examples of a Galen-generated report and description of a few handy features quality assurance specialist could use.

Definition of Galen Framework

Cascading Style Sheets, commonly referred to as CSS, describes the style of an HTML document and the way HTML elements should be displayed on a web-page. Quality Assurance specialists often struggle when testing CSS as the layout testing is routine. Besides, it requires much manual work since the browser rendering engines apply CSS with subtle differences. When added up, these differences can make the result look off. Sometimes, it is impossible to perform regression testing for every kind of devices and browsers and make sure the page layout remained stable.

Galen Framework is a Java-written open source framework for layout and functional testing, which helps to address the mentioned issues. It is a special tool for website testing and UI automation that aids in testing of automated layout and responsive design, cross-browser testing, functional testing and in reporting test results with screenshots. Though the framework is relatively new, it has already been used by subsidiaries of such large companies as eBay and SkyScanner. It has also been used by the company EastBanc Technologies to test the web-site of The Washington Post.

At Logicify, we also use Galen Framework. When applied properly, it lets our teams perform better, saving clients' money on quality assurance.

Complex Layout Testing with Galen

Galen Framework has its own language, Galen Specs, applied for describing the positioning and alignment of elements on a web-page. It is based on Selenium WebDriver and could be executed via Selenium Grid for cross-browser testing to automate web application testing and verify that it works as expected. To initiate a Galen test, one needs to prepare a spec file, with the extension .gspec. It should contain both the objects definitions and their specifications. The page elements are identified using such standard locators as id, CSS and XPATH:

@objects    
    logo                     id              brand-logo
    description        css            .survey-wizard-body-description
    next-button       xpath         //*[@class='item']/div[text(),'Next']

Afterwards, one should specify the rules for each page elements using tags (for example, “desktop,” “tablet,” “mobile”):

@on *
    logo:
        centered horizontally inside viewport

    description:                
        below logo
        centered horizontally inside viewport

@on desktop,tablet
    logo:
        width 380 px

@on mobile
    logo:
        width 200 px

Galen page example
One could use different rules depending on the size and resolution of the screen, for instance, a page logo could differ in size for screens with various resolutions.

There is a variety of commands used for page elements in a spec, for instance, to:

  • define ranges:

    # Exact range
        width 100 px
    
    #The value between range
        width 50 to 200 px
    
    #Greater than
        width > 40 px
    
    #Approximate
        width ~ 100 px
    
    #Relative ranges
        width 50 % of screen/width
    
  • specify location towards other elements:

    textfield:
        near button 10px left
    

    Galen Screenshot

    caption:
        above description 10 to 20 px
    
    description:
        below caption 10 to 20 px
    

    Galen Screenshot

  • align or center the objects:

    item-1:
        aligned horizontally top item-2
    

    Galen Screenshot

    button:
        centered horizontally inside box
    

    Galen Screenshot

Galen supports other operations, such as:

  • text verificaition:

    text lowercase is "welcome user@example.com to our website!" 
    text uppercase starts "WELCOME"
    
  • and CSS properties:

    login-button:
        css font-size is "18px" 
        css font-family starts "Helvetica"
    

Galen Framework also allows the user to search for multiple objects with an identical locator and number them sequentially, from 1 to n:

@objects
    menuItem-*     css     #menu li a

This function allows to further test all of these objects in loop:

@forEach [menuItem-*] as itemName
    ${itemName}:
        height 30 px

Galen Framework also supports importing of specs and some custom Javascript functions for more complex testing. Below is an example of a Javascript function for internalization:

this.i18n = function (name) {
    // ...
    // define a code for handling i18n
};

It could further be inserted into a .gspec:

@script i18n.function.js

greeting-text:
    text is "${i18n('header.greeting.text')}"

Running the Test with Galen Framework

Once the spec file with all objects and corresponding rules was written, it could be run either in Command Line or Galen Test Suite. To do this, one should specify the spec file name, page URL, screen size and tag (desktop, tablet, mobile) as well as the path to the folder containing the report. Below is an example of the test run in a Command Line:

galen check introduction.gspec
    --url "http://run.example.com/LkqxNd"
    --size "320x600"
    --include "mobile"
    --htmlreport "reports/introduction"

Galen Test Suite is a file with an extension .test that contains the same information with instruction for spec running:

Introduction page in chrome browser for tablet device
    selenium chrome http://run.example.com/LkqxNd 1024x768
        check introduction.gspec --include “tablet”

The same command could be extended and parameterized depending on the browser types, screen sizes and other parameters that matter for testing. For instance:

@@ set 
domain http://run.example.com/LkqxNd 1024x768

@@ parameterized
| browser |
| firefox    |
| chrome  |
| ie             |

@@ parameterized
| deviceName   | tags              | size             |
| desktop           | desktop       |1600x1000 |
| tablet               | tablet           | 1024x768  |
| mobile             | mobile         |  375x667   |

Introduction page in ${browser} browser for ${deviceName} device
    selenium ${browser} ${domain} ${size}
        check introduction.gspec --include ${tags}

where @@ set is a special annotation that is used to put commonly used data in variables, while @@ parameterized is a special annotation that lists different conditions for the test.

Galen Test Reports

The testing report is available after the spec was executed, and it presents the following information: the test name, the total number of tests, number of successful and failed ones, warnings, the time when the test was initiated and its duration.

Galen Report Example

By clicking on a corresponding test name, one could receive a more detailed data on it, including all the elements that went through testing and corresponding rules for them - whether successfully tested or failed.

Galen Wait Actions

Sometimes QA specialists have to test some elements that are rendered dynamically after the page load. Galen Framework (version 0.10.0 onwards) supports testing of dynamic content with its wait actions. Specifically, the framework allows to test these elements as they appear either after a hardcoded amount of time or due to a specific condition:

  • Strict timeout:

    Home page test
    http://example.com 640x480
    wait 10s
    check homepage.gspec
    
  • Condition-based timeout:

    Home page test
    http://example.com 640x480
    wait 1m until exist "css: div.list a"
    check homepage.gspec
    

In case the defined condition fails, Galen throws up an exception and interrupts the test.

Creating Page Dump

Another useful feature of Galen framework (version 1.3 onwards) is creating a page dump. It allows to store information about all test objects on the page together with image samples. The page dump creates an HTML report with page screenshot and fetches all page objects with detailed information about them. Also it stores each image of page object separately in png format. These images could be used for image comparison – a new technique in Galen Framework that allows to compare area of the selected object with predefined image sample. As a result, a QA specialist can get a report with a detailed description of each element and a page screenshot as well as spec suggestions for them. Galen framework automatically suggests rules for these elements, and QA Specialists could use them in spec file. This approach saves a significant amount of test creation time.

Additional Features of Galen

Some other helpful features Galen Framework offers include image comparison, running test in Selenium Grid, integration with such cloud-based services as BrowserStack and SauceLabs used to test web-applications in different browsers, Java and JavaScript API, as well as GalenPages framework for working with Page Objects pattern. The framework also allows testing on real mobile devices with Appium application.

Summary

Testing responsive web design across different browsers and various devices is a huge challenge for QA specialists, and also not the most pleasant one given lots of repetitive work. Galen Framework is one of the many tools that help in this. Though the framework does have a few disadvantages (for instance, it does not allow even simplest arithmetic operations in spec files; it only tests what is described in the spec, so it is left on a QA’s conscience whether the test specs are comprehensive and extensive enough; and it is not that much convenient for complex sites), combining the framework with other tools while testing proved to be very effective and time-saving. For instance, combining Selenium and Galen framework techniques offers better results in more maintainable test suites.

Related articles

Tags