Automated Software Testing with Cypress

Narayanan Palani

• Publisher: Auerbach Publications
• Year: 2021
• Print ISBN:
  9780367699543
  9780367759681
• eBook ISBN:
  9781000377231
• Edition: 1
• Topic: Informatica
• Subtopic: Informatica generale

Unit Integration Testing (UIT) had been a challenge because there was no tool that could help in XHR programming and unit integration validations in an efficient way until Cypress arrived. Cypress started releasing versions in 2015 and became popular in 2018 with version 2.0.0.

This book explores Cypress scripts that help implement 'shift left testing', which is a dream come true for many software testers. Shift left occurs in the majority of testing projects, but could not be implemented fully because tools were unavailable and knowledge was lacking about the possibilities of testing early in the life cycle. Shift left is a key testing strategy to help testing teams focus less on defect identifications and more on developing practices to prevent defects.

Cypress scripts can help front-end developers and quality engineers to work together to find defects soon after web components are built. These components can be tested immediately after they are built with Cypress Test Driven Development (TDD) scripts. Thus, defects can be fixed straight away during the development stage. Testing teams do not have to worry about finding these same defects in a later development stage because Cypress tests keep verifying components in the later stages. Defect fixing has become much cheaper with Cypress than when other tools are used. The book also covers Behaviour Driven Development (BDD)-based Gherkin scripts and the Cypress Cucumber preprocessor, which can improve test scenario coverage.

Automated Software Testing with Cypress is written to fulfil the BDD and TDD needs of testing teams. Two distinct open source repositories are provided in Github to help start running Cypress tests in no time!

Chapter 1

Visual Regression

Traditionally, organisations perform functional testing along with non-functional testing such as performance and security/vulnerability testing in order to release their code. But earlier, in application life cycle, there was no solution to the User Interface (UI) problems such as alignment changes and minor object relocation, which look bad when launching the web pages, but never be possible to notice all of them during the functional test runs. If a text box appears on the left-hand side of a page (as per the requirement) and even if it is re-aligned to the bottom right-hand side of the page, the automated tests would still pass since the object (in this case, text box) is found in this scenario. But, as per the requirement, it was difficult in the past to verify the text box exactly at a particular height and width and manual testing was the only solution to get this verification done until visual testing has been introduced.

Visual testing is nothing but the pixel by pixel comparison of the images of the test pages with the pre-saved screenshots of those pages to see how they look like when the latest code has been deployed. Since there are target pages getting saved historically, it is easy to compare them in automated code.

What if you haven't done your visual testing yet and hence, no records of images have been saved for comparison?

Do not worry—during your first run of the tests, it will save the target screenshots and you can verify them before running your next test on the re-deployed next version of the code.

Benefits of Visual Testing with Cypress

As a Software Development Engineer in Test (SDeT), I have been using visual tests within the selenium framework and finding issues to get them fixed at UI level. But, performing visual tests at right side of the test cycle is really late since the code has already been gone through unit testing, system testing and progressing towards regression testing. That's true!

A majority of the projects use selenium kind of automation tools at the right-hand side of the project to automate only when the QA has some time to automate. The major disadvantage of this approach is the expense occurred to fix every defect of the application since it has already crossed several stages of the application development and testing.

When visual testing is performed in Cypress, it provides the right solution to test visuals right after the page has been built. It means the application has not even gone through unit testing yet!

At this stage, finding a defect makes less expense compared to finding the same defect after System Integration Testing (SIT) or during regression test phase.

Since Cypress is used at application development level to make changes to UI code and rerun tests to see the user behaviour, running visual testing helps in fixing the alignment issues, colour issues and height-weight related issues during application development itself. Hence, it is an extreme shift level model of performing UI visual testing!

Basics

Usually, visual regression tests produce three types of snapshots:

• Base files: When the tests were run in the past, the first screenshot of the page or object during the successful test was stored in the base file location as the first snapshot to be used for any visual comparison. In Cypress, this file gets stored at the following path:
  \cypress\snapshots\features
• Screenshots: Every time the test runs successfully at the following path:
  \cypress\snapshots\features
• Differences: Any differences between the base file (of first successful run) and the latest screenshot evidence is kept under the following path:
  \cypress\snapshots\features\<name>.feature\__diff_output__

Note: .feature file name varies based on the type and name of the feature file used for visual testing.

Sample Visual Test Defect using Cypress

Visual test • Check visual test failed ○ TEST • LOG Given I open homepage • VISIT/ • LOG And I capture snapshot and compare “loginpage” • TASK Matching image snapshot, Object{3} • SCREENSHOT loginpage • TASK Recording snapshot result • THEN function(){}
Error Image was 0.6824884792626728% different from saved snapshot with 5924 different pixels. See diff for details: E:\Udemy\github\cypress\cypress-test-techniques\cypress\snapshots\features\loginVisualTest.feature_diff_output_\loginpage.diff.png View stack trace Error: Image was 0.6824884792626728% different from saved snapshot with 5924 different pixels. See diff for details: E:\Udemy\github\cypress\cypress-test-techniques\cypress\snapshots\features\loginVisualTest.feature\__diff_output__\loginpage.diff.png at Context. (https:/​/​opensource-demo.orangehrmlive.com/​__cypress/​tests?p=cypress\integration\features\loginVisualTest.feature-406:10316:17) From previous event: at Context.thenFn (https:/​/​opensource-demo.orangehrmlive.com/​__cypress/​runner/​cypress_runner.js:132255:23)

Let us understand on how to configure such simple but powerful visual regression tests using Cypress.io! I am using two tools for Cypress visual regression at package.json:

“cypress-visual-regression”: “1.0.4”, “cypress-image-snapshot”: “3.1.1”,

Since both these tools are compatible with Cypress 4.4.1, I have installed both of them with the commands below:

Installation from command line or Github or terminal using the commands:

npm install cypress-visual-regression -g npm install cypress-image-snapshot -g

The reason we need two different tools for visual regression is pretty simple. Cypress-image-snapshot provides us the snapshots from each tests being run as part of visual regression, and cypress-visual-regression helps in finding any mismatches between the base file and the current test's screenshot to provide differences in the folder __diff_output__ as a visual test result!

Once both these tools are installed along with Cypress, update commands javascript file at \cypress\support\commands.js with the commands below:

const compareSnapshotCommand = require('cypress-visual-regression/dist/command') import {addMatchImageSnapshotCommand} from 'cypress-image-snapshot/command' import 'cypress-file-upload'; compareSnapshotCommand() addMatchImageSnapshotCommand({ failureThreshold: 0.00, failureThresholdType: 'percent', customDiffConfig: {threshold: 0.0}, capture: 'fullPage', timeout: '60000', })

Once commands.js has been updated, next is to update index javascript file in plugins folder at the following path \cypress\plugins\index.js for the configurations of snapshot comparisons. Make sure to add both getCompareSnapshotsPlugin and addMatchImageSnapshotPlugin within modules.export along with cucumber preprocessor if you are running cucumber tests:

const getCompareSnapshotsPlugin = require('cypress-visua...

Table of contents

• Cover
• Half Title
• Title Page
• Copyright Page
• Dedication
• Contents
• About the Author
• Introduction
• 1 Visual Regression
• 2 Web Accessibility Testing
• 3 Running Cypress Tests in Docker
• 4 Test Reports
• 5 Cypress Tests in Jenkins
• 6 Behaviour Driven Development (BDD)-based Feature File Writing Techniques using Cypress.io and Cucumber
• 7 Differences Between Selenium and Cypress.io
• 8 Cypress Cucumber Preprocessor Errors
• 9 Device Compatibility Testing Using Cypress.io
• 10 Disabled Object Verification Through Force: True
• 11 Upload File Using Cypress.io
• 12 Conditional Tests: A Comparison Between Selenium and Cypress.io
• 13 Testing Dropdowns Using Cypress.io
• 14 BeforeEach and AfterEach Hooks
• 15 Generic Test Automation Architecture of Cypress.io
• 16 Cypress-based API Testing
• 17 BrowserStack-based Cypress Test Execution
• 18 Capture/Playback Approach of Test Automation
• 19 Test Applications with Slow Speed Using Cypress.io and URL Throttler
• 20 DAST with Cypress.io and BurpSuite
• 21 Click Function Using Cypress.io
• 22 Adjusting Default Timeouts or Wait Times in Cypress.json
• 23 Double Click Function Using Cypress.io
• 24 Cypress Retry on Failed Tests
• 25 Mouse Events Using Cypress.io
• 26 Checkboxes Verification Using Cypress.io
• Index