In the era of testing, web automation plays an important role. There are ample tools like selenium, webdriverIO, protractor, etc available in the market to do automation but what if one wants to extract data from the same web application which is termed as web scraping so here Google Puppeteer comes into the picture to automate web application as well as scraping.
What is a Puppeteer?
Puppeteer is a Node library that provides a high-level API to control Chrome or chromium driver over the DevTools protocol.
By default, Puppeteer runs in headless mode but with configuration, we can run this in full chrome or chromium-browser(non-headless). It was made for chrome browser by the Chrome DevTools team to help automated or headless browser testing. It also provides the facility to perform automation over cross-browser and now it supports Firefox browser too.
Headless browsers are the browsers that run as an app process without any user interface elements. Google has been working to ensure that the DevTools community is supported with sufficient support.
Automation Made Easy:
When it comes to automation testing, Selenium leads the industry. The testing community with open arms has accepted this wonderful, open-sourced tool. But Selenium testing requires installed browsers, regardless of whether Selenium is installed on the cloud or on a local machine, and this is a big one. Add the complexity of integrations with a variety of components, and it becomes too hard to run smoothly for the process.
Headless Chrome has become the go-to guy when it comes to using automated scripts into the browser. the use of Karma, Mocha, and Puppeteer along with it made it easier to test using automated scripts.
Visual UI Regression Testing:
It is possible to use Puppeteer for regression testing of different UI elements. It is advisable to have the latest chromium version to ensure better API control in order to test the website’s UI. In order to have regressive testing on the different UI characteristics, Mocha is typically used along with Puppeteer. Furthermore, Mocha offers you hooks that can be used before or after testing.
In particular, Puppeteer is successful because it simplifies the features associated with automated tasks performed on browsers.
In the puppeteer itself, Puppeteer-screenshot-tester is a library that allows the generation of screenshots while testing.
Similarly, ava-puppeteer is another library that will soon be available on npm, which is available on GitHub.
One of the least popular forms of testing is often considered end-to-end or acceptance testing. Selenium was used for acceptance testing earlier, and it is quite vulnerable to error and has a lot of bugs. Then came Puppeteer with an improved focus on the control of the API that helped disentangle the bugs that came in the form of complications. One of Puppeteer’s superpowers that don’t need to know your stack and can work with any structure, whether it’s React, Angular, or any other library.
Prerequisites for the Puppeteer.
- NPM (It will get installed automatically with installing node.js)
- VS Code.
Steps for installing Puppeteer.
- Create a folder with the name “PuppeteerProj” in any drive (can create a folder with any name according to you)
- Open VS Code and select the PuppeteerProj that you created recently.
- Now trigger the below command on terminal
npm init -y
- After completion of the above command. it will create a default package.json file in the ‘webdriverioProject’ folder.
- Now install puppeteer. Trigger below command
npm install puppeteer
- This Command will create a node_modules directory automatically in the ‘PuppeteerProj’ Folder.
How to run tests with Puppeteer:
- Now Make a directory under ‘PuppeteerProj’ with name specs/ tests and store all test files that you are going to create( In my case I have created sampleTest.js in the tests directory ).
This test will run in non-headless mode and by default, it uses chromium-browser to execute tests. So here in this test, we are going to launch a browser -> open google and search for some contents.
Now let’s take a look at a test that runs headlessly and brings some data from web applications which is termed as web scraping.
- To run your tests, open the terminal, and trigger the below command.