Performance Comparison between Selenium-3 and Selenium-4
SUBSCRIBE TO OUR BLOG
Get our new blogs delivered straight to your inbox.
THANKS FOR SIGNING UP!
We’ll make sure to share the best materials crafted for you!
5 minutes read
Performance Comparison between Selenium-3 and Selenium-4
Written by Emin Bilgic
Software Engineer at Thundra
Summary
Selenium-4 was officially released On October 13, 2021, and brought lots of upgrades with this release. The most prominent change in this release is the deprecation of JSON wire Protocol. It changed Selenium architecture in a significant way. Besides that, there are also lots of standardizations and stabilizations about the Selenium domain. In this article, we are going to compare coding practices between Selenium-3 and Selenium-4, and we will make a performance comparison between them.
Architectural upgrades on Selenium-4
The most prominent change in this release is the deprecation of JSON Wire Protocol. Actually, this can be considered as the main reason for the Selenium-4 upgrade and Selenium-4 does not include JSON Wire Protocol anymore.
Selenium used to use this protocol for communicating with the web browser. But Selenium-4 has direct communication between the client and server using the W3C (World Wide Web Consortium) protocol, offering lots of advantages compared to the Web Driver used in the versions before Selenium-4.
Architecture of Selenium-3
Native support has been removed for the Opera Driver, and the Opera Driver and browser are no longer active. Users can use Chrome as the Opera browser which is based on Chromium.
Architecture of Selenium-4
After W3C Protocol is replaced with JSON Wire Protocol, you do not need to encode and decode the requests. This direct communication brings more stability and consistency.
The Deprecates from Selenium-3
Find Element methods:
The utility methods to find element(s) in the ‘FindBy’ interface have been removed as they have been used internally only. Instead of this, the ‘By’ class is annotated to increase stability.
Old:
driver.findElementByClassName("className");New:
driver.findElement(By.className("className"));Desired Capabilities:
Desired capabilities were used to define test environments primarily in Selenium Grid. These are replaced with Options in Selenium-4, so you are able to define specific options for drivers.
Old:
DesiredCapabilities caps = DesiredCapabilities.firefox();
caps.setCapability("platform", "Windows 10");
caps.setCapability("version", "92");
caps.setCapability("build", myTestBuild);
caps.setCapability("name", myTestName);
WebDriver driver = new RemoteWebDriver(new URL(cloudUrl), caps);New:
FirefoxOptions browserOptions = new FirefoxOptions();
browserOptions.setPlatformName("Windows 10");
browserOptions.setBrowserVersion("92");
Map<String, Object> cloudOptions = new HashMap<>();
cloudOptions.put("build", myTestBuild);
cloudOptions.put("name", myTestName);
browserOptions.setCapability("cloud:options", cloudOptions);
WebDriver driver = new RemoteWebDriver(new URL(cloudUrl), browserOptions);Actions:
Actions class is for simulating user gestures like a double click, click and hold, and etc. There are newly added methods in this class and they are replaced with the old ones.
Old:
moveToElement(toElement).click()>New:
click(webElement)These are the significant changes from Selenium 3 and there are lots of deprecated methods that came with Selenium-4. Some of the classes and methods that include them are:
- Timeout Parameters
- Merging Capabilities
- Wait parameters
- Browser Interface
- Firefox Legacy
- Browser type
What is new in Selenium-4
There comes lots of remarkable innovations that will make testing easier for you with Selenium-4. For instance, getting screenshots of elements, opening a new tab or window used to be painful. With Selenium-4, these operations are standardized with selenium libraries.
Screenshot of an Element
This was a very painful event in older Selenium versions. You had to take a screenshot of the screen and then you had to crop it using desired element’s coordinates. With Selenium-4, it can be accomplished with a few lines of code:
Old
WebDriver driver = new ChromeDriver();
driver.get("https://www.google.com/");
WebElement logoElement = driver.findElement(By.className("lnXdpd"));
File screenshot = ((TakesScreenshot) driver).getScreenshotAs(OutputType.FILE);
BufferedImage fullImg = ImageIO.read(screenshot);
Point point = logoElement.getLocation();
int eleWidth = logoElement.getSize().getWidth();
int eleHeight = logoElement.getSize().getHeight();
BufferedImage eleScreenshot = fullImg.getSubimage(point.getX(), point.getY(),
eleWidth, eleHeight);
ImageIO.write(eleScreenshot, "png", screenshot);
File screenshotLocation = new File("outputs/elementShotFromScreenShot.png");
FileUtils.copyFile(screenshot, screenshotLocation);
driver.quit();
New
WebDriver driver = new ChromeDriver();
driver.get("https://www.google.com/");
WebElement logoElement = driver.findElement(By.className("lnXdpd"));
File myObj = logoElement.getScreenshotAs(OutputType.FILE);
File DestFile = new File("outputs/elementShotTest.png");
FileUtils.copyFile(myObj, DestFile);
driver.quit();
This brings us a significant performance improvement on taking screenshot because there was lots of operations like crop and save in the old version.
In my example; I ran a test method 3 times which uses the old version of taking screenshots of an element. Then I updated this function with using the new technique and ran it 3 times again.
You can notice a significant reduction in the execution times of the tests in the latter 3 runs than the first 3 runs.
Reducing the execution times of taking a photo of an element in migrating from Selenium-3 to Selenium-4
Multiple Tabs
When you wanted to open a new tab or a new window for your tests, you had to send which shortcut key sequences should be pressed on your screen or you had to run some javascript commands with javascript executed.
With Selenium-4, you can simply tell the driver to switch new tab now.
Old:
String keys = Keys.chord(Keys.CONTROL,"t");
driver.findElement(By.cssSelector("body")).sendKeys(keys);
//or
JavascriptExecutor jse = (JavascriptExecutor)driver;
jse.executeScript("window.open()");
New:
driver.switchTo().newWindow(WindowType.TAB);
Relative Locators
Locating elements might be painful even if you have a wide acquaintance with UI testing. With relative locators, you don’t need to reach each element with the ‘findElement’ methods. You can locate an element from any element you already located.
WebDriver driver = new ChromeDriver();
driver.get("https://www.google.com/");
WebElement searchBar = driver.findElement(By.name("q"));
searchBar.sendKeys("Relative Locators");
WebElement searchButton = driver.findElement(with(By.tagName("input")).below(searchBar));
searchButton.click();
driver.quit();
This code block finds the search bar in google and sends ‘Relative Locator’ keys to it. Then it checks if there are any elements with an ‘input’ tag below in the located search bar.
Connection Conditions
You can set some connection features for your tests with Selenium-4 like latency, going offline, and setting download or upload throughput.
‘new ChromiumNetworkConditions().setOffline(true)’Full page screenshot on firefox
Taking a full-page screenshot in Selenium-4 doesn’t mean the same thing as taking a screenshot in an older version of Selenium. In older versions, you can only take a screenshot of the current view, not the whole page. With Selenium-4 you can take a screenshot of the infinite scrolled page.
WebDriver augmentedDriver = new Augmenter().augment(driver);
File file = ((HasFullPageScreenshot) augmentedDriver).getFullPageScreenshotAs(OutputType.FILE);Performance comparison
Since W3C was replaced with JSON Wire Protocol, our tests can directly communicate with the web browser. It may sound like Selenium is more efficient and fast now, but it actually does not make much difference in the speed of the tests. As the Selenium web driver and web browsers use the same protocol, tests will execute only more consistently. So flakiness of Selenium tests will be reduced substantially.
We already talked about stability and consistency improvements in Selenium-4. Now we are going to compare if there is any performance improvement in this release.
To do this, I have created a repository that contains 2 projects which are use Selenium-4 and Selenium-3 jars from maven and they contain the same test method with annotation @RepeatedTest(15).
@RepeatedTest(15)
public void testAQuery() throws InterruptedException {
WebElement searchBar = driver.findElement(By.name("q"));
searchBar.sendKeys("Selenium");
WebElement searchButton = driver.findElement(By.name("btnK"));
Thread.sleep(500);
searchButton.click();
List<WebElement> results = driver.findElements(By.tagName("h3")) ;
results.get(0).click();
}
I created 2 different GitHub workflows for running them separately and scheduled them to run 6 times a day.
Now it is time to monitor these tests.
To be able to monitor my workflows, I connected my repository to Thundra Foresight while my workflows generate data.
After nearly a week I got this graph from Thundra which means there is no significant performance difference between Selenium-3 and Selenium-4.
Workflow run times of Selenium-3 and Selenium-4
In my performance comparison test, I’ve only used 15 tests that process a simple google search on a single driver. This may not be a very healthy comparison but we can tell that our tests are more consistent and stable with Selenium-4.
SHARE
×
SUBSCRIBE TO OUR BLOG
Get our new blogs delivered straight to your inbox.
THANKS FOR SIGNING UP!
We’ll make sure to share the best materials crafted for you!
5 minutes read
Performance Comparison between Selenium-3 and Selenium-4
Written by Emin Bilgic
Software Engineer at Thundra
Summary
Selenium-4 was officially released On October 13, 2021, and brought lots of upgrades with this release. The most prominent change in this release is the deprecation of JSON wire Protocol. It changed Selenium architecture in a significant way. Besides that, there are also lots of standardizations and stabilizations about the Selenium domain. In this article, we are going to compare coding practices between Selenium-3 and Selenium-4, and we will make a performance comparison between them.
Architectural upgrades on Selenium-4
The most prominent change in this release is the deprecation of JSON Wire Protocol. Actually, this can be considered as the main reason for the Selenium-4 upgrade and Selenium-4 does not include JSON Wire Protocol anymore.
Selenium used to use this protocol for communicating with the web browser. But Selenium-4 has direct communication between the client and server using the W3C (World Wide Web Consortium) protocol, offering lots of advantages compared to the Web Driver used in the versions before Selenium-4.
Architecture of Selenium-3
Native support has been removed for the Opera Driver, and the Opera Driver and browser are no longer active. Users can use Chrome as the Opera browser which is based on Chromium.
Architecture of Selenium-4
After W3C Protocol is replaced with JSON Wire Protocol, you do not need to encode and decode the requests. This direct communication brings more stability and consistency.
The Deprecates from Selenium-3
Find Element methods:
The utility methods to find element(s) in the ‘FindBy’ interface have been removed as they have been used internally only. Instead of this, the ‘By’ class is annotated to increase stability.
Old:
driver.findElementByClassName("className");New:
driver.findElement(By.className("className"));Desired Capabilities:
Desired capabilities were used to define test environments primarily in Selenium Grid. These are replaced with Options in Selenium-4, so you are able to define specific options for drivers.
Old:
DesiredCapabilities caps = DesiredCapabilities.firefox();
caps.setCapability("platform", "Windows 10");
caps.setCapability("version", "92");
caps.setCapability("build", myTestBuild);
caps.setCapability("name", myTestName);
WebDriver driver = new RemoteWebDriver(new URL(cloudUrl), caps);New:
FirefoxOptions browserOptions = new FirefoxOptions();
browserOptions.setPlatformName("Windows 10");
browserOptions.setBrowserVersion("92");
Map<String, Object> cloudOptions = new HashMap<>();
cloudOptions.put("build", myTestBuild);
cloudOptions.put("name", myTestName);
browserOptions.setCapability("cloud:options", cloudOptions);
WebDriver driver = new RemoteWebDriver(new URL(cloudUrl), browserOptions);Actions:
Actions class is for simulating user gestures like a double click, click and hold, and etc. There are newly added methods in this class and they are replaced with the old ones.
Old:
moveToElement(toElement).click()>New:
click(webElement)These are the significant changes from Selenium 3 and there are lots of deprecated methods that came with Selenium-4. Some of the classes and methods that include them are:
- Timeout Parameters
- Merging Capabilities
- Wait parameters
- Browser Interface
- Firefox Legacy
- Browser type
What is new in Selenium-4
There comes lots of remarkable innovations that will make testing easier for you with Selenium-4. For instance, getting screenshots of elements, opening a new tab or window used to be painful. With Selenium-4, these operations are standardized with selenium libraries.
Screenshot of an Element
This was a very painful event in older Selenium versions. You had to take a screenshot of the screen and then you had to crop it using desired element’s coordinates. With Selenium-4, it can be accomplished with a few lines of code:
Old
WebDriver driver = new ChromeDriver();
driver.get("https://www.google.com/");
WebElement logoElement = driver.findElement(By.className("lnXdpd"));
File screenshot = ((TakesScreenshot) driver).getScreenshotAs(OutputType.FILE);
BufferedImage fullImg = ImageIO.read(screenshot);
Point point = logoElement.getLocation();
int eleWidth = logoElement.getSize().getWidth();
int eleHeight = logoElement.getSize().getHeight();
BufferedImage eleScreenshot = fullImg.getSubimage(point.getX(), point.getY(),
eleWidth, eleHeight);
ImageIO.write(eleScreenshot, "png", screenshot);
File screenshotLocation = new File("outputs/elementShotFromScreenShot.png");
FileUtils.copyFile(screenshot, screenshotLocation);
driver.quit();
New
WebDriver driver = new ChromeDriver();
driver.get("https://www.google.com/");
WebElement logoElement = driver.findElement(By.className("lnXdpd"));
File myObj = logoElement.getScreenshotAs(OutputType.FILE);
File DestFile = new File("outputs/elementShotTest.png");
FileUtils.copyFile(myObj, DestFile);
driver.quit();
This brings us a significant performance improvement on taking screenshot because there was lots of operations like crop and save in the old version.
In my example; I ran a test method 3 times which uses the old version of taking screenshots of an element. Then I updated this function with using the new technique and ran it 3 times again.
You can notice a significant reduction in the execution times of the tests in the latter 3 runs than the first 3 runs.
Reducing the execution times of taking a photo of an element in migrating from Selenium-3 to Selenium-4
Multiple Tabs
When you wanted to open a new tab or a new window for your tests, you had to send which shortcut key sequences should be pressed on your screen or you had to run some javascript commands with javascript executed.
With Selenium-4, you can simply tell the driver to switch new tab now.
Old:
String keys = Keys.chord(Keys.CONTROL,"t");
driver.findElement(By.cssSelector("body")).sendKeys(keys);
//or
JavascriptExecutor jse = (JavascriptExecutor)driver;
jse.executeScript("window.open()");
New:
driver.switchTo().newWindow(WindowType.TAB);
Relative Locators
Locating elements might be painful even if you have a wide acquaintance with UI testing. With relative locators, you don’t need to reach each element with the ‘findElement’ methods. You can locate an element from any element you already located.
WebDriver driver = new ChromeDriver();
driver.get("https://www.google.com/");
WebElement searchBar = driver.findElement(By.name("q"));
searchBar.sendKeys("Relative Locators");
WebElement searchButton = driver.findElement(with(By.tagName("input")).below(searchBar));
searchButton.click();
driver.quit();
This code block finds the search bar in google and sends ‘Relative Locator’ keys to it. Then it checks if there are any elements with an ‘input’ tag below in the located search bar.
Connection Conditions
You can set some connection features for your tests with Selenium-4 like latency, going offline, and setting download or upload throughput.
‘new ChromiumNetworkConditions().setOffline(true)’Full page screenshot on firefox
Taking a full-page screenshot in Selenium-4 doesn’t mean the same thing as taking a screenshot in an older version of Selenium. In older versions, you can only take a screenshot of the current view, not the whole page. With Selenium-4 you can take a screenshot of the infinite scrolled page.
WebDriver augmentedDriver = new Augmenter().augment(driver);
File file = ((HasFullPageScreenshot) augmentedDriver).getFullPageScreenshotAs(OutputType.FILE);Performance comparison
Since W3C was replaced with JSON Wire Protocol, our tests can directly communicate with the web browser. It may sound like Selenium is more efficient and fast now, but it actually does not make much difference in the speed of the tests. As the Selenium web driver and web browsers use the same protocol, tests will execute only more consistently. So flakiness of Selenium tests will be reduced substantially.
We already talked about stability and consistency improvements in Selenium-4. Now we are going to compare if there is any performance improvement in this release.
To do this, I have created a repository that contains 2 projects which are use Selenium-4 and Selenium-3 jars from maven and they contain the same test method with annotation @RepeatedTest(15).
@RepeatedTest(15)
public void testAQuery() throws InterruptedException {
WebElement searchBar = driver.findElement(By.name("q"));
searchBar.sendKeys("Selenium");
WebElement searchButton = driver.findElement(By.name("btnK"));
Thread.sleep(500);
searchButton.click();
List<WebElement> results = driver.findElements(By.tagName("h3")) ;
results.get(0).click();
}
I created 2 different GitHub workflows for running them separately and scheduled them to run 6 times a day.
Now it is time to monitor these tests.
To be able to monitor my workflows, I connected my repository to Thundra Foresight while my workflows generate data.
After nearly a week I got this graph from Thundra which means there is no significant performance difference between Selenium-3 and Selenium-4.
Workflow run times of Selenium-3 and Selenium-4
In my performance comparison test, I’ve only used 15 tests that process a simple google search on a single driver. This may not be a very healthy comparison but we can tell that our tests are more consistent and stable with Selenium-4.
SHARE
Summary
Selenium-4 was officially released On October 13, 2021, and brought lots of upgrades with this release. The most prominent change in this release is the deprecation of JSON wire Protocol. It changed Selenium architecture in a significant way. Besides that, there are also lots of standardizations and stabilizations about the Selenium domain. In this article, we are going to compare coding practices between Selenium-3 and Selenium-4, and we will make a performance comparison between them.
Architectural upgrades on Selenium-4
The most prominent change in this release is the deprecation of JSON Wire Protocol. Actually, this can be considered as the main reason for the Selenium-4 upgrade and Selenium-4 does not include JSON Wire Protocol anymore.
Selenium used to use this protocol for communicating with the web browser. But Selenium-4 has direct communication between the client and server using the W3C (World Wide Web Consortium) protocol, offering lots of advantages compared to the Web Driver used in the versions before Selenium-4.
Architecture of Selenium-3
Native support has been removed for the Opera Driver, and the Opera Driver and browser are no longer active. Users can use Chrome as the Opera browser which is based on Chromium.
Architecture of Selenium-4
After W3C Protocol is replaced with JSON Wire Protocol, you do not need to encode and decode the requests. This direct communication brings more stability and consistency.
The Deprecates from Selenium-3
Find Element methods:
The utility methods to find element(s) in the ‘FindBy’ interface have been removed as they have been used internally only. Instead of this, the ‘By’ class is annotated to increase stability.
Old:
driver.findElementByClassName("className");New:
driver.findElement(By.className("className"));Desired Capabilities:
Desired capabilities were used to define test environments primarily in Selenium Grid. These are replaced with Options in Selenium-4, so you are able to define specific options for drivers.
Old:
DesiredCapabilities caps = DesiredCapabilities.firefox();
caps.setCapability("platform", "Windows 10");
caps.setCapability("version", "92");
caps.setCapability("build", myTestBuild);
caps.setCapability("name", myTestName);
WebDriver driver = new RemoteWebDriver(new URL(cloudUrl), caps);New:
FirefoxOptions browserOptions = new FirefoxOptions();
browserOptions.setPlatformName("Windows 10");
browserOptions.setBrowserVersion("92");
Map<String, Object> cloudOptions = new HashMap<>();
cloudOptions.put("build", myTestBuild);
cloudOptions.put("name", myTestName);
browserOptions.setCapability("cloud:options", cloudOptions);
WebDriver driver = new RemoteWebDriver(new URL(cloudUrl), browserOptions);Actions:
Actions class is for simulating user gestures like a double click, click and hold, and etc. There are newly added methods in this class and they are replaced with the old ones.
Old:
moveToElement(toElement).click()>New:
click(webElement)These are the significant changes from Selenium 3 and there are lots of deprecated methods that came with Selenium-4. Some of the classes and methods that include them are:
- Timeout Parameters
- Merging Capabilities
- Wait parameters
- Browser Interface
- Firefox Legacy
- Browser type
What is new in Selenium-4
There comes lots of remarkable innovations that will make testing easier for you with Selenium-4. For instance, getting screenshots of elements, opening a new tab or window used to be painful. With Selenium-4, these operations are standardized with selenium libraries.
Screenshot of an Element
This was a very painful event in older Selenium versions. You had to take a screenshot of the screen and then you had to crop it using desired element’s coordinates. With Selenium-4, it can be accomplished with a few lines of code:
Old
WebDriver driver = new ChromeDriver();
driver.get("https://www.google.com/");
WebElement logoElement = driver.findElement(By.className("lnXdpd"));
File screenshot = ((TakesScreenshot) driver).getScreenshotAs(OutputType.FILE);
BufferedImage fullImg = ImageIO.read(screenshot);
Point point = logoElement.getLocation();
int eleWidth = logoElement.getSize().getWidth();
int eleHeight = logoElement.getSize().getHeight();
BufferedImage eleScreenshot = fullImg.getSubimage(point.getX(), point.getY(),
eleWidth, eleHeight);
ImageIO.write(eleScreenshot, "png", screenshot);
File screenshotLocation = new File("outputs/elementShotFromScreenShot.png");
FileUtils.copyFile(screenshot, screenshotLocation);
driver.quit();
New
WebDriver driver = new ChromeDriver();
driver.get("https://www.google.com/");
WebElement logoElement = driver.findElement(By.className("lnXdpd"));
File myObj = logoElement.getScreenshotAs(OutputType.FILE);
File DestFile = new File("outputs/elementShotTest.png");
FileUtils.copyFile(myObj, DestFile);
driver.quit();
This brings us a significant performance improvement on taking screenshot because there was lots of operations like crop and save in the old version.
In my example; I ran a test method 3 times which uses the old version of taking screenshots of an element. Then I updated this function with using the new technique and ran it 3 times again.
You can notice a significant reduction in the execution times of the tests in the latter 3 runs than the first 3 runs.
Reducing the execution times of taking a photo of an element in migrating from Selenium-3 to Selenium-4
Multiple Tabs
When you wanted to open a new tab or a new window for your tests, you had to send which shortcut key sequences should be pressed on your screen or you had to run some javascript commands with javascript executed.
With Selenium-4, you can simply tell the driver to switch new tab now.
Old:
String keys = Keys.chord(Keys.CONTROL,"t");
driver.findElement(By.cssSelector("body")).sendKeys(keys);
//or
JavascriptExecutor jse = (JavascriptExecutor)driver;
jse.executeScript("window.open()");
New:
driver.switchTo().newWindow(WindowType.TAB);
Relative Locators
Locating elements might be painful even if you have a wide acquaintance with UI testing. With relative locators, you don’t need to reach each element with the ‘findElement’ methods. You can locate an element from any element you already located.
WebDriver driver = new ChromeDriver();
driver.get("https://www.google.com/");
WebElement searchBar = driver.findElement(By.name("q"));
searchBar.sendKeys("Relative Locators");
WebElement searchButton = driver.findElement(with(By.tagName("input")).below(searchBar));
searchButton.click();
driver.quit();
This code block finds the search bar in google and sends ‘Relative Locator’ keys to it. Then it checks if there are any elements with an ‘input’ tag below in the located search bar.
Connection Conditions
You can set some connection features for your tests with Selenium-4 like latency, going offline, and setting download or upload throughput.
‘new ChromiumNetworkConditions().setOffline(true)’Full page screenshot on firefox
Taking a full-page screenshot in Selenium-4 doesn’t mean the same thing as taking a screenshot in an older version of Selenium. In older versions, you can only take a screenshot of the current view, not the whole page. With Selenium-4 you can take a screenshot of the infinite scrolled page.
WebDriver augmentedDriver = new Augmenter().augment(driver);
File file = ((HasFullPageScreenshot) augmentedDriver).getFullPageScreenshotAs(OutputType.FILE);Performance comparison
Since W3C was replaced with JSON Wire Protocol, our tests can directly communicate with the web browser. It may sound like Selenium is more efficient and fast now, but it actually does not make much difference in the speed of the tests. As the Selenium web driver and web browsers use the same protocol, tests will execute only more consistently. So flakiness of Selenium tests will be reduced substantially.
We already talked about stability and consistency improvements in Selenium-4. Now we are going to compare if there is any performance improvement in this release.
To do this, I have created a repository that contains 2 projects which are use Selenium-4 and Selenium-3 jars from maven and they contain the same test method with annotation @RepeatedTest(15).
@RepeatedTest(15)
public void testAQuery() throws InterruptedException {
WebElement searchBar = driver.findElement(By.name("q"));
searchBar.sendKeys("Selenium");
WebElement searchButton = driver.findElement(By.name("btnK"));
Thread.sleep(500);
searchButton.click();
List<WebElement> results = driver.findElements(By.tagName("h3")) ;
results.get(0).click();
}
I created 2 different GitHub workflows for running them separately and scheduled them to run 6 times a day.
Now it is time to monitor these tests.
To be able to monitor my workflows, I connected my repository to Thundra Foresight while my workflows generate data.
After nearly a week I got this graph from Thundra which means there is no significant performance difference between Selenium-3 and Selenium-4.
Workflow run times of Selenium-3 and Selenium-4
In my performance comparison test, I’ve only used 15 tests that process a simple google search on a single driver. This may not be a very healthy comparison but we can tell that our tests are more consistent and stable with Selenium-4.
Summary
Selenium-4 was officially released On October 13, 2021, and brought lots of upgrades with this release. The most prominent change in this release is the deprecation of JSON wire Protocol. It changed Selenium architecture in a significant way. Besides that, there are also lots of standardizations and stabilizations about the Selenium domain. In this article, we are going to compare coding practices between Selenium-3 and Selenium-4, and we will make a performance comparison between them.
Architectural upgrades on Selenium-4
The most prominent change in this release is the deprecation of JSON Wire Protocol. Actually, this can be considered as the main reason for the Selenium-4 upgrade and Selenium-4 does not include JSON Wire Protocol anymore.
Selenium used to use this protocol for communicating with the web browser. But Selenium-4 has direct communication between the client and server using the W3C (World Wide Web Consortium) protocol, offering lots of advantages compared to the Web Driver used in the versions before Selenium-4.
Architecture of Selenium-3
Native support has been removed for the Opera Driver, and the Opera Driver and browser are no longer active. Users can use Chrome as the Opera browser which is based on Chromium.
Architecture of Selenium-4
After W3C Protocol is replaced with JSON Wire Protocol, you do not need to encode and decode the requests. This direct communication brings more stability and consistency.
The Deprecates from Selenium-3
Find Element methods:
The utility methods to find element(s) in the ‘FindBy’ interface have been removed as they have been used internally only. Instead of this, the ‘By’ class is annotated to increase stability.
Old:
driver.findElementByClassName("className");New:
driver.findElement(By.className("className"));Desired Capabilities:
Desired capabilities were used to define test environments primarily in Selenium Grid. These are replaced with Options in Selenium-4, so you are able to define specific options for drivers.
Old:
DesiredCapabilities caps = DesiredCapabilities.firefox();
caps.setCapability("platform", "Windows 10");
caps.setCapability("version", "92");
caps.setCapability("build", myTestBuild);
caps.setCapability("name", myTestName);
WebDriver driver = new RemoteWebDriver(new URL(cloudUrl), caps);New:
FirefoxOptions browserOptions = new FirefoxOptions();
browserOptions.setPlatformName("Windows 10");
browserOptions.setBrowserVersion("92");
Map<String, Object> cloudOptions = new HashMap<>();
cloudOptions.put("build", myTestBuild);
cloudOptions.put("name", myTestName);
browserOptions.setCapability("cloud:options", cloudOptions);
WebDriver driver = new RemoteWebDriver(new URL(cloudUrl), browserOptions);Actions:
Actions class is for simulating user gestures like a double click, click and hold, and etc. There are newly added methods in this class and they are replaced with the old ones.
Old:
moveToElement(toElement).click()>New:
click(webElement)These are the significant changes from Selenium 3 and there are lots of deprecated methods that came with Selenium-4. Some of the classes and methods that include them are:
- Timeout Parameters
- Merging Capabilities
- Wait parameters
- Browser Interface
- Firefox Legacy
- Browser type
What is new in Selenium-4
There comes lots of remarkable innovations that will make testing easier for you with Selenium-4. For instance, getting screenshots of elements, opening a new tab or window used to be painful. With Selenium-4, these operations are standardized with selenium libraries.
Screenshot of an Element
This was a very painful event in older Selenium versions. You had to take a screenshot of the screen and then you had to crop it using desired element’s coordinates. With Selenium-4, it can be accomplished with a few lines of code:
Old
WebDriver driver = new ChromeDriver();
driver.get("https://www.google.com/");
WebElement logoElement = driver.findElement(By.className("lnXdpd"));
File screenshot = ((TakesScreenshot) driver).getScreenshotAs(OutputType.FILE);
BufferedImage fullImg = ImageIO.read(screenshot);
Point point = logoElement.getLocation();
int eleWidth = logoElement.getSize().getWidth();
int eleHeight = logoElement.getSize().getHeight();
BufferedImage eleScreenshot = fullImg.getSubimage(point.getX(), point.getY(),
eleWidth, eleHeight);
ImageIO.write(eleScreenshot, "png", screenshot);
File screenshotLocation = new File("outputs/elementShotFromScreenShot.png");
FileUtils.copyFile(screenshot, screenshotLocation);
driver.quit();
New
WebDriver driver = new ChromeDriver();
driver.get("https://www.google.com/");
WebElement logoElement = driver.findElement(By.className("lnXdpd"));
File myObj = logoElement.getScreenshotAs(OutputType.FILE);
File DestFile = new File("outputs/elementShotTest.png");
FileUtils.copyFile(myObj, DestFile);
driver.quit();
This brings us a significant performance improvement on taking screenshot because there was lots of operations like crop and save in the old version.
In my example; I ran a test method 3 times which uses the old version of taking screenshots of an element. Then I updated this function with using the new technique and ran it 3 times again.
You can notice a significant reduction in the execution times of the tests in the latter 3 runs than the first 3 runs.
Reducing the execution times of taking a photo of an element in migrating from Selenium-3 to Selenium-4
Multiple Tabs
When you wanted to open a new tab or a new window for your tests, you had to send which shortcut key sequences should be pressed on your screen or you had to run some javascript commands with javascript executed.
With Selenium-4, you can simply tell the driver to switch new tab now.
Old:
String keys = Keys.chord(Keys.CONTROL,"t");
driver.findElement(By.cssSelector("body")).sendKeys(keys);
//or
JavascriptExecutor jse = (JavascriptExecutor)driver;
jse.executeScript("window.open()");
New:
driver.switchTo().newWindow(WindowType.TAB);
Relative Locators
Locating elements might be painful even if you have a wide acquaintance with UI testing. With relative locators, you don’t need to reach each element with the ‘findElement’ methods. You can locate an element from any element you already located.
WebDriver driver = new ChromeDriver();
driver.get("https://www.google.com/");
WebElement searchBar = driver.findElement(By.name("q"));
searchBar.sendKeys("Relative Locators");
WebElement searchButton = driver.findElement(with(By.tagName("input")).below(searchBar));
searchButton.click();
driver.quit();
This code block finds the search bar in google and sends ‘Relative Locator’ keys to it. Then it checks if there are any elements with an ‘input’ tag below in the located search bar.
Connection Conditions
You can set some connection features for your tests with Selenium-4 like latency, going offline, and setting download or upload throughput.
‘new ChromiumNetworkConditions().setOffline(true)’Full page screenshot on firefox
Taking a full-page screenshot in Selenium-4 doesn’t mean the same thing as taking a screenshot in an older version of Selenium. In older versions, you can only take a screenshot of the current view, not the whole page. With Selenium-4 you can take a screenshot of the infinite scrolled page.
WebDriver augmentedDriver = new Augmenter().augment(driver);
File file = ((HasFullPageScreenshot) augmentedDriver).getFullPageScreenshotAs(OutputType.FILE);Performance comparison
Since W3C was replaced with JSON Wire Protocol, our tests can directly communicate with the web browser. It may sound like Selenium is more efficient and fast now, but it actually does not make much difference in the speed of the tests. As the Selenium web driver and web browsers use the same protocol, tests will execute only more consistently. So flakiness of Selenium tests will be reduced substantially.
We already talked about stability and consistency improvements in Selenium-4. Now we are going to compare if there is any performance improvement in this release.
To do this, I have created a repository that contains 2 projects which are use Selenium-4 and Selenium-3 jars from maven and they contain the same test method with annotation @RepeatedTest(15).
@RepeatedTest(15)
public void testAQuery() throws InterruptedException {
WebElement searchBar = driver.findElement(By.name("q"));
searchBar.sendKeys("Selenium");
WebElement searchButton = driver.findElement(By.name("btnK"));
Thread.sleep(500);
searchButton.click();
List<WebElement> results = driver.findElements(By.tagName("h3")) ;
results.get(0).click();
}
I created 2 different GitHub workflows for running them separately and scheduled them to run 6 times a day.
Now it is time to monitor these tests.
To be able to monitor my workflows, I connected my repository to Thundra Foresight while my workflows generate data.
After nearly a week I got this graph from Thundra which means there is no significant performance difference between Selenium-3 and Selenium-4.
Workflow run times of Selenium-3 and Selenium-4
In my performance comparison test, I’ve only used 15 tests that process a simple google search on a single driver. This may not be a very healthy comparison but we can tell that our tests are more consistent and stable with Selenium-4.
*** This is a Security Bloggers Network syndicated blog from Thundra blog authored by Emin Bilgic. Read the original post at: https://blog.thundra.io/performance-comparison-between-selenium-3-and-selenium-4
