Selenium – Automate anything you want with this framework!

IntroductiontoChromeDevTools

ChromeDevToolsisasetoftoolsbuiltdirectlyintoChromium-basedbrowserssuchasChrome,Opera,andMicrosoftEdgetohelpdevelopersdebugandstudywebsites.

WithChromeDevTools,developerscangaindeeperaccesstowebsitesandbeableto:

• InspectelementsintheDOM
• EditelementsandCSSonthefly
• Checkandmonitorwebsiteperformance
• Simulatetheuser’sgeographiclocation
• Simulatefaster/slowernetworkspeeds
• ExecuteanddebugJavaScript
• Viewconsolelog
• etc.

Selenium4ChromeDevToolsAPI

Seleniumisacomprehensivesetoftoolsandlibrariesthatsupportwebbrowserautomation.Selenium4addsnativesupportfortheChromeDevToolsAPI.WiththesenewAPIs,ourtestscannow:

• Captureandmonitornetworktrafficandperformance
• Simulatedgeolocationforlocation-awaretesting,localizationandinternationalizationtesting
• Changedevicemodesandtestyourapp’sresponsiveness

Thisisjustthetipoftheiceberg!

Selenium4introducesthenewChromiumDriverclass,whichincludestwomethodsforaccessingChromeDevTools:getDevTools()andexecuteCdpCommand().

ThegetDevTools()methodreturnsanewDevToolsobjectthatallowsyoutosendbuilt-inSeleniumcommandsforCDPusingthesend()method.ThesecommandsarewrappermethodsthatmakecallingCDPfunctionsclearerandeasier.

TheexecuteCdpCommand()methodalsoallowsyoutoexecuteCDPmethods,butismoreprimitive.InsteadofusingawrappedAPI,itallowsyoutodirectlypassinaChromeDevToolscommandandtheargumentstothatcommand.IfthereisnoSeleniumwrapperAPIforaCDPcommand,oryouwanttocallitdifferentlythantheSeleniumAPI,youcanuseexecuteCdpCommand().

Chromium-baseddriverslikeChromeDriverandEdgeDrivernowinheritfromChromiumDriver,soyoucanaccesstheSeleniumCDPAPIfromthesedriversaswell.

Let’sexplorehowyoucanleveragethesenewSelenium4APIstosolvevarioususecases.

Emulateddevicemode

Mostoftheappswebuildtodayareresponsivetomeettheneedsofendusersfromavarietyofplatforms,devices(e.g.mobile,tablet,wearables,desktop)andscreenorientations.

Astesters,wemaywanttoplaceourappsindifferentsizestotriggerresponsivenessintheapp.

HowcanweachievethisusingSelenium’snewCDPfunctionality?

TheCDPcommandformodifyingdevicemetricsisEmulation.setDeviceMetricsOverride,andthiscommandrequiresinputofwidth,height,mobiledeviceflag,anddevicescalingfactor.Thesefourkeysarerequiredinthisscenario,buttherearesomeoptionalkeys.

InourSeleniumtests,wecanusetheDevTools::send()methodandusethebuilt-insetDeviceMetricsOverride()command,butthisSeleniumAPIaccepts12parameters-inadditiontothe4requiredparameters,thereare8optionalonesparameter.Foranyofthese8optionalparametersthatwedon’tneedtosend,wecanpassOptional.empty().

However,tosimplifythisprocessandonlypasstherequiredparameters,IwillusetheoriginalexecuteCdpCommand()methodinthecodebelow.

packagecom.devtools;

importorg.openqa.selenium.chrome.ChromeDriver;
importorg.openqa.selenium.devtools.DevTools;
importjava.util.HashMap;
importjava.util.Map;

publicclassSetDeviceMode{

finalstaticStringPROJECT_PATH=System.getProperty("user.dir");

publicstaticvoidmain(String[]args){
System.setProperty("webdriver.chrome.driver",PROJECT_PATH+"/src/main/resources/chromedriver");
ChromeDriverdriver;
driver=newChromeDriver();

DevToolsdevTools=driver.getDevTools();
devTools.createSession();
MapdeviceMetrics=newHashMap()
{<!---->{
put("width",600);
put("height",1000);
put("mobile",true);
put("deviceScaleFactor",50);
}};
driver.executeCdpCommand("Emulation.setDeviceMetricsOverride",deviceMetrics);
driver.get("https://www.google.com");
}
}

Online19,Icreateamapcontainingthekeysrequiredforthiscommand.

Thenonline26,IcalltheexecuteCdpCommand()method,passingtwoparameters:thecommandname”Emulation.setDeviceMetricsOverride”,andthedevicemetricsmapcontainingtheparameters.

Online27,Iopenthe”Google”homepagerenderedwiththespecsIprovided,asshownbelow.

WithasolutionlikeApplitoolsEyes,wecannotonlyquicklytestondifferentviewportsusingthesenewSeleniumcommands,butwecanalsomaintainanyinconsistenciesatscale.EyesissmartenoughnottoreportincorrectresultsforsmallandimperceptiblechangesintheUIduetodifferentbrowsersandviewports.

Simulatedlocation

Inmanycasesweneedtotestspecificlocation-basedfeaturessuchasoffers,location-basedpricing,etc.ForthiswecanusetheDevToolsAPItosimulatethelocation.

@Test
publicvoidmockLocation(){
devTools.send(Emulation.setGeolocationOverride(
Optional.of(48.8584),
Optional.of(2.2945),
Optional.of(100)));
driver.get("https://mycurrentlocation.net/");
try{
Thread.sleep(30000);
}catch(InterruptedExceptione){
e.printStackTrace();
}
}

Simulatenetworkspeed

ManyusersaccesswebapplicationsthroughhandhelddevicesconnectedtoWi-Fiorcellularnetworks.Itiscommontoexperienceaweaknetworksignalandthereforeaslowinternetconnection.

Insituationswhereyouhaveaslowinternetconnection(2G)orintermittentoutages,itmaybeimportanttotesthowyourapplicationbehavesundersuchconditions.

TheCDPcommandthatfakesnetworkconnectionsisNetwork.emulateNetworkConditions.Informationabouttherequiredandoptionalparametersofthiscommandcanbefoundinthedocumentation.

ByaccessingChromeDevTools,youcansimulatethesescenarios.Let’sseehowtodothis.

packagecom.devtools;

importorg.openqa.selenium.chrome.ChromeDriver;
importorg.openqa.selenium.devtools.DevTools;
importorg.openqa.selenium.devtools.network.Network;
importorg.openqa.selenium.devtools.network.model.ConnectionType;

importjava.util.HashMap;
importjava.util.Map;
importjava.util.Optional;

publicclassSetNetwork{

finalstaticStringPROJECT_PATH=System.getProperty("user.dir");

publicstaticvoidmain(String[]args){
System.setProperty("webdriver.chrome.driver",PROJECT_PATH+"/src/main/resources/chromedriver");
ChromeDriverdriver;
driver=newChromeDriver();

DevToolsdevTools=driver.getDevTools();
devTools.createSession();
devTools.send(Network.enable(Optional.empty(),Optional.empty(),Optional.empty()));
devTools.send(Network.emulateNetworkConditions(
false,
20,
20,
50,
Optional.of(ConnectionType.CELLULAR2G)
));
driver.get("https://www.google.com");
}
}

Online21,wegettheDevToolsobjectbycallingthegetDevTools()method.Wethencallthesend()methodtoenableNetworkandcallthesend()methodagainpassingthebuilt-incommandNetwork.emulateNetworkConditions()andtheparameterswewanttosendwiththiscommand.

Finally,weopentheGooglehomepageusingsimulatednetworkconditions.

CaptureHTTPrequests

UsingDevTools,wecancaptureHTTPrequestsmadebyapplicationsandaccessmethods,data,headers,andmore.

Let’sseehowtocapturetheHTTPrequest,URI,andrequestmethodusingsamplecode.

packagecom.devtools;

importorg.openqa.selenium.chrome.ChromeDriver;
importorg.openqa.selenium.devtools.DevTools;
importorg.openqa.selenium.devtools.network.Network;

importjava.util.Optional;

publicclassCaptureNetworkTraffic{

privatestaticChromeDriverdriver;
privatestaticDevToolschromeDevTools;

finalstaticStringPROJECT_PATH=System.getProperty("user.dir");

publicstaticvoidmain(String[]args){
System.setProperty("webdriver.chrome.driver",PROJECT_PATH+"/src/main/resources/chromedriver");
driver=newChromeDriver();
chromeDevTools=driver.getDevTools();
chromeDevTools.createSession();

chromeDevTools.send(Network.enable(Optional.empty(),Optional.empty(),Optional.empty()));
chromeDevTools.addListener(Network.requestWillBeSent(),
entry->{
System.out.println("RequestURI:"+entry.getRequest().getUrl()+"\
"
+"Withmethod:"+entry.getRequest().getMethod()+"\
");
entry.getRequest().getMethod();
});
driver.get("https://www.google.com");
chromeDevTools.send(Network.disable());
}
}

TheCDPcommandtostartcapturingnetworktrafficisNetwork.enable.Informationabouttherequiredandoptionalparametersofthiscommandcanbefoundinthedocumentation.

Inourcode,line22usestheDevTools::send()methodtosendtheNetwork.enableCDPcommandtoenablenetworktrafficcapture.

Line23addsalistenerthatlistensforallrequestssentbytheapplication.Foreachrequestcapturedbytheapplication,weextracttheURLusinggetRequest().getUrl()andtheHTTPmethodusinggetRequest().getMethod().

Online29,weopentheGooglehomepageandprinttheURIandHTTPmethodofallrequestsmadebythispageontheconsole.

Oncewehavefinishedcapturingtherequest,wecansendtheNetwork.disableCDPcommandtostopcapturingnetworktraffic,asshownonline30.

InterceptionofHTTPresponses

InordertointercepttheresponsewewillusetheNetwork.responseReceivedevent.ThiseventistriggeredwhenanHTTPresponseisavailable,wecanlistenfortheURL,responseheaders,responsecode,etc.Togettheresponsebody,usetheNetwork.getResponseBodymethod.

@Test
publicvoidvalidateResponse(){
finalRequestId[]requestIds=newRequestId[1];
devTools.send(Network.enable(Optional.of(100000000),Optional.empty(),Optional.empty()));
devTools.addListener(Network.responseReceived(),responseReceived->{
if(responseReceived.getResponse().getUrl().contains("api.zoomcar.com")){
System.out.println("URL:"+responseReceived.getResponse().getUrl());
System.out.println("Status:"+responseReceived.getResponse().getStatus());
System.out.println("Type:"+responseReceived.getType().toJson());
responseReceived.getResponse().getHeaders().toJson().forEach((k,v)->System.out.println((k+":"+v)));
requestIds[0]=responseReceived.getRequestId();
System.out.println("ResponseBody:\
"+devTools.send(Network.getResponseBody(requestIds[0])).getBody()+"\
");
}
});
driver.get("https://www.zoomcar.com/bangalore");
driver.findElement(By.className("search")).click();
}

AccessConsoleLog

Weallrelyonlogsfordebuggingandanalyzingfailures.Whentestingandworkingwithapplicationswithspecificdataorspecificconditions,logscanhelpusdebugandcaptureerrormessages,providingadditionalinsightspublishedintheconsoletabofChromeDevTools.

WecancaptureconsolelogsthroughourSeleniumscriptbycallingtheCDPlogcommandasshownbelow.

packagecom.devtools;

importorg.openqa.selenium.chrome.ChromeDriver;
importorg.openqa.selenium.devtools.DevTools;
importorg.openqa.selenium.devtools.log.Log;

publicclassCaptureConsoleLogs{

privatestaticChromeDriverdriver;
privatestaticDevToolschromeDevTools;
finalstaticStringPROJECT_PATH=System.getProperty("user.dir");

publicstaticvoidmain(String[]args){
System.setProperty("webdriver.chrome.driver",PROJECT_PATH+"/src/main/resources/chromedriver");
driver=newChromeDriver();
chromeDevTools=driver.getDevTools();
chromeDevTools.createSession();

chromeDevTools.send(Log.enable());
chromeDevTools.addListener(Log.entryAdded(),
logEntry->{
System.out.println("log:"+logEntry.getText());
System.out.println("level:"+logEntry.getLevel());
});
driver.get("https://testersplayground.herokuapp.com/console-5d63b2b2-3822-4a01-8197-acd8aa7e1343.php");
}
}

Inourcode,line19usesDevTools::send()toenableconsolelogcapture.

Wethenaddalistenertocaptureallconsolelogsloggedbytheapplication.Foreachlogcapturedbytheapplication,weextractthelogtextusingthegetText()methodandtheloglevelusingthegetLevel()method.

Finally,opentheapplicationandcapturetheconsoleerrorlogpostedbytheapplication.

Captureperformancemetrics

Intoday’sfast-pacedworld,whereweiterativelybuildsoftwareatsuchafastpace,weshouldalsoiterativelydetectperformancebottlenecks.Poorlyperformingwebsitesandpagesthatloadslowlycanleavecustomersdissatisfied.

Canweverifythesemetricsoneverybuild?Yeswecan!

TheCDPcommandtocaptureperformancemetricsisPerformance.enable.Informationaboutthiscommandcanbefoundinthedocumentation.

Let’sseehowthisprocessisdoneinSelenium4andChromeDevToolsAPI.

packagecom.devtools;

importorg.openqa.selenium.chrome.ChromeDriver;
importorg.openqa.selenium.devtools.DevTools;
importorg.openqa.selenium.devtools.performance.Performance;
importorg.openqa.selenium.devtools.performance.model.Metric;
importjava.util.Arrays;
importjava.util.List;
importjava.util.stream.Collectors;

publicclassGetMetrics{

finalstaticStringPROJECT_PATH=System.getProperty("user.dir");

publicstaticvoidmain(String[]args){
System.setProperty("webdriver.chrome.driver",PROJECT_PATH+"/src/main/resources/chromedriver");
ChromeDriverdriver=newChromeDriver();
DevToolsdevTools=driver.getDevTools();
devTools.createSession();
devTools.send(Performance.enable());

driver.get("https://www.google.org");

List<Metric>metrics=devTools.send(Performance.getMetrics());
List<String>metricNames=metrics.stream()
.map(o->o.getName())
.collect(Collectors.toList());

devTools.send(Performance.disable());

List<String>metricsToCheck=Arrays.asList(
"Timestamp","Documents","Frames","JSEventListeners",
"LayoutObjects","MediaKeySessions","Nodes",
"Resources","DomContentLoaded","NavigationStart");

metricsToCheck.forEach(metric->System.out.println(metric+
"is:"+metrics.get(metricNames.indexOf(metric)).getValue()));
}
}

First,wecreateasessionbycallingthecreateSession()methodofDevTools,asshowninline19.

Next,weenableDevToolstocaptureperformancemetricsbysendingthePerformance.enable()commandtosend()asshownonline20.

Onceperformancecaptureisenabled,wecanopentheapplicationandsendthePerformance.getMetrics()commandtosend().ThiswillreturnalistofMetricobjects,whichwecanstreamtogetthenamesofallcapturedmetrics,asshownonline25.

WethendisabletheperformancecapturebysendingthePerformance.disable()commandtosend()asshownonline29.

Toviewthemetricsweareinterestedin,wedefinealistcalledmetricsToCheckandthenprintthevaluesofthemetricsbyloopingoverthelist.

BasicAuthentication

InSelenium,itisnotpossibletointeractwithabrowserpopupbecauseitcanonlyinteractwithDOMelements.Thisposesachallengeforpop-upssuchasauthenticationdialogs.

WecanbypassthisissuebyusingtheCDPAPItohandleauthenticationdirectlywithDevTools.TheCDPcommandthatsetsadditionalheadersforarequestisNetwork.setExtraHTTPHeaders.

Here’showtocallthiscommandinSelenium4.

packagecom.devtools;

importorg.apache.commons.codec.binary.Base64;
importorg.openqa.selenium.By;
importorg.openqa.selenium.chrome.ChromeDriver;
importorg.openqa.selenium.devtools.DevTools;
importorg.openqa.selenium.devtools.network.Network;
importorg.openqa.selenium.devtools.network.model.Headers;
importjava.util.HashMap;
importjava.util.Map;
importjava.util.Optional;

publicclassSetAuthHeader{

privatestaticfinalStringUSERNAME="guest";
privatestaticfinalStringPASSWORD="guest";
finalstaticStringPROJECT_PATH=System.getProperty("user.dir");

publicstaticvoidmain(String[]args){
System.setProperty("webdriver.chrome.driver",PROJECT_PATH+"/src/main/resources/chromedriver");
ChromeDriverdriver=newChromeDriver();

//CreateDevToolssessionandenableNetwork
DevToolschromeDevTools=driver.getDevTools();
chromeDevTools.createSession();
chromeDevTools.send(Network.enable(Optional.empty(),Optional.empty(),Optional.empty()));

//Openwebsite
driver.get("https://jigsaw.w3.org/HTTP/");

//Sendauthorizationheader
Map<String,Object>headers=newHashMap<>();
StringbasicAuth="Basic"+newString(newBase64().encode(String.format("%s:%s",USERNAME,PASSWORD).getBytes()));
headers.put("Authorization",basicAuth);
chromeDevTools.send(Network.setExtraHTTPHeaders(newHeaders(headers)));

//Clickauthenticationtest-thisnormallyinvokesabrowserpopupifunauthenticated
driver.findElement(By.linkText("BasicAuthenticationtest")).click();

StringloginSuccessMsg=driver.findElement(By.tagName("html")).getText();
if(loginSuccessMsg.contains("Yourbrowsermadeit!")){
System.out.println("Loginsuccessful");
}else{
System.out.println("Loginfailed");
}

driver.quit();
}
}

WefirstcreateasessionusingtheDevToolsobjectandenableNetwork.Thisisshowninlines25-26.

Next,weopenourwebsiteandcreatetheauthenticationheaderstosend.

Online35,wesendthesetExtraHTTPHeaderscommandtosend(),alongwiththeheader’sdata.Thispartwillauthenticateusandallowustobypassbrowserpop-ups.

Totestthisfeature,weclickedontheBasicAuthenticationtestlink.Ifyoutrythismanually,youwillseeabrowserpop-upaskingyoutologin.Butsincewesenttheauthenticationheader,thispopupwillnotappearinourscript.

Instead,wereceivethemessage”Yourbrowserloginwassuccessful!”.

Summary

SeleniumhasbecomeevenmorepowerfulwiththeadditionoftheCDPAPI.WecannowenhanceourteststocaptureHTTPnetworktraffic,collectperformancemetrics,handleauthentication,andsimulategeolocation,timezone,anddevicemode.AndanyotherfeaturesthatmightcomeinChromeDevTools!

Finally,Iwouldliketothankeveryonewhoreadmyarticlecarefully.Reciprocityisalwaysnecessary.Althoughitisnotaveryvaluablething,ifyoucanuseit,youcantakeitdirectly:

Thisinformationshouldbethemostcomprehensiveandcompletepreparationwarehousefor[softwaretesting]friends.Thiswarehousehasalsoaccompaniedtensofthousandsoftestengineersthroughthemostdifficultjourney.Ihopeitcanalsohelpyou!