selenium is a tool for web application testing. Selenium tests run directly in the browser, just like real users.
By writing Selenium scripts that imitate user operations, you can drive the browser to perform specific actions from the end user’s perspective. This feature is very friendly for us to crawl pages dynamically rendered by JavaScript.
Because the JavaScript on pages dynamically rendered by JavaScript is usually compiled and packaged, all you see are shortcodes, which are very difficult to read.
1. Selenium basics
Official documentation: https://selenium.dev/selenium/docs/api/py/api.html
Selenium automated testing tool: can drive the browser to perform specific actions (for example: click, drop-down, etc.). At the same time, you can also obtain the source code of the page currently rendered by the browser.
1. Install Selenium
pip install selenium
Commonly used package imports: ?
from selenium import webdriver from selenium.webdriver import ActionChains from selenium.webdriver.common.by import By from selenium.webdriver.common.keys import Keys from selenium.webdriver.support import expected_conditions as EC from selenium.webdriver.support.wait import WebDriverWait import time
2. Install Google Chrome and ChromeDriver
ChromeDriver mirror website: http://npm.taobao.org/mirrors/chromedriver/[1]
ChromeDriver installation steps: ① Download chromedriver.exe (note that the browser version must be consistent) ② Configure chromedriver.exe to the system environment variable
Path(it is recommended to directly copy the Python installation directory Scripts directory)
First write a simple small function to demonstrate (the browser automatically opens Baidu search for “Zeng Qinglin”):
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
browser = webdriver.Chrome()
# Get Google Chrome
browser.get('https://www.baidu.com')
# Open Baidu
input = browser.find_element_by_id('kw')
# Find the search box with id kw
input.send_keys('Zeng Qinglin')
# Set search text
input.send_keys(Keys.ENTER)
# Send the Enter key
print(browser.current_url)
# Get the current address of the browser
print(browser.get_cookies())
# Get browser page cookies
print(browser.page_source)
# Get the browser page source file
Code execution results
3. Create a browser object?
from selenium import webdriver browser = webdriver.Chrome()
Or use PhantomJS (a “headless” browser based on Webkit)?
browser = webdriver.PhantomJS()
4. Load the web page?
browser.get(url)
2. Browser operation
1. Load the web page?
browser.get(url)
2. Execute js?
browser.execute_script(js code string)
3. Screenshot?
browser.save_screenshot("filename.png")
4. Forward and backward?
# Page back browser.back() # Page forward browser.forward()
5.Tab switching?
browser.switch_to.window(browser.window_handles[subscript])
6.Close?
# Close the current tab or window browser.close() # Close all tabs or windows browser.quit()
3. Node selection
1. Single node selection?
browser.find_element_by_id('kw') # Native syntax
browser.find_element_by_css_selector('#kw') # CSS syntax
browser.find_element_by_xpath("//*[@id='kw']") # XPath syntax
The element types obtained by the above three methods are all WebElement types. The following two methods are equivalent: ?
browser.find_element_by_id('kw')
browser.find_element(By.ID, 'kw')
2. Multi-node selection
Similar to single node selection, just change element to elements. ?
browser.find_elements_by_id() browser.find_elements_by_name() browser.find_elements_by_tag_name() browser.find_elements_by_class_name() browser.find_elements_by_xpath() browser.find_elements_by_css_selector() browser.find_elements_by_link_text() browser.find_elements_by_partial_link_text()
3. Obtain the page source code?
browser.page_source
4. Extract node data
| Syntax | Data |
| element.get_attribute(attribute name) | Get node attribute value |
| element.text | Get node text content |
| element.id | Get node ID |
| element.location | Get node location |
| element.tag_name | Get node tag |
| element.size | Get node size |
5. Action
1. Get nodes
?Direct method?
element = browser.find_element_by_XXX(value)
?Use By type (need to import By)?
from selenium.webdriver.common.by import By element = browser.find_element(By.ID, value)
2. Operations on node elements
| Action | Description |
| element.click() | Click |
| element.submit() | Submit |
| element.send_keys (data) | Enter data into the input element |
| element.clear() | Clear the data in the input element |
6. Action chain
Used to simulate mouse dragging, mouse button pressing and other operations.
1. Create an action chain object?
from selenium.webdriver import ActionChains actions = ActionChains(browser)
2. Action
(1) Mouse operation
| Action | Description |
| actions.click(on_element) | Click |
| actions.double_click(on_element) | Double-click |
| actions .context_click(on_element) | Right click |
| actions.click_and_hold(on_element) | Click and hold |
| actions.drag_and_drop(source, target) | Drag and drop |
| actions.drag_and_drop_by_offset(source, target, xoffset, yoffset ) | Drag to the relative position of the element |
| actions.move_by_offset(xoffset, yoffset) | Move the cursor |
| actions.move_to_element(to_element) | Move the cursor to the element |
| actions.move_to_element_with_offset(to_element, xoffset, yoffset) | Move the cursor to the relative position of the element |
(2) Keyboard operation
| Action | Description |
| actions.send_keys(data) | Send information to the currently focused element |
| actions.send_keys_to_element(element, data) | Send information to the element |
| actions.key_down(value, element) | Press the key |
| actions.key_up(value, element) | Release button |
3. Execute action chain?
actions.perform()
7. Delayed waiting
1. Fixed waiting (forced waiting)
Waits for a fixed amount of time, regardless of page loading. ?
time.sleep(seconds)
2. Implicit waiting
When the page is loaded or times out, proceed to the next step. ?
browser.implicitly_wait(time_to_wait)
3.Show waiting
The next step will be executed only when the condition is met. If it times out, a TimeoutException will be thrown. ?
from selenium.webdriver.support import expected_conditions as EC wait = WebDriverWait(browser, timeout) element = wait.until(EC.condition)
4. Waiting conditions
| Waiting conditions | Meaning |
| title_is | The title is some content |
| title_contains | The title contains some content |
| presence_of_element_located | When the node is loaded, the positioning tuple needs to be passed in (for example: (By.ID, ‘q’)) |
| visibility_of_element_located | Node courseware, pass in the positioning tuple |
| visibility_of | The incoming node object is visible |
| presence_of_all_elements_located | All nodes visible |
| text_to_be_present_in_element | A node contains a certain text |
| element_to_be_clickable | Node can be clicked |
| staleness_of | To determine whether a node is still in the DOM, you can determine the page Whether it has been refreshed |
| alert_is_present | Determine whether the warning box pops up |
8. Cookie operation
1. Get cookies?
browser.get_cookies()
2. Add cookie?
browser.add_cookie({"name": name, "value": value})
3. Delete cookies?
browser.delete_cookie(name) browser.delete_all_cookies()
Reference: https://www.jianshu.com/p/a36f92f74e75
Take action, it’s better to be on the road than to wait and see. In the future, you will definitely thank yourself for working hard now! If you want to learn and improve but can’t find the information and there is no one to answer your questions, please join the group in time: 786229024. There are various test and development materials and technologies in which you can communicate together.
Finally: The complete software testing video tutorial below has been compiled and uploaded. Friends who need it can get it by themselves[Guaranteed 100% Free]
Software testing interview document
We must study to find a high-paying job. The following interview questions are the latest interview materials from first-tier Internet companies such as Alibaba, Tencent, Byte, etc., and some Byte bosses have given authoritative answers. After finishing this set I believe everyone can find a satisfactory job based on the interview information.
The knowledge points of the article match the official knowledge files, and you can further learn relevant knowledge. Python entry skill treeWeb crawlerSelenium386997 people are learning the system