Appium touch operations (short press, long press, zoom, slide, etc.) [2]

Table of Contents

1.Introduce the object: TouchAction: touch

A. Click (tap) operation:

B. Slide the screen multiple times:

C. Long press

2.MultiAction —– Multi-touch zoom in and out operation

2.1 Zoom in and out based on analytical gesture operations

2.2 Another method for zooming in/out: zoom/pinch

3. Sliding screen operation:

A. driver.swipe(x1, y1, x2, y2,duration)

B.driver.scroll

C. driver.drag_and_drop

**To summarize the above three methods:

---

1. Introduce object: TouchAction: touch

from appium.webdriver.common.touch_action import TouchAction

TouchAction(driver)
– TouchAction objects include (tab), press (short press), move_to (slide to a certain coordinate) and other methods
– Through the TouchAction object, add tap, move_to and other operations, and then perform () execution, you can realize functions such as unlocking the screen
The events available in the specification are:

Press, release, moveTo, tap, wait,
Long press (longPress), cancel (cancel), execute (perform)

A. Click (tap) operation:

driver.tap (coordinate x, coordinate y, continuous click time/s) – coordinate positioning is recommended to be used sparingly and only as a last resort
In addition to positioning clicks on elements, coordinate clicks can also be implemented through positioning expressions.

The first type:
TouchAction(driver).tap(x=155, y=706, count=1).perform()
The second type:
TouchAction(driver).tap(ele, count=1).perform()

B. Swipe the screen multiple times:

It is recommended to use less coordinate positioning. The following are just examples

from appium.webdriver.common.touch_action import TouchAction
# You need to release after a short press. You also need to release after a long press.
action=TouchAction(driver)
action.press(x=220,y=700).move_to(x=840, y=700).move_to(x=220, y=1530).move_to(x=840, y=1530).release(). perform()

Example: Unlock the nine-square grid in WeChat service

import time
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from appium import webdriver
from appium.webdriver.common.mobileby import MobileBy
from appium.webdriver.common.touch_action import TouchAction

desired_caps = {
    'platformName': 'Android',
    'platformVersion': '8.1',
    'deviceName': 'ye_shen_Android',
    'appPackage': 'com.tencent.mm',
    'appActivity': 'com.tencent.mm.ui.LauncherUI',
    'noReset': True,
    'autoAcceptAlerts': True,
    'newCommandTimeout': 240
}

driver = webdriver.Remote('http://127.0.0.1:4723/wd/hub', desired_caps)
driver.implicitly_wait(10)

# Click WeChat Mine
loc = (MobileBy.ANDROID_UIAUTOMATOR, 'new UiSelector().resourceId("com.tencent.mm:id/j5t")')
WebDriverWait(driver, 30).until(EC.visibility_of_element_located(loc))
TouchAction(driver).tap(x=540, y=1255).wait(200).perform()

# Click Service
locc = (MobileBy.ANDROID_UIAUTOMATOR, 'new UiSelector().text("Service")')
WebDriverWait(driver, 30).until(EC.visibility_of_element_located(locc))
ele = driver.find_element(MobileBy.ANDROID_UIAUTOMATOR, 'new UiSelector().text("Service")')
TouchAction(driver).tap(ele).wait(200).perform()

# Check Please enter the gesture password
loccs = (MobileBy.ANDROID_UIAUTOMATOR, 'new UiSelector().text("Please enter the gesture password")')
WebDriverWait(driver, 30).until(EC.visibility_of_element_located(loccs))

# Get the starting coordinates - the upper left vertex of the nine-square grid
loc = driver.find_element(MobileBy.ID, "com.tencent.mm:id/hrs").location
print(loc)
# Get the size of the nine-square grid element height width width and height
loc_size = driver.find_element(MobileBy.ID, "com.tencent.mm:id/hrs").size
print(loc_size)

# Get the mean value after segmentation, which is the step size of the nine-square grid to be used below
step = loc_size["width"] / 6
step_x = loc_size["width"] / 6
step_y = loc_size["height"] / 6

# Calculate the coordinates of each point
point1 = (loc["x"] + step, loc["y"] + step * 5)
point2 = (point1[0], point1[1] - step * 2)
point3 = (point2[0], point2[1] - step * 2)
point4 = (point3[0] + step * 2, point3[1] + step * 2)
point5 = (point4[0] + step * 2, point4[1] + step * 2)
point6 = (point5[0], point5[1] - step * 2)
point7 = (point6[0], point6[1] - step * 2)


# Perform sliding operation
TouchAction(driver).press(x=point1[0], y=point1[1]).wait(500). \
    move_to(x=point2[0], y=point2[1]). \
    move_to(x=point3[0], y=point3[1]). \
    move_to(x=point4[0], y=point4[1]). \
    move_to(x=point5[0], y=point5[1]). \
    move_to(x=point6[0], y=point6[1]). \
    move_to(x=point7[0], y=point7[1]). \
    release().perform()

time.sleep(10)
driver.quit()

As shown in the picture:

C. Long press

For example: an example of long press: It is recommended to use coordinate positioning less often. The following is just an example

import time
from appium import webdriver
from appium.webdriver.common.touch_action import TouchAction
from appium.webdriver.webdriver import By

des = {
"platformName": "Android",
"platformVersion": "9",
"deviceName": "windwos virtual machine",
"appPackage": "com.android.settings",
"appActivity": "com.android.settings.Settings",
"udid": "192.168.0.101:5555",
"noReset": "True"
}

driver = webdriver.Remote('http://127.0.0.1:4723/wd/hub', des)
driver.implicitly_wait(10)


touch = TouchAction(driver)
#touch.press(x=720, y=1910).perform()
touch.long_press(x=720, y=1910).perform()

2.MultiAction —– Multi-touch Zoom operation

Click here to learn from the article
Background: When using map apps and zooming in and out of images, we often need to zoom in on the interface to more conveniently view the location or zoom in on the image.

2.1 Zoom in and out based on analysis gesture operation

• (1) Appium provides the MultiAction class for zooming in and out operations. Simulate real finger operations, as shown in the figure:

• 

• (2) MultiAction is a multi-touch class that can simulate user multi-touch operations. It mainly includes two methods: add() and perform(). MultiAction can be combined with the previously learned ActionTouch to simulate the user’s multiple finger sliding operation effects.

from appium.webdriver.common.multi_action import MultiAction
from appium.webdriver.common.touch_action import TouchAction

add method: touch_actions is one or more TouchAction objects, describing the action to be performed by a finger

 def add(self, *touch_actions: 'TouchAction') -> None:
        """Add TouchAction objects to the MultiAction, to be performed later.

        Args:
            touch_actions: one or more TouchAction objects describing a chain of actions to be performed by one finger

        Usage:
            | a1 = TouchAction(driver)
            | a1.press(el1).move_to(el2).release()
            | a2 = TouchAction(driver)
            | a2.press(el2).move_to(el1).release()
            | MultiAction(driver).add(a1, a2)

        Returns:
            `MultiAction`: Self instance
        """
        for touch_action in touch_actions:
            if self._touch_actions is None:
                self._touch_actions = []

            self._touch_actions.append(copy.copy(touch_action))

perform method

 def perform(self: T) -> T:
        """Perform the actions stored in the object.

        Usage:
            | a1 = TouchAction(driver)
            | a1.press(el1).move_to(el2).release()
            | a2 = TouchAction(driver)
            | a2.press(el2).move_to(el1).release()
            | MultiAction(driver).add(a1, a2).perform()

        Returns:
            `MultiAction`: Self instance
        """
        self._driver.execute(Command.MULTI_ACTION, self.json_wire_gestures)

        # clean up and be ready for the next batch
        self._touch_actions = []

        return self

usage:

from appium.webdriver.common.multi_action import MultiAction
from appium.webdriver.common.touch_action import TouchAction

a1 = TouchAction(driver)
a1.press(el1).move_to(el2).release()

a2 = TouchAction(driver)
a2.press(el2).move_to(el1).release()

MultiAction(driver).add(a1, a2).perform()

Code practice: Zoom in and out of the map in WeChat

import time
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from appium import webdriver
from appium.webdriver.common.mobileby import MobileBy
from appium.webdriver.common.touch_action import TouchAction
from appium.webdriver.common.multi_action import MultiAction


desired_caps = {
    'platformName': 'Android',
    'platformVersion': '8.1',
    'deviceName': 'ye_shen_Android',
    'appPackage': 'com.tencent.mm',
    'appActivity': 'com.tencent.mm.ui.LauncherUI',
    'noReset': True,
    'autoAcceptAlerts': True,
    'newCommandTimeout': 240,
    # 'automationName': 'uiautomator2'
}

driver = webdriver.Remote('http://127.0.0.1:4723/wd/hub', desired_caps)
driver.implicitly_wait(10)

# Click on WeChat chat box
loc = (MobileBy.ANDROID_UIAUTOMATOR, 'new UiSelector().text("Address Book")')
WebDriverWait(driver, 30).until(EC.visibility_of_element_located(loc))
# el = driver.find_element(MobileBy.ANDROID_UIAUTOMATOR,'new UiSelector().resourceId("com.tencent.mm:id/bth")')
TouchAction(driver).tap(x=152, y=315).wait(200).perform()

# Click on the map inside
locc = (MobileBy.ANDROID_UIAUTOMATOR, 'new UiSelector().textContains("Xixiang")')
WebDriverWait(driver, 30).until(EC.visibility_of_element_located(locc))
ele = driver.find_element(MobileBy.ANDROID_UIAUTOMATOR, 'new UiSelector().textContains("Xixiang")')
TouchAction(driver).tap(ele).wait(200).perform()

def getSize(driver):
    x = driver.get_window_size()['width']
    y = driver.get_window_size()['height']
    return x, y

# Zoom in
def narrow():
    x, y = getSize(driver)
    action1 = TouchAction(driver)
    action2 = TouchAction(driver)
    zoom_action = MultiAction(driver)
    action1.press(x=x*0.2, y=y*0.2).wait(1000).move_to(x=x*0.4, y=y*0.4).wait(1000).release()
    action2.press(x=x*0.8, y=y*0.8).wait(1000).move_to(x=x*0.6, y=y*0.6).wait(1000).release()
    zoom_action.add(action1, action2)
    zoom_action.perform()

# Zoom out
def enlarge():
    x, y = getSize(driver)
    action1=TouchAction(driver)
    action2=TouchAction(driver)
    zoom_action=MultiAction(driver)
    action1.press(x=x*0.4,y=y*0.4).wait(1000).move_to(x=x*0.2,y=y*0.2).wait(1000).release()
    action2.press(x=x*0.6,y=y*0.6).wait(1000).move_to(x=x*0.8,y=y*0.8).wait(1000).release()
    zoom_action.add(action1,action2)
    zoom_action.perform()


if __name__ == '__main__':
    for i in range(5):
        narrow()

    for i in range(5):
        enlarge()

2.2 Another method for zooming in/out: zoom/pinch

Parameters: percent: perform a magnification operation on a certain control, the default magnification ratio is 200%, steps: indicates how many steps the magnification/zooming action is completed, the default is 50

– zoom(self, element=None, percent=200, steps=50)
– pinch(self, element=None, percent=200, steps=50)

# Zoom in
driver.zoom(element)
# Zoom out
driver.pinch(element)

3. Sliding screen operation:

Click here to learn from the article

A. driver.swipe(x1, y1, x2, y2,duration)

swipe(startX,startY,endX,endY)
StartX – EndX: Distance X slides
Start Y – End Y: distance Y slides

Slide from coordinates (x1, x2) to coordinates (x2, y2), duration: duration/milliseconds, sliding time (the sliding coordinates cannot exceed the width and height of the screen)

size = driver.get_window_size()
print(driver.get_window_size())
What is returned is: {'width': 1080, 'height': 1776}
Use the swipe(self, start_x, start_y, end_x, end_y, duration=0) method:
start_x = size['width'] * 0.9 # Width
start_y = size['height'] * 0.5 # Height
end_x = size['width'] * 0.1
end_y = size['height'] * 0.5

The method of moving a screen up, down, left, and right is as follows:

import time
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from appium import webdriver
from appium.webdriver.common.mobileby import MobileBy

desired_caps = {
    'platformName': 'Android',
    'platformVersion': '8.1',
    'deviceName': 'ye_shen_Android',
    'appPackage': 'com.tencent.mm',
    'appActivity': 'com.tencent.mm.ui.LauncherUI',
    'noReset': True,
    'autoAcceptAlerts': True,
    'newCommandTimeout': 240
}

driver = webdriver.Remote('http://127.0.0.1:4723/wd/hub', desired_caps)

loc = (MobileBy.ANDROID_UIAUTOMATOR, 'new UiSelector().resourceId("com.tencent.mm:id/cn_")')
WebDriverWait(driver, 10).until(EC.visibility_of_element_located(loc))



#Get screen size width and height
def getSize(driver):
    x = driver.get_window_size()['width']
    y = driver.get_window_size()['height']
    return (x, y)

#Screen slide up
def swipeUp(driver,t=1000):
    l = getSize(driver)
    x1 = int(l[0] * 0.5) #x coordinate
    y1 = int(l[1] * 0.75) #Start y coordinate
    y2 = int(l[1] * 0.25) #End point y coordinate
    driver.swipe(x1, y1, x1, y2,t)

#Screen slide down
def swipeDown(driver,t=1000):
    l = getSize(driver)
    x1 = int(l[0] * 0.5) #x coordinate
    y1 = int(l[1] * 0.25) #Start y coordinate
    y2 = int(l[1] * 0.75) #End point y coordinate
    driver.swipe(x1, y1, x1, y2,t)
#Screen slide left
def swipLeft(driver,t):
    l=getSize(driver)
    x1=int(l[0]*0.75)
    y1=int(l[1]*0.5)
    x2=int(l[0]*0.05)
    driver.swipe(x1,y1,x2,y1,t)
#Screen slide right
def swipRight(driver,t=1000):
    l=getSize(driver)
    x1=int(l[0]*0.05)
    y1=int(l[1]*0.5)
    x2=int(l[0]*0.75)
    driver.swipe(x1,y1,x2,y1,t)
 
#Call the slide down method
swipeDown(driver) 

B.driver.scroll

• Scroll from one element to another, only by sliding between two elements.
• The first picture is a pull-down to open the mini program, and the second picture is a bottle of water instead of the subscription account message location.

Method details:

Parameters: origin_el – the starting element to scroll; destination_el – the end element to scroll to; duration – duration, in milliseconds, the default is 600 ms

def scroll(self: T, origin_el: WebElement, destination_el: WebElement, duration: Optional[int] = None) -> T:
    if self.w3c and duration is None:
            duration=600

        action = TouchAction(self)
        if duration is None:
            action.press(origin_el).move_to(destination_el).release().perform()
        else:
            action.press(origin_el).wait(duration).move_to(destination_el).release().perform()
        return self

Code practice:

from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from appium import webdriver
from appium.webdriver.common.mobileby import MobileBy

desired_caps = {
    'platformName': 'Android',
    'platformVersion': '8.1',
    'deviceName': 'ye_shen_Android',
    'appPackage': 'com.tencent.mm',
    'appActivity': 'com.tencent.mm.ui.LauncherUI',
    'noReset': True,
    'autoAcceptAlerts': True,
    'newCommandTimeout': 240,
    # 'automationName': 'uiautomator2'
}

driver = webdriver.Remote('http://127.0.0.1:4723/wd/hub', desired_caps)
driver.implicitly_wait(10)

#click WeChat
loc = (MobileBy.ANDROID_UIAUTOMATOR, 'new UiSelector().text("Address Book")')
WebDriverWait(driver, 30).until(EC.visibility_of_element_located(loc))

# Subscription account message
stop_element = driver.find_element(MobileBy.XPATH, '//*[@resource-id="com.tencent.mm:id/gkp"]/android.widget.LinearLayout[2]')

# Bottled water
start_element = driver.find_element(MobileBy.XPATH, '//*[@resource-id="com.tencent.mm:id/gkp"]/android.widget.LinearLayout[7]')

# Execute sliding operation. Choose one of the following two to execute.
# Pull down WeChat to mini program
driver.scroll(stop_element,start_element,3000)
# Bottled water instead of subscription account message
driver.scroll(start_element,stop_element,3000)

Scroll down to mini program

Note: There is inertia in the operation process, and the duration parameter needs to be added. The larger the parameter value, the smaller the inertia.

C. driver.drag_and_drop

Dragging from one element to another causes the second element to take its original position on the screen.
Method: Parameters: origin_el – the starting element of the page to be slid; destination_el – the end element of the page to be slid.

def drag_and_drop(self: T, origin_el: WebElement, destination_el: WebElement) -> T:
    action = TouchAction(self)
    action.long_press(origin_el).move_to(destination_el).release().perform()
    return self

Code practice:

from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from appium import webdriver
from appium.webdriver.common.mobileby import MobileBy

desired_caps = {
    'platformName': 'Android',
    'platformVersion': '8.1',
    'deviceName': 'ye_shen_Android',
    'appPackage': 'com.tencent.mm',
    'appActivity': 'com.tencent.mm.ui.LauncherUI',
    'noReset': True,
    'autoAcceptAlerts': True,
    'newCommandTimeout': 240,
    # 'automationName': 'uiautomator2'
}

driver = webdriver.Remote('http://127.0.0.1:4723/wd/hub', desired_caps)
driver.implicitly_wait(10)

#click WeChat
loc = (MobileBy.ANDROID_UIAUTOMATOR, 'new UiSelector().text("Address Book")')
WebDriverWait(driver, 30).until(EC.visibility_of_element_located(loc))

# Subscription account message
stop_element = driver.find_element(MobileBy.XPATH, '//*[@resource-id="com.tencent.mm:id/gkp"]/android.widget.LinearLayout[2]')

# Bottled water
start_element = driver.find_element(MobileBy.XPATH, '//*[@resource-id="com.tencent.mm:id/gkp"]/android.widget.LinearLayout[7]')

# Execute sliding operation. Choose one of the following two to execute.
# Pull down WeChat to mini program
driver.drag_and_drop(stop_element,start_element)
# Bottled water instead of subscription account message
driver.drag_and_drop(start_element,stop_element)

driver.drag_and_drop method description: The duration cannot be set and there is no inertia.

**Summary of the above three methods:

Sliding and dragging are nothing more than considering whether it has “inertia” and whether the passed parameters are “elements” or “coordinates”.
1) scroll: has “inertia”, pass in the “element”, and you can control the inertia by setting the duration parameter.
2) drag_and_drop: No “inertia”, pass in “element”
3) swipe: There is “inertia”. Pass in “coordinates” and you can control the inertia by setting the duration parameter.

original