Table of contents
- How riot automated tests are built ?
- 1. How does appium work ?
- 2. Project organisation
- 3. Create / maintain tests
- Update objects repository
- How to inspect UI elements from application under test:
- Use multiple devices in one test
- 4. Tests execution and tests reports
- Execute tests via your IDE
- Execute tests via maven-surefire-plugin (suitable to continuous integration)
- 5. Tweaks
How riot automated tests are built ?
This project relies on appium to interact with riot-android and riot-ios applications.
Appium is an open source test automation framework for mobile applications. He is both android and ios compliant (+ windows applications).
1. How does appium work ?
- Appium has a client/server architecture. Appium server exposes a REST API and receives connections from a client. The server listens and executes commands on a mobile device, and responds to the clients. Appium interacts with vendor-provided framework (UIAutomator for Android and XCTest for iOS).
- Appium server is written in Node.js.
- Appium client exists in multiple languages. We chose JAVA.
- It's possible for the server and the client to be on differents machines. It's not the case for the moment on this project.
More informations on Appium here.
2. Project organisation
What is Page Object Model (POM)
POM is a design pattern to create object repository for UI elements.
It allows to write more readable, maintainable and readable automated tests.
For each page (login page, home page, settings page for example) of the application under test, a class exists and references objects of the page (buttons, labels, textfields, checkboxes, containers, etc), and some page specific methods. Addresses of these objects are only referenced in these POM classes and test classes don't have to worry about how to accede these objects.
Example in riot-automated-test:
-
Some testcase snippet:
RiotRoomPageObjects roomA=new RiotRoomPageObjects(appiumFactory.getiOsDriver1()); //3. User A write something in the message bar but don't sen`d it. roomA.sendKeyTextView.setValue(notSentMsg); roomA.getPendingCallView("Offline");
... uses sendKeyTextView object and getPendingCallView() method from RiotRoomPageObjects POM class.
-
RiotRoomPageObjects POM class snippet:
public MobileElement hangUpCallButton; @iOSFindBy(accessibility="EncryptedRoomIcon") public MobileElement e2eIconImage; @iOSFindBy(accessibility="GrowingTextView") public MobileElement sendKeyTextView;
Project file tree
/images/riot-automated-test_file_tree_larger.001.jpeg
What's inside a test class
/images/test_class_anatomy_with_annotations_cropped.jpeg
- (1) Use the ScreenshotUtility interface. Mandatory to use the automatic screenshots functionality.
- (2) Each test class extends the RiotParentTest. This class contains set up and tear down function allowing to start or stop the Appium drivers. To use this functions it's mandatory to use the TestNG @Test annotation using the "groups="" " formula (5).
- (3) Variables shared between tests and set up/tear down functions. In our configuration, we try to have dedicated riot users for each theme. Since our test classes are dedicated to a theme, the same user(s) are used for each test case of the test class. Note that riotUserADisplayName and riotUserBDisplayName are used by the checkIfUserLogged function (12).
- (4) Detailed test steps described in java doc.
- (5) We use the TestNG framework to make assertions and, in this example, make use of very usefull to test features such as groups or priority.
In this examplegroups={"1driver_android","1checkuser"}, "1driver_android" will call a setup method located in RiotParentTest which has the@BeforeGroups(groups="1driver_android")annotation. This method will start an Appium Android driver.
Then, after all the tests wearing the "1driver_android" key are executed, a teardown function in RiotParentTest with the@AfterGroups(groups={"2drivers_android","1driver_android","1driver_ios","2drivers_ios"})annotation will kill the driver(s).
The "priority=0" is used to prioritise test cases execution. - (6) Call a method from utility.HttpsRequestsToMatrix class. The method in this example creates a room with a user A and invites user B, using requests to the homeserver. This method is used to set up the test case, and avoid to create the room using the application under test, which requires time, an allow to focus only on the core of the test.
- (7) Instantiate a POM class, RiotHomePageTabObjects. An android Appium driver (from the appiumFactory singleton class) is passed to the POM constructor class. This driver is directly linked to the device under test.
- (8) Assertion, with the Assert object from TestNG. Validates the test subject.
- (9) Method used by the test to quickly set up the test. See (6).
- (10) This method is executed after each test, thanks to the @AfterMethod TestNG annotation. According to the test name, it will execute the right tear down function.
- (11) Test case tear down function, cleans the test datas created by set up function (6).
- (12) This method is very important to secure the test execution flow. It will make sure that the right user is logged on the right homeserver.
3. Create / maintain tests
Update objects repository
Since we are using the Page Object Model design pattern, editing POM classes is necessary to add references to new objects from Riot GUI, or editing / deleting them.
Objects from the GUI are referenced as MobileElement in POM classes.
Example:
/*
* TOP BAR : contains back button, and confirm button
*/
@AndroidFindBy(id="im.vector.alpha:id/action_bar")
public MobileElement actionBarView;
@AndroidFindBy(xpath="//android.widget.ImageButton[@content-desc='Navigate up']")
public MobileElement backButton;
- @AndroidFindBy is the mechanism used to locate the element. For iOS, we use @iOSFindBy.
- id is the method used to locate the element. More used in Android than iOS. It's the id of the element.
More methods are available:
- accessibility: refers to the accessibility id on iOS elements.
- classname: name of the class
- xpath: path to the objects through the GUI hierarchy. Jokers can be used to bypass elements. Very powerfull but very slow on iOS.
- and 2 more: UIAutomator (deprecated since new iOS automation framework) and tagName.
Adding a new element in a POM class consists in adding a new MobileElement as showed above.
Note that it's possible to directly fetch a list of MobileElements, example:
@iOSFindBy(accessibility="InviteRecentTableViewCell")
public List<MobileElement> invitationCells;
-> Fetch all the elements whose accessibility id is InviteRecentTableViewCell.
How to inspect UI elements from application under test:
Before adding new objects in POM classes, an inspector can be used to take cognisance of objects properties.
Note that if you have access to the application under test source code, inspector is not always mandatory. You can set properties or guess it without using inspector.
- Android
Launch Appium inspector or google UIAutomatorViewer.
UIAutomatorViewer:
/images/google_uiautomator.png
Relations between @AndroidFindBy from POM classes and properties showed by Inspector:
| @AndroidFindBy(method) | Property showed by Inspector | Example |
|---|---|---|
id |
resource-id | @AndroidFindBy(id="im.vector.alpha:id/circular_room_view") |
xpath |
hierarchy + properties | @AndroidFindBy(xpath="//android.widget.RelativeLayout[@resource-id='im.vector.alpha:id/circular_room_view']") |
className |
class | @AndroidFindBy(className="android.widget.RelativeLayout") |
- iOS
Launch Appium inspector or Appium iOS inspector (used in browser).
Appium iOS Inspector: download.
Used only while an iOS test is launched. Tip: set a breakpoint in the testcase, launch the test in debug mode, and double clic on iOS Inspector.html.
/images/ios_appium_inspector.png
Relations between @iOSFindBy from POM classes and properties showed by Inspector:
| @iOSFindBy(method) | Property showed by Inspector | Example |
|---|---|---|
accessibility |
Accessibility Id | @iOSFindBy(accessibility="ContactsTableVCTableView") |
xpath |
hierarchy + properties | @iOSFindBy(xpath="//XCUIElementTypeCell[last()]") |
className |
Type | @iOSFindBy(className="XCUIElementTypeCell") |
Use multiple devices in one test
Some tests need more than one device to be executed on, for example if you want to test a call between 2 devices or if you want to test the typing indicator on the room page.
It means that during the execution, the test will switch between appium drivers (linked to real devices) and will perform actions and verification on them.
One instance of a test can use several devices, from 1 to 4. This project can handle 2 ios devices and 2 android devices, thanks to the utility/AppiumFactory class.
One appium driver is linked to one appium server, because an appium server can handle only one appium driver.
Note that for the moment it's not possible to perform parallel execution on a set of devices.
How to switch between devices:
When instantiating a POM class you have to specify which driver from the AppiumFactory class you want to use, example:
RiotIncomingCallPageObjects incomingCallDevice2= new RiotIncomingCallPageObjects(appiumFactory.getAndroidDriver2());
In this example, the second android driver is used, so all the actions linked to this POM class (RiotIncomingCallPageObjects) will be played on the second android device. By playing actions on a POM class linked to an other driver, you will be able to switch between devices.
Don't forget to set the appropriate group following the set of devices needed in the test, because the appium drivers and appium drivers are instanciated in setUp methods from RiotParentTest class.
Example:
@Test(groups={"2drivers_android","2checkuser"}, description="call from device 1 answered by device 2")
The "2drivers_android" group will start 2 appiums servers and 2 android drivers.
How to link your real devices with the drivers:
To be able to choose which real device will be attached to a driver in the test, you can configure the config/device.yaml file.
The udid is the most important parameter because it identifies the device.
4. Tests execution and tests reports
Execute tests via your IDE
You can execute your tests via your IDE, Eclipse for example, using the TestNG plug-in. You can either execute the all test class or each test individually, by right clicking on the test class or test method, selecting "Run AS" -> "TestNG Test".
Execute tests via maven-surefire-plugin (suitable to continuous integration)
This method use maven to execute tests in command line. There are multiple ways to do it.
It will generate html reports, see here.
/images/maven surefire reports.png
5. Tweaks
In the config/config.yaml file it's possible to configure things such as choosing the use of a custom home server or the handle mode of appium's servers.
Configure the starting appium server mode:
Change the value of starting_server_mode to chose the mode of execution.
- Choose 0 if you want to start appium's servers manually and be able to see the logs on terminal windows.
- Choose 1 or 2 if you want the tests to handle the servers. 2 is the best
Configure the ios development team
Change the value of development_team and code_sign_identity to set what team will be use to start the ios application under test.
Configure the use of a synapse home server
- Change the value of homeserver to set the name of your custom home server.
- Change the value of homeserverlocal to true or false if you want your test to connect to a custom home server or not.
- Change homeserverlocal to true or false if your homeserver has the same IP address of your appium server.
- Change homeserver_address and homeserver_port to make sure will connect to your custom home server.