18 Sep 2018
Belle

Functional UI testing in Exist with EarlGrey and Sencha

I recently wrote about some of the frameworks I've tried for writing iOS tests. For functional UI testing, I've used EarlGrey by Google, and Sencha, which is a sugar wrapper around EarlGrey. Gio mentioned on Twitter that he'd like to see more detail about how I've used these frameworks in my tests, so let's take a look at some real-world examples.

First up, I should explain functional UI testing: although we're testing our app via the UI, and interacting with it as a user would, these tests run as if they were unit tests in Xcode. This means you have access to your app's code, and you can do things like mocking objects and using fake data to test your app, which isn't so easy with ordinary UI tests. I use functional UI tests a lot. I use them even more than unit tests, probably, and I don't use normal UI tests at all.

With EarlGrey and Sencha, these tests are really easy to write, and they're useful for making sure the app looks and behaves as it should.

Testing onboarding and log in

I'll start with a simple example. Exist for iOS requires an account to use the app, so when a user first downloads it they'll see a log in screen. They can log in, after which they should see a series of onboarding screens, or they can tap the link to sign up for a new account. This link first takes the user through a series of welcome screens (like an onboarding, but rather than turning on settings, it explains the app's features), before they get to a sign up screen.

I like to use "given, when, then" as the three steps for my tests, but often the whole "given" step is covered by my setUp method, which is run before each test to set up the environment. Here's the setUp method for my onboarding and log in tests:

 let a = BCAppCoordinator.sharedAppCoordinator
 
 override func setUp() {
        super.setUp()
        
        // turn off google analytics inside EarlGrey/Sencha
        GREYConfiguration.sharedInstance().setValue(false, forConfigKey: kGREYConfigKeyAnalyticsEnabled)
        clearAllTokens()
        
        open(viewController: a, modally: false, embedInNavigation: false)
        a.start(defaults: mockDefaults()!, date: compsForTestDate())
}

I always use the setUp method to make sure analytics is turned off in EarlGrey, so I'm not sending data back to Google. This is obviously optional, but it's turned on by default.

I'm also clearing user tokens from the keychain here, to make sure the keychain is empty before I run a new test.

Finally, I use the Sencha convenience method to open the view controller I want to use, which is my shared app coordinator object. And I call start on the app coordinator, passing it some dummy arguments. Now, onto the tests!

Here's a test to make sure the onboarding screens show up after the user logs in:

func testOnboarding_isTheFirstScreen_afterLogin() {
        // when
        type(text: "TEST_USERNAME", inElementWith: .accessibilityID("log in username field"))
        type(text: "TEST_PASSWORD", inElementWith: .accessibilityID("log in password field"))
        tap(.text("Log in"))
        
        // then
        assertVisible(.text("Track your mood daily with Exist")) // first screen of onboarding is the mood tracking set up screen
}

Since my setUp method has already opened and started my app coordinator, and it cleared the keychain of any tokens, we should be seeing the log in screen at this point, as the app coordinator would have bumped us to the log in screen when it couldn't find a valid user token. If we're not on the log in screen, this test will fail, because step one is to find an element with the accessibilityID of log in username field, which won't be found if we're on the wrong screen.

In this test I'm using Sencha methods that wrap up EarlGrey's more ugly and verbose API. Sencha makes it quick and easy (and more readable) to write these tests. All we do here is type the username and password of a test user into the fields and tap the "Log in" button. I find it quicker and easier to use Sencha's .text matcher than the accessibilityID matcher, which requires me to go add IDs in my production code while I'm writing tests. So even though there's an actual log in button on this screen, I'm just telling Sencha to find the text on the button which says "Log in" and tap on that.

Finally, my assertion is written with Nimble. Nimble make it easy to write concise, readable assertions for your tests. In this case, I use Nimble to make sure the text "Track your mood daily with Exist" is on screen, which means the log in worked, and we've moved to the start of the onboarding screens.

Hopefully you can tell that these tests are really simple and easy to write. Let's look at a couple of other examples.

Here's a super simple test that makes sure the welcome screens are shown if the user taps the sign up link instead of logging in. It uses the same setUp method as above, so all we need to do is find the "Create a new account" button, tap on it, then assert the welcome screen text is visible.

func testWelcome_isShown_afterClickingSignUpLink() {
        
        // when
        tap(.text("Create a new account"))
        
        // then
        assertVisible(.text("Welcome to Exist"))
}

And here's a test where I needed to use EarlGrey directly. Although Sencha supposedly supports scrolling, I've never been able to get it to work, and it doesn't support swiping, so I always use EarlGrey directly for both of those.

This test follows on from the previous one. If the previous test passed, we know the welcome screens show up when the user taps the sign up button on the log in screen. But I want to also make sure that when the user swipes all the way through the welcome screens, they're shown a sign up form.

    func testSignUp_isShown_afterWelcome() {
        
        // when
        tap(.text("Create a new account"))
        EarlGrey.select(elementWithMatcher: grey_text("Track everything together. Understand your behaviour."))
            .perform(grey_swipeFastInDirection(GREYDirection.left))
        EarlGrey.select(elementWithMatcher: grey_text("Exist works with these services and more."))
            .perform(grey_swipeFastInDirection(GREYDirection.left))
        EarlGrey.select(elementWithMatcher: grey_text("Learn what makes you happy and healthy."))
            .perform(grey_swipeFastInDirection(GREYDirection.left))
        EarlGrey.select(elementWithMatcher: grey_text("Get the full picture with mood and custom tags."))
            .perform(grey_swipeFastInDirection(GREYDirection.left))
        tap(.text("Get started"))
        
        // then
        assertVisible(.text("Sign up for Exist"))
}

This test is basically just a series of swipes and taps. First, we tap the "Create a new account" button. This will show the Welcome screens. Then, for each of the Welcome screens, I find some unique text on that screen, and use it to swipe to the left, so the next screen is shown. EarlGrey requires you to select an element before you can perform a swipe on it. I could find the scroll view itself or something else using an accessibilityID but as I said, I like using text matchers better. I also like that I've built extra assertions into this test by using text matchers, because if the text doesn't match on any of these pages, the test will fail, even though the test is technically just for checking what happens after swiping through all the screens.

Anyway, we swipe through each screen, and tap on the text "Get started" on the final screen. Then, we assert that the sign up form's title text is shown. That's it! This kind of test is really fun to watch run in the simulator, because it's kind of like a user using your app really quickly while you watch.

Summary screen tests

Let's take a look at some other functional UI tests. I recently shipped a new screen in Exist for iOS, called the summary screen. It looks like this:

This screen has three types of cards on it: an insights card at the top, following by a goals card, and finally a review card. Each card holds a pager that lets you swipe horizontally through multiple pages, and each one shows a different type of data.

There's a lot of testing needed for a screen like this, with a lot of fake data, to ensure everything shows up correctly. Let's look at a few tests I wrote with EarlGrey and Sencha for this screen.

    func testSummaryScren_showsReviewCard() {
        // given
        let s = summaryScreen()
        open(viewController: s, modally: false, embedInNavigation: true)

        // when
        EarlGrey.select(elementWithMatcher: grey_accessibilityID("SummaryScreenController scrollView"))
            .perform(grey_swipeFastInDirection(GREYDirection.up))

        // then
        assertVisible(.text("Review".uppercased()))
}

This test just makes sure that the review card is showing up. Because this card is last, it tends to be off the screen, so my when step does a fast swipe up (I found this a bit more reliable than trying to scroll to the element itself, though that's supposedly possible) before the then step asserts the review card's title is visible. You can also see here I've once again used Sencha's handy open method to show my view controller.

func testSummaryScreen_withoutInsights_doesntShowInsightsCard() {
        // given
        let s = summaryScreen(includeInsights: false)
        open(viewController: s, modally: false, embedInNavigation: true)
        
        // then
        assertVisible(.text("Goals".uppercased()))
        assertNotVisible(.text("Insights".uppercased()))
}

Because Exist is different for every user, depending on their personal data set, I need to write lots of tests for different combinations of data to test that the app handles them all correctly. For this screen, I need to test that certain cards don't show up at all in some cases. In the test above I'm asserting that when I create the Summary screen without insights data, the insights card doesn't show up at all, with Nimble's assertNotVisible method.

I'm also asserting that the goals card is visible, just to make sure the test doesn't pass due to a quirk where no cards are showing up, or we're on the wrong screen, for example.

    func testSummaryScreen_swipingGoalsCard_showsTheNextPage() {
        // given
        let s = summaryScreen()
        open(viewController: s, modally: false, embedInNavigation: true)
        
        // when
        EarlGrey.select(elementWithMatcher: grey_accessibilityID("SummaryScreenController goals card view"))
            .perform(grey_swipeFastInDirection(GREYDirection.left))
        
        // then
        assertVisible(.text("06:11"))
        assertVisible(.text("01:23"))
}

To test that the pagers inside the cards work, I write tests like this one, where I use EarlGrey to find the card and swipe it to the left. Then I assert that the data showing on the card is from the second page. I use fake data for these tests so I can know exactly what to expect in my assertions.

---

Hopefully that gives you an idea of how EarlGrey and Sencha can help you write functional UI tests. There's a lot more EarlGrey can do, and you can even write custom matchers if it doesn't support your specific needs.