Calabash iOS Tutorial

One of my favorite part of Ruby on Rails was how easy it was to do Test Driven Development (TDD). I LOVE the confident feeling that comes with making HUGE changes to my code base when my project is well-tested.

In fact, my favorite days coding include committing a branch where I got to delete 1000+ lines of code only to replace it all with 200 lines – this type of coding becomes pretty worrisome when there are no tests there to help you see if you messed up an important edge case.

Unfortunately, with iOS, it has not been as easy to learn testing and to actually do it. I’ve fallen out of the TDD habit. Recently, Apple has made some huge strides in the testing arena – XCTest is really nice, and iOS 8 will have even better testing tools (including performance testing).

So in addition to XCTest, I’ve been researching iOS integration testing tools – mainly Calabash iOS and KIF.

From my research, I’ve found that Calabash is great to use if you have a QA person there to write the tests for you, while KIF is really easy to get started with if the iOS developers are expected to write the integration tests. In my personal projects, I’m going to use KIF, but I learned a lot about Calabash, and I’d like to share my learnings here while they’re still fresh in my mind.

The Setup

To get started with Calabash iOS, just follow the straight-forward instructions in the README guide. If you’re having issues installing the calabash-ios ruby gem, check out the XCode 5.1 specific installation instructions in the Calabash Wiki here. In general, I highly recommend going through the Calabash Wiki – it has a ton of useful information you’ll need to look up over and over again.

Once you’ve set up your Calabash, the last step is to change into your project folder in your Terminal and type in “cucumber”. All the pre-set tests should pass with green colors.

For this tutorial, I’m going to use a simple Calculator app as an example. I searched Github for a calculator app that works, and found one by Oscar Del Ben called iPhoneCaculator.

I forked it, modified it slightly, and added Calabash tests. You can see my iPhoneCalculator Fork on Github here. Here is what the Calculator looks like (it’s an old project, so it only works on a 3.5 inch screen):

iPhone Calculator

Accessibility Labels

One of the most important ways Calabash finds different views on the screen is through the use of accessibility labels. So for each button on the Calculator, I added an accessibility label of it’s text. For example, button with the number 3 on it has the accessibility label of 3. For the result label (where the result of the calculation is displayed), I added the accessibility label called “result”.

Each view has a accessibilityLabel property, so it’s really easy to set!

To view the accessibility labels for each view, open up your iPhone Simulator, go to the Settings app, go to General, go to Accessibility, turn on the Accessibility Inspector. You’ll see a purplish Accessibility Inspector pop up:

iPhone Simulator Accessibility Inspector

To use the Accessibility Inspector, click on the X to expand it, then click on any view, and it’ll give you the Accessibility details for that view.

iPhone Simulator Accessibility Inspector Open Example

Click on the X again to close the Accessibility Inspector to navigate through your apps normally.

Feature Statement

First, rename your my_first.feature file to the name of your feature – in my case calculate.feature, and update your Feature statement. The Feature statement is never run, so it can be anything, but keep it to the format of a user story: As a… I want to… So I can… 

My calculator app’s feature statement is as follows:

# calculate.feature
Feature: Calculate numbers
  As a calculator user
  I want to compute numbers accurately
  So I can go back to business fast

Quick Start

Calabash iOS comes with some very easy Predefined Steps, so you can get started very quickly. So, to add two numbers, this is all I have to do:

Scenario: Add two numbers
  Given I am on the Welcome Screen
  When I touch the "2" button
  And I touch the "+" button
  And I touch the "2" button
  Then I see the text "4"

Now, when I run “cucumber” in my terminal, this test will pass. Change the 4 to 5 in the last sentence and run the tests again to make sure they fail! Now change back to 4. Remember, that in TDD, seeing the Red is just as important as seeing the Green.

Keep in mind, however, that the predefined steps are meant to get you started quickly. It is not best practice to actually use them for your whole project. This is what it says in the Predefined Steps Wiki:

“Please note that pre-defined steps is intended for people to get started quickly, and is not considered best practice.”

Hello Ruby

The point of Calabash is to be a very readable requirements testing framework. Ideally, the business person will tell you what each feature scenario is, and all you have to do is write the code for it.

If you think about it, predefined steps are not too readable – I click this, then I click that, then I click the other, then I see a result. Compare what we had in the previous predefined steps addition example to something like this:

Scenario: Multiply two numbers
  Given the input "2*2"
  When the calculator is run
  Then the output should be "4"

Calabash, when combined with some basic ruby, is very powerful for creating very easy to read and straight-forward tests.

All you have to do is match the regular expression of each sentence in a scenario, and write what happens. Here are my step definitions for the above example:

# you can be more specific with your regular expression if needed
Given(/^the input "(.*?)"$/) do |expression|
    
    # the expression in this case is "2*2"
    # so it is split into an array of components [2, *, 2]
    inputs = expression.split(//)
    
    # then each array component is tapped
    inputs.each {|input| tap input}
end


When(/^the calculator is run$/) do
    tap "="
end


Then(/^the output should be "(.*?)"$/) do |expected_output|
    # here we find all the labels with accessibility label "result",
    # get the first (and only) result label that is returned
    result_label = query("label marked:'result'")[0]
    
    # get the text value of the result label
    actual_output = result_label["text"]
    
    # compare the expected output (4 in this case)
    # with the text in the result label
    assert actual_output.eql? expected_output
end

Since the regular expressions and steps in my ruby code are pretty generic, I can now easily write additional scenarios in my feature file to make various calculations. Both of these will work, for example:

Scenario: Divide two numbers
  Given the input "2/2"
  When the calculator is run
  Then the output should be "1"
Scenario: Subtract two numbers
  Given the input "25-5"
  When the calculator is run
  Then the output should be "20"

A more complex expression will work as well (although the calculator does not give the correct result, so the test will be red, indicating the calculator needs to be re-written if this is a real user scenario):

Scenario: Do a complex calculation
  Given the input "25+5*3"
  When the calculator is run
  Then the output should be "40"

Refactor

As you’ve probably noticed – the above feature scenarios are sort of repetitive. If you want to test more types of equations (like getting a negative result, for example), you not have to repeat yourself again. And what you decide to rewrite one of the step definitions? You’ll have to update every single scenario! So you know it’s time to refactor.

One cool feature of Calabash are the Scenario Outlines, which would be perfect for this exact use case. You can condense all the feature scenarios from the last section into one very straight forward and easy to modify scenario outline:

Scenario Outline: Caculate Expressions
    Given the table input <Input>
    When the calculator is run
    Then the output from table should be <Output>
    Examples:
      | Input   | Output |
      | "3+5"   | 8      |
      | "25-4"  | 21     |
      | "10/2"  | 5      |
      | "25+5*3"| 40     |
      | "4-5"   | -1     |

Note how you can keep adding even more equations to this scenario outline without writing additional scenarios!

The first two step definitions are the same as in the previous examples, and the last step is only slightly different, since the regular expression will expect a digit as an output vs a string:

Given(/^the input "(.*?)"$/) do |expression|
    # so it is split into an array of components
    inputs = expression.split(//)
    
    # then each array component is tapped
    inputs.each {|input| tap input}
end


When(/^the calculator is run$/) do
    tap "="
end

# note that the regular expression now matches
# a positive or negative digit specifically
Then(/^the output from table should be (-?\d+)$/) do |expected_output|
    # here we find all the labels with accessibility label "result",
    # get the first (and only) result label that is returned
    result_label = query("label marked:'result'")[0]
    
    # get the text value of the result label
    actual_output = result_label["text"]
    
    # compare the expected output (4 in this case)
    # with the text in the result label
    assert actual_output.eql? expected_output
end

Tags

As your scenario list grows, you might want to run only one at a time as you’re working on it. To do this, simple add an @ tag before each scenario:

@multiply
Scenario: Multiply two numbers
  Given the input "2*2"
  When the calculator is run
  Then the output should be "4"

So now, simply use in the –tags (or -t) option with cucumber in your terminal to run only the multiply scenario:

$cucumber --tags @multiply

You can see all other shortcuts and options if you run “cucumber –help” in your terminal. Enjoy!

Advanced

To really understand the philosophy behind Cucumber (which powers Calabash), I highly recommend reading The Cucumber Book: Behaviour-Driven Development for Testers and Developers.

Do you have additional Cucumber tips and tricks? Let me know in the comments!

You can view the full source code for the calculator app on Github here.

Enjoy the article? Join over 14,500+ Swift developers and enthusiasts who get my weekly updates.

  • Pihu

    Thanks for the tutorial. I am not able to understand couple of concept which are as follows:
    1. We are using cucumber to create feature and step definition.Later we use ruby or any other tools like Robotium or Selenium (in case of android) to perform the action. If we take example of android testing with Calabash, we can achieve same with the cucumber and robotium. Why we need calabash.
    2. Can Calabash work without cucumber?
    3. Why we can not go with the cucumber and ruby in the above case.