What Do I Need to Start With Xamarin Test Cloud?
This is going to be a rather short paragraph. Without further ado:
Before you proceed to testing, let’s cover the basics.
Xamarin Test Cloud is a cloud-based service that provides an automatable way for UI Acceptance Testing of mobile apps. You’ll find thousands of various devices within the cloud. As it happens usually, different functionalities of an application will be checked by the software.
It is not a mystery that manual tests are expensive. Sometimes, there are just too many features for manual testing—you might have no choice but to automate. That’s where automated UI tests step in.
The Xamarin UITest is a framework based on the popular NUnit testing library, which enables developers to write tests in C#.
It is especially recommended for developers who have previous experience with automating their tests. To read more about NUnit library, please refer to this link.
Calabash is a framework that enables developers to write their tests in Ruby, using the Cucumber tool. These tests are very close to behavior-driven development methodology. The big advantage of using this framework is the ability to write the tests in a business language. Literally—anyone following grammar rules imposed by the Cucumber tool can write the tests.
The decision about which framework you should use is up to you—both of them will test the proper behavior of the application.
Also, remember that they’re based on Calabash.
Simple architecture and flow of Xamarin Test Cloud testing with listed actions:1. Developer creates an app package with tests,
2. The app package is uploaded to the Xamarin Test Cloud,
3. Xamarin Test Cloud installs the app on its test devices,
4. Once the tests are finished, Xamarin Test Cloud prepares a report about the tests.
Developers can submit app packages with tests manually. With larger apps, they can also use continuous integration:
Here’s how it goes:1. Developer commits changes to the repository,
2. Build server gets the latest committed version and an app package,
3. Application package with tests is uploaded to Xamarin Test Cloud,
4. Xamarin Test Cloud tests the app on different devices,
5. Report with results is sent to developer.
Test Cloud Frameworks. How Do They Work?
The frameworks mentioned above (UITest or Calabash) test the mobile app functionality, but they do not interact with the app directly. For that, you need something called Xamarin Test Cloud Agent.
And so, Xamarin Test Cloud Agent is responsible for running tests on the devices. It’s a separate app installed by Xamarin Test Cloud alongside your app.
Calabash and UITest frameworks use a client-server architecture:
- Test Cloud Server Application—a lightweight server app runs on the device which listens for clients to initiate a selected test. It communicates over HTTP via JSON.
- Test Cloud Agent—responsible for providing information for the server about which the test should be executed.
Xamarin Test Cloud Agent on Android
Test Cloud Server Application runs as a separate application with permission to cooperate with the actual application for tests.
Such a scenario is possible because Test Cloud signs both apps with the same key.
You do not have to add any additional references in your code to make it run.
On iOS, Test Cloud looks different. Test Cloud Server Application is built into the tested application—but in order to achieve that, you have add a NuGet package (Xamarin Test Cloud Agent).
The agent should be included in debug mode only. When you proceed to releasing your app, it should be removed by Xamarin.iOS Linker.
Run Tests in Xamarin Test Cloud
By now, we’ve talked about the testing framework structure and how it works. When you upload your application to the Test Cloud, it hosts the tests and runs them on different devices (user does not have to run tests manually from the command prompt):
Calabash or UITest—Which Framework Should I Choose?
There is no one good answer for that—you will achieve what you have planned with both of them.
The diagram below, provided by Xamarin, should help you choose the right way:
If you have your application already written in C# (Xamarin), you probably should choose UITest framework because you are familiar with the language.
What’s more, you can run all tests from the IDE (Visual Studio or Xamarin Studio). UITest is based on the popular NUnit framework, so there is also big support from the community.
For native applications (Java, Swift, Obj-C, or hybrid apps), Calabash framework will be a good choice. You can create cross-platform tests in any text editor (Ruby IDE, too) without any special tools.
What Kind of Tests Do We Have in Mobile Development?
Let’s start from the bottom.
- Unit Tests—this is the type of test that is the most costly. The purpose of it is to test a small unit of the application (often this is a method), isolating it and verifying if it’s working properly.
- Components Tests —this is the type of test that verifies if components (like classes or objects) are working properly. For example, testing the whole functionality of “Car” class is called a component test.
- Integration Tests—the purpose of this type of test is to check proper integration between classes—for instance if class “Car” behaves properly with “Tire” class.
- Acceptance Tests—these tests are carried out to validate the functionality of the application. They just check if the app functionality meets the expectations related to the functionality agreed at the beginning of development.
- UI Tests—these tests are very often done by a quality assurance (QA) team manually. They check if the user interface is implemented correctly (by clicking buttons, testing different flows). This is something that can be done with Xamarin Test Cloud.
The last type of test is called a Regression Test, and it is not included in the picture. This type of test is being used to check if our current code works with the previous versions (for instance, if newly a written class works properly with others).
Now you know what the possible test frameworks are and how tests are performed on Android and iOS.
Long story short, I described the kinds of tests that are available with reference to mobile development.
Now, let’s dig deeper into how to actually write the tests and run them with Xamarin Test Cloud to create automated UI Tests.
How to Create Automated UI Tests with Xamarin Test Cloud