Overview of the Language
Overview of the Concordia declarations:

A
Feature
defines a desired behavior for the software, a piece of functionality that gives some business value. A Scenario
is a high-level usage scenario of a Feature
. A Feature can have many Scenarios.Since
Feature
and Scenario
are business-oriented declarations, they are not used by Concordia Compiler to generate test cases. You can write them freely. Although, we recommend you to use the User Story template for a Feature
and the Given-When-Then template for a Scenario
. Example:Feature: Sales Report
As a manager
I would like to generate a sales report
In order to track company sales
Scenario: Daily sales report
Given that I am authenticated as a manager
When I open the sales report
and I choose the option "Daily report"
Then I see a list with all the today's sales
and the total value
As a <role>
I would like to <goal to perform>
In order to <received benefit>
(orSo that <received benefit>
)
Given <some context or precondition>
When <some action is carried out>
Then <a result or postcondition is observed>
In the GWT template, additional sentences of each kind are written in the next line and receive the prefix
and
.Features and Scenarios must have a name, but their description are not required. Thus, you can write the above example like this (although we recommend against):
Feature: Sales Report
Scenario: Daily sales report
Feature files receive the extension
.feature
. Only one feature per file is allowed.A
Variant
is a test-oriented declaration that describes a possible interaction between a user and the system in order to accomplish a Scenario
. You can write more than one Variant
for a same Scenario
.A
Variant
must use the GWT template and first person singular ("I
") in its sentences. Concordia Compiler uses Natural Language Processing (NLP) techniques to understand Variant
sentences, aiming to transform them into test cases. Example:Feature: New User
Scenario: Successful registration
Variant: Register with a new account
Given that I am on "https://example.com/register"
When I fill <#name> with "Bob"
and I fill <#dateOfBirth> with "11/30/1990"
and I enter with "[email protected]" in <#email>
and I inform "m123456" in <#pass>
and I type "m123456" in <#confirmation>
and I click <#ok>
Then I see "Welcome, Bob"
Concordia Compiler understands the sentences from line
7
to11
as being variations of the same action, fill
.Of course, the vocabulary is limited. Although, you can extend it whether you need (by adding new synonyms to a specific dictionary file). Actions and common variations are documented and suggestions are always welcomed.
In the sentences above, the values were denoted between quotation marks, e.g.
"[email protected]"
. Numeric values are also accepted and do not need quotation marks. Example: 97
.Widgets (of the user interface) are written between
<
and >
. Example: <#email>
. Concordia adopts the well-known CSS query selectors to locate widgets, regardless the user interface (its plugins convert them to the format adopted by the testing framework). You can also use XPath (although some plug-ins may not accept it). Examples:#
(hashtag) to find a widget by its id. Example:#email
.
(dot) to find a widget by its class, if applicable. Example:.colorful
@
(at) to find a widget by its name. Example:@pass
~
(tilde) to find a widget by its mobile name, if applicable. Example:~pass
- no prefixed characters, to find a widget by its type. Example:
input
//
(double slash) to find a widget by its XPath. Example://table/tbody/tr[1]
Whether you are testing a web application, you may find Katalon-Concordia useful. It's a browser extension - available for Google Chrome and Mozilla Firefox - that extends Katalon Recorder to save a recorded interaction as
Variant
sentences. Using it may reduce the time needed to locate the widgets of an existing application. A State is denoted by a text between tilde (
~
), such as ~a state~
, and must be written inside Variant sentences only. Example:Then I have a ~user logged in~
A Variant can both produce or require states. A State is produced by a
Then
sentence, like in the prior example. A State is required by a Given
or a When
sentence. For instance:Given that I have ~user logged in~
By declaring a required State, you're establishing a dependency among the current Variant and the ones that can produce it. Since that State is often produced by a Variant from another Feature, you have to import the corresponding Feature file. Concordia Compiler will only search for states from imported Feature files.
- Remove any (
Then
) sentences with a produced State; and - Replace any (
Given
orWhen
) sentences with a required State by sentences from the Variant that can produce it.
Whether there are different Variants that can produce a (same) required State, Concordia Compiler will be able to combine each of them with the current Variant, aiming to produce Test Cases that explore different execution paths. For more information, please read the section State Based-Combination.
Let's say that we have a simple Login feature like this:
login.feature
Feature: Login
Scenario: Sucessful login
Variant: Login with username and password
Given that I am on the [Login Screen]
When I enter with "bob" in {Username}
and I enter with "123456" in {Password}
and I click on {OK}
Then I see "Welcome"
and I have ~user logged in~
Constants:
- "Login Screen" is "/page"
UI Element: Username
UI Element: Password
UI Element: OK
- type is button
Now let's suppose that a user has to be logged in to access its account details:
my-account.feature
import "login.feature"
Feature: My Account
Scenario: See my account data
Variant: Show basic data by default
Given that I have ~user logged in~
and I am on the [Account Screen]
Then I see {User} with "bob"
and I see {Name} with "Robert Downey Jr"
Constants:
- "Account Screen" is "/account"
UI Element: User
Concordia Compiler will produce
my-account.testcase
with a Test Case like the following, that includes steps from both the features:import "my-account.feature"
@scenario(1)
@variant(1)
Test Case: See my account data - 1
Given that I am on the "/login"
When I enter with "bob" in <username>
and I enter with "123456" in <password>
and I click on <ok>
Then I see "Welcome"
Given that I am on the "/account
Then I see <user> with "bob"
and I see <name> with "Robert Downey Jr"
Concordia Compiler can optimize the combination among Features in order to reduce the amount of produced test cases and, therefore, the total execution time. You can also opt to get full coverage - which is ideal before releasing a feature.
A
Test Case
represents a test case produced for a certain Variant
. Concordia Compiler can generate many Test Case
declarations from a same Variant
, each one covering different states, input combinations and oracles. The generation considers the following declarations:- Variant
- states
- UI Element
- Constants
- Table
- Database
The simplest
Test Case
is one whose sentences are exactly equal to the Variant
's. That's the case when the Variant does not use other declarations and it explicits all the input test data to use.For comparison, whether we simply remove
with "Bob"
from the sentenceWhen I fill <#name> with "Bob"
, Concordia Compiler will generate a pseudo-random value to complete the sentence, and it will be like this: When I fill <#name> with "A~!39esljdns so2u1 *ns%"
.Concordia Compiler uses a single seed to generate pseudo-random numbers in its algorithms and values. Everytime it runs, a new seed is produced. You can set the seed using the CLI parameter
--seed
. By setting the seed, the compiler will pick the same paths and produce the same values - which is desirable for reproducing the same behavior in the application under test, but not for discovering new defects.Don't worry about writing Test Cases. Write Variants and let Concordia generates the Test Cases for you.
A
UI Element
represents a user interface element and can denote its expected behavior through properties.Although a Widget is easy to declare, it can have some shortcomings:
1) It can make a sentence hard to read, specially when it uses XPath or CSS locators.
For example, the sentence
When I click on <//*[@id="section1"]/div[1]/div[2]>
does inform what the action really means. The same sentence with a UI Element reference becomes
When I click on {Continue}
Then the UI Element can be declared as:
UI Element: Continue
- locator: //*[@id="section1"]/div[1]/div[2]
Here
locator
is a property that denotes how to find it in the UI.2. It can be hard to maintain. When a Widget appears in more than one Scenario and its locator needs to change (e.g. you have changed it in the UI), you need to search and replace all its occurrences in the document. With a UI Element, you just need to change the property
locator
.3. It does not offer resources for test generation other than its locator.
A
UI Element
offers properties that can describe the behavior related to it. These properties are used to infer the test data and test oracles to generate.Example:
UI Element: Net Price
- minimum value is 10.00
Otherwise I see "Net Price must be greater than U$ 10"
Concordia Compiler infers
6
possible test cases from the above declaration, applying different testing techniques.These are the UI Element properties:
id
orlocator
⇨ how to locate the widgettype
⇨ widget type, liketextbox
,button
, etc.editable
⇨ whether the UI Element is editable (or not)data type
⇨ data type of aneditable
UI Elementrequired
⇨ whether the UI Element is required (or not)format
⇨ data format, denoted by a regular expressionvalue
⇨ defines how the value is producedminimum value
⇨ defines how the minimum value is producedmaximum value
⇨ defines how the maximum value is producedminimum length
⇨ defines how the minimum length is producedmaximum length
⇨ defines how the maximum length is produced
By default, declaring a UI Element without properties is the same as declaring it with:
id
orlocator
equal to the UI Element name in camelCase (e.g., "Net Price" becomes "netPrice"). The adopted case is configurable.type
equal totextbox
data type
equal tostring
Concordia Compiler is able to guess some properties from the others. For instance,
editable
is enabled when type
is a widget capable of receiving input data; data type
can be inferred from any attributed value; etc.Another example:
UI Element: Profession
- value is in [ "Programmer", "Tester", "Analyst" ]
Concordia Compiler infers
4
possible test cases from the above declaration, applying different testing techniques.Please see the language reference for more details. Let's proceed with the other declarations for now.
A
Import
declaration allows you to import declarations from another .feature
file. Example:Import "login.feature"
Imported declarations are Feature, Constants, Table, Database, and UI Element.
A
Constants
block defines one or more constants. Every Constant holds an immutable value that is accessible through the declared name. Example:Constants:
- "Register Page" is "https://example.com/register"
- "Coupon Number" is 12345
Constants are accessed by their name between
[
and ]
. Example: Given that I am on the [Register Page]
When I fill {Coupon} with [Coupon Number]
Constants are global declarations and cannot have duplicate names. You can use them in
UI Element
properties and Variant
sentences.A
Table
declares values in rows and columns to use in UI Element properties. Example:Table: Customer
| name | age |
| Ada Lovelace | 36 |
| Dennis Ritchie | 70 |
| Donald Knuth | 82 |
| George Boole | 49 |
| Niklaus Wirth | 83 |
UI Element: Name
- value comes from "SELECT name FROM [Customer]"
Otherwise I see "Customer not found."
Concordia Compiler infers
4
possible test cases from the above declaration, applying different testing techniques.Note that SQL is used to select values from the declared table, which is denoted between
[
and ]
(like Constants). Table names are global and share the same namespace as Constant names.Database: AcmeDB
- type is "mysql"
- name is "acme"
- host is "http://127.0.0.1"
- username is "tester"
- password is "testing123"
UI Element: Name
- value comes from "SELECT name FROM [AcmeDB.customer]"
Otherwise I see "Customer not found."
Concordia Compiler infers
4
possible test cases from the above declaration, applying different testing techniques.SQL is used to select values from a table or view from the declared database, which is denoted between
[
and ]
. Its table or view names are separated by a dot (.
). Database names are global and share the same namespace as Tables and Constants.In order to set the execution environment up, testers can define events that can run database commands or command-line scripts. These events can occur before or after Scenarios or Features:
Before Each Scenario
: executes before every test of each Scenario of the current Feature.After Each Scenario
: executes after every test of each Scenario of the current Feature.Before Feature
: executes once, before all the tests of the current Feature.After Feature
: executes once, after all the tests of the current Feature.Before All
: executes once, before all the tests of the application under test.After All
: executes once, after all the tests of the application under test.
Test Events are written like Variants, using Given-When-Then. Since they usually perform actions, it is very likely to use
When
in most sentences. Example:Before Feature:
Given that I connect to [AcmeDB]
When I run the script 'DELETE FROM customer'
and I run the script 'INSERT INTO customer ( name, age ) VALUES ( "Alice", 20 ), ( "Bob", 30 )'
and I run the command 'rmdir log'
Commands and database scripts are denoted between apostrophe (
'
).Last modified 2yr ago