A Small Example
This example covers the generation of test data and oracles and the combination of features using states.
Let's suppose that we have to create two simple features for a web application: "Login" and "Add Product". After discussing them with stakeholders and the team, we created the following Concordia specification (see the two tabs):
login.feature
add-product.feature
1
Feature: Login
2
As a user
3
I would like to authenticate myself
4
In order to access the application
5
6
Scenario: Successful login
7
Given that I can see the login screen
8
When I enter with valid credentials
9
Then I can access the application's main screen
10
11
Variant: Login with username and password
12
Given that I visit the [Login Screen]
13
When I fill {Username}
14
And I fill {Password}
15
And I click on {OK}
16
Then I see "Welcome"
17
And I have a ~user logged in~
18
19
Table: Users
20
| username | password |
21
| bob | 123456 |
22
| alice | 4l1c3pass |
23
24
UI Element: Username
25
- required
26
Otherwise I must see "Please inform the username."
27
- value comes from "SELECT username FROM [Users]"
28
Otherwise I must see "Invalid username."
29
30
UI Element: Password
31
- required
32
Otherwise I must see "Please inform the password."
33
- value comes from "SELECT password FROM [Users] WHERE username = {Username}"
34
Otherwise I must see "Invalid password."
35
36
UI Element: OK
37
- type is button
38
39
Constants:
40
- "Login Screen" is "/login"
41
Copied!
1
import "login.feature"
2
3
Feature: Add Product
4
As an employee
5
I would like to add a product
6
In order to manage its data and sale it
7
8
Scenario: Add a new product with basic data
9
Given that I am logged in
10
When I add a product with SKU, description, department, price, and quantity
11
Then I am able to register it
12
13
Variant: Basic data
14
Given that I have a ~user logged in~
15
and I visit the [Product Screen]
16
When I fill {SKU}
17
and I fill {Description}
18
and I fill {Price}
19
and I fill {Quantity}
20
and I click on {Save}
21
Then I have ~product registered~
22
and I see "Saved."
23
24
UI Element: SKU
25
- required
26
Otherwise I see "Please inform the SKU."
27
- format is "/[A-Z]{3}\-[0-9]{3}/"
28
Otherwise I see "Invalid SKU format. Please use 3 letters, dash, 3 numbers."
29
30
UI Element: Description
31
- minimum length is 2
32
Otherwise I see "Description needs at least 2 characters."
33
- maximum length is 100
34
Otherwise I see "Description must have at most 100 characters."
35
36
UI Element: Price
37
- required
38
Otherwise I see "Please inform the price."
39
- minimum value is 0.01
40
Otherwise I see "Minimum price is one cent."
41
42
UI Element: Quantity
43
- data type is integer
44
45
UI Element: Save
46
- type is button
47
48
Constants:
49
"Product Screen" is "/products/new"
50
Copied!
These features could correspond to the following sketches:
There is an experimental application that can generate User Interface Prototypes (UIP) from a Concordia specification. The current version has a plug-in to create HTML-based UIPs and it's really easy to adapt it for other technologies.
The specification helps stakeholders, analysis, testers and developers to create a shared understanding about the application and its business rules. It helps with the requirements validation, knowledge documentation, project design, implementation and testing.
Different from Use Cases and other requirements specification formats, with Concordia you probably don't need to define different scenarios for validation. Instead of having to manually define a set scenarios for validation, you just need to define how the UI elements should behavior - in their declaration - and the compiler will generate different scenarios for you in the form of Test Cases. Whether your application has a lot of such rules, your team will certainly benefit from it.
<TO-DO (UNDER CONSTRUCTION)>
Remember that a Variant express a possible interaction between a user and the application in order to complete its (business-focused) Scenario. lt follows the GWT syntax and always uses the word "I" to represent the user or user role denoted in the Feature's description.
Let's run Concordia Compiler with a seed to produce the same results over and over again:
1
npx concordia --seed="example" --no-run --no-result
Copied!
It will generate login.testcase with the following content:
login.testcase
add-employee.testcase
1
<TO-DO (UNDER CONSTRUCTION)>
Copied!
1
<TO-DO (UNDER CONSTRUCTION)>
Copied!
If a seed is not given, Concordia Compiler assumes the current date and time. The seed is always printed in the console so that you or your team can reproduce the same paths, test data, and test oracle that were able to expose a bug in your application. Here we are giving it for a sake of reproduction.
Using the same seed over and over again will make Concordia Compiler produce the same results. Options no-run and no-result avoid running the test scripts and getting execution results.

Execution without generation

This will not generate test cases or test scripts, but it will execute them and get their results:
1
$ concordia --plugin=codeceptjs --no-test-case --no-script
Copied!

Output

login.testcase:
1
# Generated with ❤ by Concordia
2
#
3
# THIS IS A GENERATED FILE - MODIFICATIONS CAN BE LOST !
4
5
import "login-en.feature"
6
7
@generated
8
@scenario(1)
9
@variant(1)
10
Test case: Successful login with valid credentials - 1
11
Given that I am in the "http://localhost/login" # [Login Screen]
12
When i fill <username> with "*RM)O," # invalid: inexistent element
13
And i fill <password> with "" # valid: last element
14
And I click on <ok> # {OK}
15
Then I must see "Invalid username" # from <username>
16
17
@generated
18
@scenario(1)
19
@variant(1)
20
Test case: Successful login with valid credentials - 2
21
Given that I am in the "http://localhost/login" # [Login Screen]
22
When i fill <username> with "bob" # valid: filled
23
And i fill <password> with -8655972838932479 # invalid: inexistent element
24
And I click on <ok> # {OK}
25
Then I must see "Invalid password" # from <password>
26
27
@generated
28
@scenario(1)
29
@variant(1)
30
Test case: Successful login with valid credentials - 3
31
Given that I am in the "http://localhost/login" # [Login Screen]
32
When i fill <username> with "alice" # valid: random element
33
And i fill <password> with "4l1c3pass" # valid: random element
34
And I click on <ok> # {OK}
35
Then I see "Welcome"
36
37
@generated
38
@scenario(1)
39
@variant(1)
40
Test case: Successful login with valid credentials - 4
41
Given that I am in the "http://localhost/login" # [Login Screen]
42
When i fill <username> with "alice" # valid: last element
43
And i fill <password> with "4l1c3pass" # valid: filled
44
And I click on <ok> # {OK}
45
Then I see "Welcome"
46
47
@generated
48
@fail
49
@scenario(1)
50
@variant(1)
51
Test case: Successful login with valid credentials - 5
52
Given that I am in the "http://localhost/login" # [Login Screen]
53
When i fill <username> with "" # invalid: not filled
54
And i fill <password> with "" # invalid: not filled
55
And I click on <ok> # {OK}
56
Then I see "Welcome"
57
58
@generated
59
@scenario(1)
60
@variant(1)
61
Test case: Successful login with valid credentials - 6
62
Given that I am in the "http://localhost/login" # [Login Screen]
63
When i fill <username> with "bob" # valid: first element
64
And i fill <password> with 123456 # valid: first element
65
And I click on <ok> # {OK}
66
Then I see "Welcome"
Copied!
test/login.js:
1
// Generated with ❤ by Concordia
2
// source: c:\code\tmp\concordia-test-pt\login-en.testcase
3
//
4
// THIS IS A GENERATED FILE - MODIFICATIONS CAN BE LOST !
5
6
Feature("Login");
7
8
Scenario("Successful login | Successful login with valid credentials - 1", (I) => {
9
I.amOnPage("http://localhost/login"); // (61,5) [Login Screen]
10
I.fillField("username", "*RMO,"); // (11,5) invalid: inexistent element
11
I.fillField("password", ""); // (12,7) valid: last element
12
I.click("ok"); // (64,7) {OK}
13
I.see("Invalid username"); // (14,5) from <username>
14
});
15
16
Scenario("Successful login | Successful login with valid credentials - 2", (I) => {
17
I.amOnPage("http://localhost/login"); // (61,5) [Login Screen]
18
I.fillField("username", "bob"); // (21,5) valid: filled
19
I.fillField("password", "-8655972838932479"); // (22,7) invalid: inexistent element
20
I.click("ok"); // (64,7) {OK}
21
I.see("Invalid password"); // (24,5) from <password>
22
});
23
24
Scenario("Successful login | Successful login with valid credentials - 3", (I) => {
25
I.amOnPage("http://localhost/login"); // (61,5) [Login Screen]
26
I.fillField("username", "alice"); // (31,5) valid: random element
27
I.fillField("password", "4l1c3pass"); // (32,7) valid: random element
28
I.click("ok"); // (64,7) {OK}
29
I.see("Welcome"); // (65,5)
30
});
31
32
Scenario("Successful login | Successful login with valid credentials - 4", (I) => {
33
I.amOnPage("http://localhost/login"); // (61,5) [Login Screen]
34
I.fillField("username", "alice"); // (41,5) valid: last element
35
I.fillField("password", "4l1c3pass"); // (42,7) valid: filled
36
I.click("ok"); // (64,7) {OK}
37
I.see("Welcome"); // (65,5)
38
});
39
40
Scenario("Successful login | Successful login with valid credentials - 5", (I) => {
41
I.amOnPage("http://localhost/login"); // (61,5) [Login Screen]
42
I.fillField("username", ""); // (52,5) invalid: not filled
43
I.fillField("password", ""); // (53,7) invalid: not filled
44
I.click("ok"); // (64,7) {OK}
45
I.see("Welcome"); // (65,5)
46
});
47
48
Scenario("Successful login | Successful login with valid credentials - 6", (I) => {
49
I.amOnPage("http://localhost/login"); // (61,5) [Login Screen]
50
I.fillField("username", "bob"); // (62,5) valid: first element
51
I.fillField("password", "123456"); // (63,7) valid: first element
52
I.click("ok"); // (64,7) {OK}
53
I.see("Welcome"); // (65,5)
54
});
Copied!

3. Analyze the results

Whether you ran the test scripts above, you probably saw they fail. That's because they didn't find a web application running at http://localhost/login or because the application was found but it did not match the expected behavior. You may adapt your application and run the tests again.
Concordia shows a report that indicates the failures' locations. They help you to decide if a failure was caused by the application under test (e.g., it did not behave as expected) or because of the requirements specification (e.g., it is outdated in relation to the application).
Now keeping your specification updated has a new clear benefit: you can use it to generate tests and discover existing defects in your application!

See also

Last modified 7mo ago
Export as PDF
Copy link