A Small Example
This example covers the generation of test data and oracles and the combination of features using states.
Let's suppose that we have to create two simple features for a web application: "Login" and "Add Product". After discussing them with stakeholders and the team, we created the following Concordia specification (see the two tabs):
login.feature
add-product.feature
Feature: Login
As a user
I would like to authenticate myself
In order to access the application
Scenario: Successful login
Given that I can see the login screen
When I enter with valid credentials
Then I can access the application's main screen
Variant: Login with username and password
Given that I visit the [Login Screen]
When I fill {Username}
And I fill {Password}
And I click on {OK}
Then I see "Welcome"
And I have a ~user logged in~
Table: Users
| username | password |
| bob | 123456 |
| alice | 4l1c3pass |
UI Element: Username
- required
Otherwise I must see "Please inform the username."
- value comes from "SELECT username FROM [Users]"
Otherwise I must see "Invalid username."
UI Element: Password
- required
Otherwise I must see "Please inform the password."
- value comes from "SELECT password FROM [Users] WHERE username = {Username}"
Otherwise I must see "Invalid password."
UI Element: OK
- type is button
Constants:
- "Login Screen" is "/login"
import "login.feature"
Feature: Add Product
As an employee
I would like to add a product
In order to manage its data and sale it
Scenario: Add a new product with basic data
Given that I am logged in
When I add a product with SKU, description, department, price, and quantity
Then I am able to register it
Variant: Basic data
Given that I have a ~user logged in~
and I visit the [Product Screen]
When I fill {SKU}
and I fill {Description}
and I fill {Price}
and I fill {Quantity}
and I click on {Save}
Then I have ~product registered~
and I see "Saved."
UI Element: SKU
- required
Otherwise I see "Please inform the SKU."
- format is "/[A-Z]{3}\-[0-9]{3}/"
Otherwise I see "Invalid SKU format. Please use 3 letters, dash, 3 numbers."
UI Element: Description
- minimum length is 2
Otherwise I see "Description needs at least 2 characters."
- maximum length is 100
Otherwise I see "Description must have at most 100 characters."
UI Element: Price
- required
Otherwise I see "Please inform the price."
- minimum value is 0.01
Otherwise I see "Minimum price is one cent."
UI Element: Quantity
- data type is integer
UI Element: Save
- type is button
Constants:
"Product Screen" is "/products/new"
These features could correspond to the following sketches:
There is an experimental application that can generate User Interface Prototypes (UIP) from a Concordia specification. The current version has a plug-in to create HTML-based UIPs and it's really easy to adapt it for other technologies.
The specification helps stakeholders, analysis, testers and developers to create a shared understanding about the application and its business rules. It helps with the requirements validation, knowledge documentation, project design, implementation and testing.
Different from Use Cases and other requirements specification formats, with Concordia you probably don't need to define different scenarios for validation. Instead of having to manually define a set scenarios for validation, you just need to define how the UI elements should behavior - in their declaration - and the compiler will generate different scenarios for you in the form of Test Cases. Whether your application has a lot of such rules, your team will certainly benefit from it.
<TO-DO (UNDER CONSTRUCTION)>
npx concordia --seed="example" --no-run --no-result
It will generate
login.testcase with the following content:login.testcase
add-employee.testcase
<TO-DO (UNDER CONSTRUCTION)>
<TO-DO (UNDER CONSTRUCTION)>
If a seed is not given, Concordia Compiler assumes the current date and time. The seed is always printed in the console so that you or your team can reproduce the same paths, test data, and test oracle that were able to expose a bug in your application. Here we are giving it for a sake of reproduction.
Using the same seed over and over again will make Concordia Compiler produce the same results. Options
no-run and no-result avoid running the test scripts and getting execution results.This will not generate test cases or test scripts, but it will execute them and get their results:
$ concordia --plugin=codeceptjs --no-test-case --no-script
login.testcase:# Generated with ❤ by Concordia
#
# THIS IS A GENERATED FILE - MODIFICATIONS CAN BE LOST !
import "login-en.feature"
@generated
@scenario(1)
@variant(1)
Test case: Successful login with valid credentials - 1
Given that I am in the "http://localhost/login" # [Login Screen]
When i fill <username> with "*RM)O," # invalid: inexistent element
And i fill <password> with "" # valid: last element
And I click on <ok> # {OK}
Then I must see "Invalid username" # from <username>
@generated
@scenario(1)
@variant(1)
Test case: Successful login with valid credentials - 2
Given that I am in the "http://localhost/login" # [Login Screen]
When i fill <username> with "bob" # valid: filled
And i fill <password> with -8655972838932479 # invalid: inexistent element
And I click on <ok> # {OK}
Then I must see "Invalid password" # from <password>
@generated
@scenario(1)
@variant(1)
Test case: Successful login with valid credentials - 3
Given that I am in the "http://localhost/login" # [Login Screen]
When i fill <username> with "alice" # valid: random element
And i fill <password> with "4l1c3pass" # valid: random element
And I click on <ok> # {OK}
Then I see "Welcome"
@generated
@scenario(1)
@variant(1)
Test case: Successful login with valid credentials - 4
Given that I am in the "http://localhost/login" # [Login Screen]
When i fill <username> with "alice" # valid: last element
And i fill <password> with "4l1c3pass" # valid: filled
And I click on <ok> # {OK}
Then I see "Welcome"
@generated
@fail
@scenario(1)
@variant(1)
Test case: Successful login with valid credentials - 5
Given that I am in the "http://localhost/login" # [Login Screen]
When i fill <username> with "" # invalid: not filled
And i fill <password> with "" # invalid: not filled
And I click on <ok> # {OK}
Then I see "Welcome"
@generated
@scenario(1)
@variant(1)
Test case: Successful login with valid credentials - 6
Given that I am in the "http://localhost/login" # [Login Screen]
When i fill <username> with "bob" # valid: first element
And i fill <password> with 123456 # valid: first element
And I click on <ok> # {OK}
Then I see "Welcome"
test/login.js:// Generated with ❤ by Concordia
// source: c:\code\tmp\concordia-test-pt\login-en.testcase
//
// THIS IS A GENERATED FILE - MODIFICATIONS CAN BE LOST !
Feature("Login");
Scenario("Successful login | Successful login with valid credentials - 1", (I) => {
I.amOnPage("http://localhost/login"); // (61,5) [Login Screen]
I.fillField("username", "*RMO,"); // (11,5) invalid: inexistent element
I.fillField("password", ""); // (12,7) valid: last element
I.click("ok"); // (64,7) {OK}
I.see("Invalid username"); // (14,5) from <username>
});
Scenario("Successful login | Successful login with valid credentials - 2", (I) => {
I.amOnPage("http://localhost/login"); // (61,5) [Login Screen]
I.fillField("username", "bob"); // (21,5) valid: filled
I.fillField("password", "-8655972838932479"); // (22,7) invalid: inexistent element
I.click("ok"); // (64,7) {OK}
I.see("Invalid password"); // (24,5) from <password>
});
Scenario("Successful login | Successful login with valid credentials - 3", (I) => {
I.amOnPage("http://localhost/login"); // (61,5) [Login Screen]
I.fillField("username", "alice"); // (31,5) valid: random element
I.fillField("password", "4l1c3pass"); // (32,7) valid: random element
I.click("ok"); // (64,7) {OK}
I.see("Welcome"); // (65,5)
});
Scenario("Successful login | Successful login with valid credentials - 4", (I) => {
I.amOnPage("http://localhost/login"); // (61,5) [Login Screen]
I.fillField("username", "alice"); // (41,5) valid: last element
I.fillField("password", "4l1c3pass"); // (42,7) valid: filled
I.click("ok"); // (64,7) {OK}
I.see("Welcome"); // (65,5)
});
Scenario("Successful login | Successful login with valid credentials - 5", (I) => {
I.amOnPage("http://localhost/login"); // (61,5) [Login Screen]
I.fillField("username", ""); // (52,5) invalid: not filled
I.fillField("password", ""); // (53,7) invalid: not filled
I.click("ok"); // (64,7) {OK}
I.see("Welcome"); // (65,5)
});
Scenario("Successful login | Successful login with valid credentials - 6", (I) => {
I.amOnPage("http://localhost/login"); // (61,5) [Login Screen]
I.fillField("username", "bob"); // (62,5) valid: first element
I.fillField("password", "123456"); // (63,7) valid: first element
I.click("ok"); // (64,7) {OK}
I.see("Welcome"); // (65,5)
});
Whether you ran the test scripts above, you probably saw they fail. That's because they didn't find a web application running at
http://localhost/login or because the application was found but it did not match the expected behavior. You may adapt your application and run the tests again.Concordia shows a report that indicates the failures' locations. They help you to decide if a failure was caused by the application under test (e.g., it did not behave as expected) or because of the requirements specification (e.g., it is outdated in relation to the application).
Now keeping your specification updated has a new clear benefit: you can use it to generate tests and discover existing defects in your application!
Last modified 2yr ago