What's Next

Work in progress and issues:

• Kanban Board (GitHub)

• Issues (GitHub)

Logo

Yes, this project does not have a logo yet 😱. Designers interested in helping us (for free 🙏), please contact us on our Slack Channel or open an Issue.

The logo should reflect the idea behind the language:

The tool generates test cases and test scripts that verify if the application under test meets the specified functional requirements. Therefore, the idea of "verification" or "testing" could be in the logo.

Plug-ins

Interested? See how to create a plug-in.

👉 Our next goal is a plugin for Cypress.

Suggestions:

• JavaScript and friends: Cypress (web), MacacaJS (web, mobile, desktop)

• PHP: Codeception (web)

• Java: Selenium (web), Robotium (android), AssertJ Swing (desktop)

• Ruby: Capybara (web), Calabash (mobile applications developed with Xamarin)

• Python: Selenium (web)

• C++: QtTest (desktop), DUnit (desktop)

• Delphi: DUnitX (desktop), DUnit (desktop)

• Other programming languages or frameworks are welcome! 💖

Language

Variant Background

A Variant Background defines steps that must be executed before all the Variants of a certain Scenario. Example:

Feature: Search

  Scenario: Simple Search
  
    Variant Background:
      Given that I visit the [Search Page]
      When I fill {Search} with "Hello"
      
    Variant: Search by clicking on Go
      When I click on {Go}
      Then I see "Hello World!"
      
    Variant: Search by hitting Enter
      When I press "Enter"
      Then I see "Hello World!"      

MOTIVATION: It allows to keep initial, repeated steps in a common place.

Constant names without quotation marks

Like this:

Constants:
  - Net Price is 10.0
  - hello is "hello"

MOTIVATION: Simpler syntax.

Visual comparison

Please see Issue #27.

MOTIVATION: It allows to perform visual comparison and detect related bugs.

File content comparison

To use the characters """ (three quotation marks) to denote a text content to be compared. Example:

  ...
  Then I see the file "out.txt" with
"""
This is some text to be compared.
  It will probably
  be compared to a
  text file.
"""

MOTIVATION: It facilitates comparisons with text files or multiple-line strings.

Table matching

To allow a given UI Element or UI Literal to match a certain Table.

Example 1:

Then I see the table {MyTable} as [Some Table]

Example 2:

Then I see the table <myTable> as [Some Table]

In which Some Table is declared like this:

Table: Some Table
| Name  | Age |
| Bob   | 21  |
| Suzan | 25  |

It should make target table's rows to match the declared ones.

MOTIVATION: It facilitates the verification of expected data in tables from a user interface.

NOTES: Probably it requires to convert the Concordia Action to many abstract commands that verify every table row independently. Usually testing frameworks do not provide such kind of verification (for many rows at once).

Date and time expressions inside tables

Use Concordia 2 data and time expressions inside table rows. They must be written between ` or some other character. Example:

Table: Some Table
| Name | Age            |
| Bob  | `18 years ago` |

MOTIVATION: Concordia Language allows date and time expressions inside UI Properties. Using them inside tables can be useful.

Dynamic states

States that vary according to some generated value. Example:

...
    Given that I fill {User}
    and I fill {Pass}
    and I click  on {OK}
  Then I see "Welcome"
    and I have ~{User Type} is logged in~

UI Element: User
  - value comes from "SELECT user FROM [Users]"

UI Element: Pass
  - value comes from "SELECT pass FROM [Users] WHERE user = {User}"

UI Element: User Type
  - value comes from "SELECT user_type FROM [Users] WHERE user = {User}"

Table: Users
  | user  | pass    | user_type |
  | bob   | 123456  | admin     |
  | joe   | 654321  | guest     |
  | alice | 123456  | admin     |

According to the selected user, it will produce a different State. For example, when "bob" is selected, the produced state will be admin is logged in and when "joe" is selected, the produced state will be guest is logged in .

Thus, Features could depend on static or dynamic states.

MOTIVATION: Making states more flexible.

Alternative states

To provide a Given sentence that requires one - and only one - of the provided states. Example:

Variant: Example 1
  Given that I have either ~x~ or ~y~
 ...
    
Variant: Example 2
  Given one of ~x~, ~y~
  ...

MOTIVATION: It allows to perform a XOR operation with the given states using natural language, aiming at choosing one and only one of the given states.

Annotations to parameterize combination strategies

To provide annotations parameterize a test combination strategy for a specific Feature.

For instance, to configure the combination of invalid values for a specific Feature:

@comb-invalid(1)
Feature: Example

The annotations should correspond to the CLI parameters.

MOTIVATION: Increase the flexibility of the test generation procedure.

Multi-line queries

Currently:

- value comes from "SELECT name FROM [MyDB].profession"

Proposal (to accept as valid):

- value comes from "SELECT name
                   FROM [MyDB].profession"`

Alternative proposal:

- value comes from
"""
SELECT name
FROM [MyDB].profession
"""

MOTIVATION: Make it easier to write/read SQL statements.

Compiler

Selection Options

• Select a Feature for Test Case generation, without having to include its dependencies:

  • by importance value

• Select a Scenario for Test Case generation:

  • by importance value (tag @importance)

• Test Case for test script generation;

  • by importance value (tag @importance)

Watch Mode

• Provide the option --watch to generate .testcase files when their corresponding .feature file changes or an imported .feature file changes. A new seed must be produced, except when explicitly provided (using --seed or configuration file).

Language Support

Concordia has language constructions that the Compiler does not support yet. Examples:

• Variant Background: Implement Variant Background, which is part of the Concordia Language but was not implemented yet by the Concordia Compiler.

• Support the tag @ignore in Features and Scenarios. Currently it is supported inVariants and Test Cases. Whether added to a Feature or a Scenario, it would not generate Test Cases.

• Consider global UI Elements

  Make the tool processing UI Elements tagged with @global.

  Allow inheritance of UI Elements

  Use @extends( <name> ) to extend another UI Element.

  Example:

  Copy

  UI Element: Name
  - min length is 2
  - max length is 100
  
  @extends( Name )
  UI Element: Emergency Contact Name
  
  # Emergency Contact Name inherits the properties from Name

• Multiple declared Tables per query: Allow a query to reference more than one table declared in the specification.

• Multiple declared Databases per query: Allow a query to reference more than one database declared in the specification.

New Test Cases

• Test cases that explore SQL Injection: Using declared Databases and Queries to formulate Test Cases that generate strings with SQL Injection commands.

• Test cases that use naughty strings as test data: Using a list of naughty strings as test data.

UI and Report

To provide the CLI parameter --report for indicating the corresponding plug-in. Example:

npx concordia --report html

which should be equivalent to

npx concordia --report concordialang-report-html

Tool integration

Integration with text editors

• Create new projects for auto-completion plug-ins for text editors such as VS Code, Sublime Text, Atom, etc. Example for VS Code: gherkin-autocomplete.

• Be able to run test scripts using Concordia from the IDE.

Integration with reporting tools

• Create integration with Allure. Other reporters (e.g., Macaca Reporter) can be added further.

Performance improvements

Avoid generating test scripts when their Test Case files did not change

Keep some hash control or use Git information when available. Hashes can be stored in a .json or .yml file. Example:

{
  "hashes": {
    "feature1.testcase": "ox10JBprHtu5c8822XooloNKUfk=",
    "subdir/feature2.testcase": "DMcj5b67Albe4KhpzyvphC5nVDHn1oCO",
  }
}

Generate unrelated files in parallel

NodeJS has made considerable progress since version 10.5 adding support to Worker Threads. Stable support for Worker Threads was added in version 12 LTS. Although, it would increase minimum requirements to install Concordia Compiler - currently it requires NodeJS 8.0.