# Creating a Plug-in

## Introduction

Concordia Compiler uses plug-ins for generating test scripts, running them and reading their results.

A plug-in can...

* **be coded** in JavaScript or [any language that "transpiles" to JavaScript](https://github.com/jashkenas/coffeescript/wiki/list-of-languages-that-compile-to-js), such as [TypeScript](https://typescriptlang.org) (recommended), [Dart](https://dartlang.org), or [CoffeeScript](http://coffeescript.org).
* **generate code** (test scripts) for **any programming language** or testing framework able to perform [functional tests](https://en.wikipedia.org/wiki/Functional_testing) for web, desktop or mobile applications. 

## Requirements for a plug-in

1. Its name must start with `concordialang-`, for example: `concordialang-my-awesome-plugin`.
2. It must be installable with [NPM](https://npmjs.com). We recommend you to publish it at NPM too.
3. It must implement the interface [`Plugin`](https://github.com/thiagodp/concordialang-plugin/blob/master/src/Plugin.ts) from [concordialang-plugin](https://github.com/thiagodp/concordialang-plugin).
4. Its `package.json` file must contain the property `concordiaPlugin`, like in the following example:

```javascript
{
   ...
   
   "concordiaPlugin": {
      "file": "dist/index.js",
      "class": "MyAwesomePlugin",
      "serve": "npx command-to-start-my-testing-server"
   }
}
```

Properties of `concordiaPlugin`:

* `file: string`: Relative path of the file that contains an implementation of the `Plugin` interface.
* `class: string`: Class (name) that implements`Plugin`.
* `serve: string`: Multi-platform console command to execute in order to start the testing server. Whether the testing framework does not need running a testing server, you can set something like `echo \"Running a testing server is not needed.\"`.

## Tasks to perform

A plug-in must deal with **three tasks**:

1. Transforming [Abstract Test Scripts](https://github.com/thiagodp/concordialang-plugin/blob/master/src/AbstractTestScript.ts) into test scripts (*i.e.*, source code).
2. Executing the produced test scripts, regarding some [execution options](https://github.com/thiagodp/concordialang-plugin/blob/master/src/TestScriptExecutionOptions.ts).
3. Transforming test execution results (produced by the testing framework) to [the expected resulting format](https://github.com/thiagodp/concordialang-plugin/blob/master/src/TestScriptExecutionResult.ts).

Let's get into the details on how to accomplish them.

### Transforming Abstract Test Scripts into test scripts

Think of an Abstract Test Script (ATS) as an object to transform into source code. Basically, it contains a structure that represents the actions to be converted into commands of your preferred programming language.

For instance, consider the following action:

```javascript
{
    "action": "click",
    "targets": ["#ok"]
}
```

That could be converted to the following command in [Codeception](https://codeception.com/), for PHP:

```php
I->click("#ok");
```

Or to the following command in [Cypress](https://www.cypress.io/), for JavaScript:

```javascript
cy.get('#ok').click();
```

Let's consider a more realistic example, starting with a Test Case in the hypothetical file `feature1.testcase`:

```
import "feature1.feature"

@generated
@scenario(1)
@variant(1)
Test Case: Scenario 1 | V1
  Given that I am on "https://myapp.com/register"
  When I fill <#email> with "john.doe@example.com"
    and I fill <#nome> with "John Doe"
    and I click on <#register>
  Then I see "Welcome, John Doe"
```

That would probably produce an Abstract Test Script like this:

```javascript
{
    "feature": { "name": "Feature 1" },
    "scenario": { "name": "Scenario 1" },
    "testcases": [
        {
            "scenario": "V1",
            "commands": [
                {
                    "action": "amOn",
                    "values": [ "https://myapp.com/register" ],
                    "location": { "line": 7, "column": 2 }
                },
                {
                    "action": "type",
                    "targets": [ "#email" ],
                    "values": [ "john.doe@example.com" ],
                    "location": { "line": 8, "column": 2 } 
                },            
                {
                    "action": "type",
                    "targets": [ "#name" ],
                    "values": [ "John Doe" ],
                    "location": { "line": 9, "column": 4 }                  
                },                
                {
                    "action": "click",
                    "targets": [ "#register" ],
                    "location": { "line": 10, "column": 4 } 
                },
                {
                    "action": "see",
                    "values": [ "Welcome, John Doe" ],
                    "location": { "line": 11, "column": 2 }
                }                                         
            ]
        }
    ]
}
```

A plug-in for Cypress could generate the following code from the above object:

```javascript
context("Feature 1", function() {

    it("Scenario 1 | V1", function() {
        cy.visit("https://myapp.com/register"); // (7,2)
        cy.get("#email").type("john.doe@example.com"); // (8,2)
        cy.get("#name").type("John Doe"); // (9,4)
        cy.get("#register").click(); // (10,4)
        cy.get("body").should("contain", "Welcome, John Doe"); // (11,2)
    });

});
```

To help generating such commands, you can use some **template library** such as [Mustache](http://mustache.github.io/) (recommended) or [Handlebars](http://handlebarsjs.com/).

You can observe that after every command there is a line comment containing the corresponding location in the `.testcase` file. These comments will help to track the specification in case of test script failures.

### Executing the test scripts

To execute the produced test scripts, you can use the target framework API or run the tool. Your call.

For example, whether you chose to run Cypress through `npx` and generate a report using [Mocha JUnit Reporter,](https://github.com/michaelleeallen/mocha-junit-reporter) the command to run could be the following:

```
npx cypress run --reporter junit
```

### Transforming execution results

Now the plug-in needs to read the report and transform it into an object of the class [`TestScriptExecutionResult`](https://github.com/thiagodp/concordialang-plugin/blob/master/src/TestScriptExecutionResult.ts) that will be returned to Concordia Compiler.

For instance, the [core of the plug-in for CodeceptJS](https://github.com/thiagodp/concordialang-codeceptjs-core) reads the report from a JSON file generated by using [CodeceptJS](https://codecept.io/) with [Mocha Multi Reporters](https://github.com/stanleyhlng/mocha-multi-reporters). The current implementation is available in the class [`ReportConverter`](https://github.com/thiagodp/concordialang-codeceptjs-core/blob/master/src/ReportConverter.ts).

A plug-in must be able to **track back the location of failures or errors in the corresponding `.testcase` file** from a test script. When a test script has a failure or an error, the report saves the stack trace where it occurred. For instance, `tests/feature1.js:20:4` indicates, respectively, the *file*, *line* and *column* where the problem happened.  You can use these data to read the line of the test script file and retrieve the line comment that contains the corresponding line in the `.testcase` file.&#x20;

You can use a regular expression to extract the data from the stack trace, like [`ReportConverter`](https://github.com/thiagodp/concordialang-codeceptjs-core/blob/master/src/ReportConverter.ts) did in the method `extractScriptLocationFromStackTrace`. Then you can use the class [`FileInstrumentationReader`](https://github.com/thiagodp/concordialang-plugin/blob/master/src/FileInstrumentationReader.ts) from [concordialang-plugin](https://github.com/thiagodp/concordialang-plugin) to retrieve the location from the line comment. Example:

```javascript
const scriptLocation = /* extract from stack trace */;

// Suppose that `failureLocation` is:
// {
//    path: "tests/feature1.js",
//    line: 7,
//    column: 8
// }

// Retrieve the comment with the reader:
const specLocation = reader.retrieveSpecLocation(
    scriptLocation
    );
    
// `specLocation` would be:
// {
//    path: "features/feature1.testcase",
//    line: 10,
//    column: 4
// }
```

You can read these locations in the examples described earlier. Whether `tests/feature1.js:7:8` is...&#x20;

```javascript
cy.get("#register").click(); // (10,4)
```

Retrieving the line `10` and column `4` from`feature1.testcase` would give us:

```
  and I click on <#register>
```

In [`TestScriptExecutionResult`](https://github.com/thiagodp/concordialang-plugin/blob/master/src/TestScriptExecutionResult.ts), both test failures and errors are represented as an `exception` which includes the properties `scriptLocation` and `specLocation`.

## Basic guide

We recommend you to follow these steps when creating a plug-in:

1\. [Open an issue](https://github.com/thiagodp/concordialang/issues/new) to indicate that you are interested in developing a new plug-in for a certain testing framework or programming language. That is good way to make others aware of your work.

2\. **Choose a good name** for your plug-in and check whether it registered at [NPM](https://www.npmjs.com/). Remember that it must start with `concordialang-`. We recommend that it includes the target framework, tools or language. Example: `concordialang-selenium-python` to create a plug-in that generates test scripts for the Selenium framework and the Python language. You may use the opened issue (step 1) to get feedback about the name, if you wish.

3\. **Create a repository** for it (e.g., at GitHub, GitLab, BitBucket) and **clone** the repository.

4\. **Create a NPM package** with `npm --init`.

5\. **Add the property** `concordiaPlugin` to your `package.json` file.

6\. **Install** [concordialang-plugin](https://github.com/thiagodp/concordialang-plugin) as a dependency:

```
npm install --save concordialang-plugin
```

7\. **Add your files**. We recommend the following structure, but *you can adapt it as you wish*. For example, if you use [TypeScript](https://typescriptlang.org), you will probably add a `tsconfig.json` file. If you use [Jest](https://facebook.github.io/jest/), your test folder will probably be named `__tests__`. We also recommend you to add a [`.editorconfig`](http://editorconfig.org) file and a [`.gitignore`](https://git-scm.com/docs/gitignore) file (the latter should include the folder `node_modules`).

```
 ┃
 ┣ src/            🡐 your source code
 ┣ test/           🡐 your tests
 ┣ .editorconfig
 ┣ .gitignore
 ┗ package.json
```

8\. **Implement the interface** [Plugin](https://github.com/thiagodp/concordialang-plugin/blob/master/src/Plugin.ts). We recommend you to [take a look at other plugins that did a similar job](https://app.gitbook.com/s/-LyZeGFeFI9v0gasbqLP/development/plugins.md#available-plug-ins) and encourage you to unit test all your code.

9\. **Test it with the Concordia Compiler before publishing it**. Create a simple project that uses your plug-in. You can use NPM to install your plug-in **from a directory in your computer** or your remote repository. For example:

```bash
cd path/to/your/test-project/
npm install /path/to/your-plugin
```

10\. **Publish your plug-in** at [NPM](https://www.npmjs.com/) after making sure that you are using [Semantic Versioning](http://semver.org).  *Congratulations!*  Tell everybody and ask for feedback! ✌

```bash
npm publish
```