Creating a Plug-in
Last updated
Last updated
Concordia Compiler uses plug-ins for generating test scripts, running them and reading their results.
A plug-in can...
be coded in JavaScript or , such as (recommended), , or .
generate code (test scripts) for any programming language or testing framework able to perform for web, desktop or mobile applications.
Its name must start with concordialang-
, for example: concordialang-my-awesome-plugin
.
It must be installable with . We recommend you to publish it at NPM too.
It must implement the interface from .
Its package.json
file must contain the property concordiaPlugin
, like in the following example:
Properties of concordiaPlugin
:
file: string
: Relative path of the file that contains an implementation of the Plugin
interface.
class: string
: Class (name) that implementsPlugin
.
serve: string
: Multi-platform console command to execute in order to start the testing server. Whether the testing framework does not need running a testing server, you can set something like echo \"Running a testing server is not needed.\"
.
A plug-in must deal with three tasks:
Let's get into the details on how to accomplish them.
Think of an Abstract Test Script (ATS) as an object to transform into source code. Basically, it contains a structure that represents the actions to be converted into commands of your preferred programming language.
For instance, consider the following action:
Let's consider a more realistic example, starting with a Test Case in the hypothetical file feature1.testcase
:
That would probably produce an Abstract Test Script like this:
A plug-in for Cypress could generate the following code from the above object:
You can observe that after every command there is a line comment containing the corresponding location in the .testcase
file. These comments will help to track the specification in case of test script failures.
To execute the produced test scripts, you can use the target framework API or run the tool. Your call.
A plug-in must be able to track back the location of failures or errors in the corresponding .testcase
file from a test script. When a test script has a failure or an error, the report saves the stack trace where it occurred. For instance, tests/feature1.js:20:4
indicates, respectively, the file, line and column where the problem happened. You can use these data to read the line of the test script file and retrieve the line comment that contains the corresponding line in the .testcase
file.
You can read these locations in the examples described earlier. Whether tests/feature1.js:7:8
is...
Retrieving the line 10
and column 4
fromfeature1.testcase
would give us:
We recommend you to follow these steps when creating a plug-in:
3. Create a repository for it (e.g., at GitHub, GitLab, BitBucket) and clone the repository.
4. Create a NPM package with npm --init
.
5. Add the property concordiaPlugin
to your package.json
file.
9. Test it with the Concordia Compiler before publishing it. Create a simple project that uses your plug-in. You can use NPM to install your plug-in from a directory in your computer or your remote repository. For example:
Transforming into test scripts (i.e., source code).
Executing the produced test scripts, regarding some .
Transforming test execution results (produced by the testing framework) to .
That could be converted to the following command in , for PHP:
Or to the following command in , for JavaScript:
To help generating such commands, you can use some template library such as (recommended) or .
For example, whether you chose to run Cypress through npx
and generate a report using the command to run could be the following:
Now the plug-in needs to read the report and transform it into an object of the class that will be returned to Concordia Compiler.
For instance, the reads the report from a JSON file generated by using with . The current implementation is available in the class .
You can use a regular expression to extract the data from the stack trace, like did in the method extractScriptLocationFromStackTrace
. Then you can use the class from to retrieve the location from the line comment. Example:
In , both test failures and errors are represented as an exception
which includes the properties scriptLocation
and specLocation
.
1. to indicate that you are interested in developing a new plug-in for a certain testing framework or programming language. That is good way to make others aware of your work.
2. Choose a good name for your plug-in and check whether it registered at . Remember that it must start with concordialang-
. We recommend that it includes the target framework, tools or language. Example: concordialang-selenium-python
to create a plug-in that generates test scripts for the Selenium framework and the Python language. You may use the opened issue (step 1) to get feedback about the name, if you wish.
6. Install as a dependency:
7. Add your files. We recommend the following structure, but you can adapt it as you wish. For example, if you use , you will probably add a tsconfig.json
file. If you use , your test folder will probably be named __tests__
. We also recommend you to add a file and a file (the latter should include the folder node_modules
).
8. Implement the interface . We recommend you to and encourage you to unit test all your code.
10. Publish your plug-in at after making sure that you are using . Congratulations! Tell everybody and ask for feedback! ✌