Allure report usage to the core
We all know that an allure report is used as a reporting tool. In general, we are using it as just a reporting tool with the basic available data in it. We can use the maximum extension with maximum detailing on the report.
In the allure reports, we don’t have the value for Trend, Environment, Executor, Duration Trends, Categories, Trend, etc. If we have those data in place it looks like a very cool report for the execution. Some data would be a great help like environment executed URL, Test data used for testing, environment Name, and link of the executed CI. Below is the default report.
Generating environment data on the allure-report
Once the allure-results folder is generated, there should be some mechanism to generate a environment.properties file or copy from the existing project folder and paste it inside the allure-results folder. This is based on your requirement and CI implementation on how to do it. If you have only a few environments you can keep it static and pick those based on conditions.
browser=chrome
environment=staging
application_url=https://gitlab.com/
tested_username=test@gitlab.com
tested_password=password1234
Generating executors data on the allure-report
Once the allure-results folder is generated, there should be some mechanism to generate a executor.json file or copy from the existing project folder and paste it inside the allure-results folder. In the executors.json file add the name, build Name and build Url. We can pass the build number, build name and build URL and add it to generate a executors.json file.
{
"name": "Allure Report",
"buildName": "allure-report_build_url#13",
"buildUrl": "http://gitlab.com/build#13"
}
Generating Trend data on the allure-reports
Once the allure-results folder is generated, save the folder and add it back to git or aws or any other cloud storage. When the next time you generate the allure-results add the history folder inside the allure-results. Once it's there generate the allure report with history and it will show trends.
Generating categories data on the allure-reports
Once the allure-results folder is generated, there should be some mechanism to generate a categories.jsonfile or copy from the existing project folder and paste it inside the allure-results folder.
[
{
"name": "Ignored tests",
"messageRegex": ".*ignored.*",
"matchedStatuses": ["skipped"],
"flaky": true
},
{
"name": "Infrastructure problems",
"traceRegex": ".*RuntimeException.*",
"matchedStatuses": ["broken", "failed"]
},
{
"name": "Outdated tests",
"messageRegex": ".*FileNotFound.*",
"matchedStatuses": ["broken"]
},
{
"name": "Passed",
"messageRegex": ".*",
"matchedStatuses": ["passed"]
}
]
Folder structure for allure-results
Skipped Tests on allure reports.
One of the main concerns using the open source tools to maintaining and customizing based on our requirements. We don't have a way to resolve it directly. When we used tags to execute the test it will skip the test if the tag is not available on the test file. As part of the results, we will see the test with the pending tests on the reports. This will give poor oversight of the overall test run. We should not be including the test which isn’t executed on the report. We also have an open issue on the GitHub for skipped reports from allure-report.
Workaround:
When the allure-results folder is generated, it contains several files. We can read all the .json files from the folder and we can delete the .json file containing the status skipped . We need to have a script running to read all the files and delete the files before converting the allure-results folder to the allure-report.
Conclusion :
Allure report is a powerful open source tool if we use it to the extent. I do understand there are some cons to it but there is always a workaround for that. In CI it would be a great report for the devs, QA, BA and PO to see the graphical representation with stats.
Feel free to hit clap if you like the content. Happy Automation Testing :) Cheers.
