Aggregated Allure report using Copy Artifact Jenkins plugin
How to aggregate test results from downstream Jenkins jobs to one Allure report in upstream job
A couple of weeks ago I faced a problem, on which I spend more time than I thought I should, so I decided to write this article to help you do the same thing easier and faster.
Task at hand
Let’s assume that we have two different Jenkins jobs: one for backend tests and other for UI tests. And we have one job which start these jobs in parallel with different parameters (e.g. UI tests run for several browsers). It’s possible to create Allure report for each downstream job, but we’re too lazy to open different reports each time regression tests are finished. So, we want to have one report in our upstream job which contains all test results for current run.
“Sounds very easy” I thought, but it turned out to be a bit tricky, especially for a person with poor Jenkins knowledge (yes, that’s me). So, let’s move to the solution.
I will use Jenkins Declarative Pipeline syntax in my examples.
Solution
To solve this problem, let’s decompose it into smaller ones.
In order to copy allure results from downstream job we need somehow to archive them there. So the first problem will be:
1. How to archive allure results in downstream job?
This is very easy. You just need to add a post action with archiveArtifacts to each downstream job. To archive only allure results specify relative path to them using artifacts parameter. With pipeline syntax it looks similar to this:
pipeline { ... post {
always {
archiveArtifacts artifacts: 'allure-results/*'
}
}
}
2. How to copy archived artifacts to upstream job?
This is the job for Jenkins Copy Artifact Plugin. You can find more information about the plugin here. I will just describe how to use it in this particular case. To do this you need to add post action for each stage where you run tests job with copyArtifacts plugin and you need to specify following parameters:
filter relative paths to artifacts you want to copy, in our case it is 'allure-results/*' ;
projectName name of the job we want to copy artifacts from, in our case the name of downstream job;
selector parameter that helps to select the build to copy artifacts from, there are a plenty of options, but we will use lastCompleted since our tests tend to fail sometimes.
The full step script in declarative pipeline looks like:
pipeline { ... stages{ ... stage('run tests') {
steps{
script{
build job: 'tests'
}
}
post {
always {
copyArtifacts(
filter: "allure-results/*",
projectName: 'tests',
selector: lastCompleted()
)
}
}
}
...
}
...}
3. How to select correct build of one job with different parameters?
As you might remember we also want to run our UI tests in different browsers in parallel and with the current set up there might be a problem with reports.
Let’s imagine or upstream job starts two builds of UI tests job in parallel with parameters chrome and firefox as browsers where we want to run tests. And if build for chrome is the first in your pipeline, but build for firefox is finished earlier, then Copy Artifact plugin copies test results from the firefox build twice because it is lastCompleted build in both cases.
It was actually the problem on which I spent most of the time. I tried to save build numbers to variables and a bunch of other staff, but solution it very simple.
You just need to add parameters to copyArtifacts() step. This allows you to filter builds by build parameters, e.g. to select artifacts from build with parameter browser=chrome your pipeline script should look like:
...post {
always {
copyArtifacts(
filter: "allure-results/*",
projectName: 'ui-tests',
parameters: 'browser=chrome',
selector: lastCompleted()
)
}
}...
Pulling all together
Here is the whole upstream pipeline for our needs:
pipeline {
agent { ... }
stages{ ...stage('run tests') {
parallel{
stage("run api tests") {
steps{
script{
build job: 'api-tests'
}
}
post {
always {
copyArtifacts(
filter: "allure-results/*",
projectName: 'api-tests',
selector: lastCompleted()
)
}
}
}
stage("run ui tests in chrome") {
steps{
script{
build job: 'ui-tests',
parameters: [
[$class: 'StringParameterValue', name: 'browser', value: "chrome"]
]
}
}
post {
always {
copyArtifacts(
filter: "allure-results/*",
projectName: 'ui-tests',
parameters: 'browser=chrome',
selector: lastCompleted()
)
}
}
}
stage("run ui tests in firefox") {
steps{
script{
build job: 'ui-tests',
parameters: [
[$class: 'StringParameterValue', name: 'browser', value: "firefox"]
]
}
}
post {
always {
copyArtifacts(
filter: "allure-results/*",
projectName: 'ui-tests',
parameters: 'browser=firefox',
selector: lastCompleted()
)
}
}
}
}
}
}
post('create allure report'){
always{
script {
allure([
includeProperties: false,
jdk: '',
properties: [],
reportBuildPolicy: 'ALWAYS',
results: [[path: 'target/allure-results']]
])
}
}
}
}
And don’t forget to archive artifacts in downstream jobs:
pipeline {...post {
always {
archiveArtifacts artifacts: 'allure-results/*'
}
}
}
Conclusion
The main difficulties I faced while looking for solution were not entirely clear documentation for Jenkins and Jenkins plugins and a lack of useful examples. So, I spent quite a lot of time on a task that can be easily done for 15 minutes. I will be very happy, if this information is helpful for someone who’s looking for similar solution.
