I'm trying to set up a pipeline on Azure DevOps with Cypress tests.
Locally test output file is created correctly.
I'm using npx cypress run command
I'm getting an error/warning for Publish Test Results:
##[warning]No test result files matching **/test-output-*.xml were found.
Here is my cypress.json file:
{
"reporter": "junit",
"reporterOptions": {
"mochaFile": "tests/test-output-[hash].xml",
"toConsole": true,
"attachments": true
},
"video": false,
"pluginsFile": "cypress/plugins/index.js",
"supportFile": "cypress/support/index.js"
}
Here is azure-pipelines.yml:
# Node.js
# Build a general Node.js project with npm.
# Add steps that analyze code, save build artifacts, deploy, and more:
# https://learn.microsoft.com/azure/devops/pipelines/languages/javascript
trigger:
- develop
pool:
vmImage: 'ubuntu-latest'
steps:
- task: NodeTool#0
inputs:
versionSpec: '10.x'
displayName: 'Install Node.js'
- script: |
npm install
displayName: 'npm install'
- script:
npx cypress run
displayName: 'Execute cypress tests'
- task: PublishTestResults#2
displayName: "Publish Test Results"
condition: always()
inputs:
testResultsFormat: 'JUnit'
testResultsFiles: '**/test-output-*.xml'
searchFolder: '$(System.DefaultWorkingDirectory)'
- task: PublishBuildArtifacts#1
condition: always()
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)'
ArtifactName: 'drop'
publishLocation: 'Container'
I tried to do all weird stuff, but nothing helps.
Checked all StackOverflow topics like those below:
Azure DevOps test -xml not found after running the Cypress tests
Is there any way to show Cypress Test Results in Azure DevOps Test Results Tab?
Azure DevOps test -xml not found after running the Cypress tests
No test result files were found using search pattern '...\**\TEST-*.xml
Cypress Integration with DevOps
All is looking to be set up correctly according to Cypress documentation and blogs etc.
Maybe test output file is not created on Azure?
Maybe someone has a clue?
EDIT:
I checked using ls -al command, that tests folder is not created.
But even if I created it using mkdir tests before starting cypress the folder is empty after the cypress job.
So Cypress is not creating test output report. Why locally the file is created but on Azure no?
Please check with the following steps:
Set the pipeline variable system.debug to be true, and run the pipeline again.
After the step "Execute cypress tests" is completed, check if you can get more details for troubleshooting from the debug logs on the console window.
You mentioned that the same npx cypress run command can work fine on your local machine, please try to install a self-hosted agent on your local machine to run the pipeline to see if the problem still exists.
If the problem still exists, for us to investigate this problem further, please share the complete logs of the test step with us.
Just had the same issue. Had my artifact download task set on a specific build, so I never get the new build with the correct cypress.config file. Updated the build target and now everything is working. So thanks #DuduA, I thought I'll just answer it so it's a bit easier to see if someone has the same issue.
Please add a command line task to check if cypress run command is running at which folder.
I think you might be running the cypress command in wrong folder. The issue will be resolved if you will provide the correct folder structure and run cypress run command where cypress.json file exists.
Also in cypress.json file check the path of plugins file and support file.
Related
I am trying to create a build script to restore, build, test and code coverage a .net solution and then publish the results to sonarcloud. I have succeeded in most of this, but there is a new dotnet step for collecting code-coverage that has been suggested which can be used as follows:
- name: Install dotnet coverage
run: dotnet tool install --global dotnet-coverage
- name: Coverage
run: dotnet-coverage collect dotnet test --no-build -f xml -o 'coverage.xml'
The step runs absolutely fine, collects code coverage as expected, but it took me a while to realise that because of the nested call to the dotnet test command, this step does NOT fail when a test fails. To me I'm expecting that to be the default behaviour. Now, aside from splitting out the test command into a separate step to force this, is there a way of making the above line fail when a test fails. Right now I've been forced to do this:
- name: Test
run: dotnet test --no-build
- name: Coverage
run: dotnet-coverage collect dotnet test --no-build -f xml -o 'coverage.xml'
Which basically means running the tests twice which seems inefficient. Is there an easy way around this?
You can use collect option for dotnet test command to collect coverage data while test runs. You can also set data collector, result file format, name and directory. For example:
dotnet test --collect:"Code Coverage;Format=Xml;CoverageFileName=coverage.xml" --results-directory ./coverage
Pay attention that although you can specify result directory another one directory(with Guid as name) will automatically be created inside it. Look at this issue.
Ok, so I am trying to split my Appium tests by their timing in CircleCI for running the test in parallel. My tests are written in behave(Python) and I am generating the JUnit XML file. Here is my config.yml file
version: 2.1
orbs:
macos: circleci/macos#2.2.0
jobs:
example-job:
macos:
xcode: 13.4.1
parallelism: 4
resource_class: large
steps:
- checkout
- run:
name: Install appium server
command: |
sudo npm update -g
sudo npm install -g appium
sudo npm install -g wd
- run:
name: Start appium server
command: appium --address localhost --port 4723
background: true
- run:
name: Installing Dependencies
command: pip3 install -r requirements.txt
- run:
name: Test application
command: |
TEST=$(circleci tests glob "features/featurefiles/*.feature" | circleci tests split --split-by=timings --timings-type=classname)
echo $TEST
behave $TEST --junit
- store_test_results:
path: reports
- store_artifacts:
path: ./Logs
destination: logs-file
- store_artifacts:
path: ./screenshots
workflows:
example-workflow:
jobs:
- example-job
When I am running the test, I am getting the error "No timing found for "features/featurefiles/XXX.feature" and it is splitting the test by the filename. It runs well but the split is not happening by the timing.
When the execution is done, I can see the data in the TESTS tab, also in the Timing Tab
I believe CircleCI is not able to read the JUnit file generated by Behave, it is searching for the different JUnit XML file. How can we make CircleCI read the JUnit file generated by Behave?
If anyone face such issue, please take a look at the classname in the JUnit report. There is a format in which CircleCI read the classname. In my case, the classname in the JUnit report was mentioned as
features.featurefiles.Login.feature
But the CircleCI was looking for the classname in below format
features/featurefiles/Login.feature
I had to write a utility to change the classname in the report once the execution was completed.Once it was done, CircleCI was able to read the timings.
Hope it help someone :)
I'm starting to use Cloud Build for a project, and I'm having the following issue:
Using this cloudbuild.yaml file:
steps:
- name: 'gcr.io/cloud-builders/npm'
args: ['install']
- name: 'gcr.io/cloud-builders/npm'
args: ['test']
The build run okay on the first step because it is able to install the dependencies and everything but in the second step it fails because it needs to connect to a MySQL database.
I saw in another SO post, you can use an extra build step to run the Cloud SQL proxy and connect that way. Check it out that let us know if that solution works for you.
Trying to setup remote Codeception Unit Tests in PhpStorm in a Yii2 project.
Using SSH I can log into the server go to the root directory of my Yii2 project and run :
> vendor/bin/codecept run unit
and the tests run.
I'm trying run these remote tests via PhpStorm, I've setup a Remote PHP CLI interpreter and I'm pointing to the Codeception library in my Yii2 project folder:
/var/www/vhosts/mydomain.com/httpdocs/yii2/vendor/bin/codecept
Test Runner points to:
/var/www/vhosts/mydomain.com/httpdocs/yii2/codeception.yml
Trying to run the tests the following command is executed:
> ssh://user#mydomain.com:22/opt/plesk/php/5.6/bin/php /root/.phpstorm_helpers/phpunit.php --no-configuration /var/www/vhosts/mydomain.com/httpdocs/yii2/tests
The process fails at it complains that it cannot find PHPUnit:
Process finished with exit code 1
Cannot find PHPUnit in include path (.:/opt/plesk/php/5.6/share/pear)
How do I get PhpStorm to look for PHPUnit in the yii2/vendor folder? Can I just tell PhpStorm to run a different command instead of this phpstorm_helpers? It seems that the documentation is out of date and the screenshots JetBrains provides are from a different version of PhpStorm, I'm running PhpStorm 2017.3
So after a LOT of digging, the issue was with the Run/Debug Configuration. Despite adding Codeception to the Test Frameworks section, clicking the run button still tried to execute a pure PHPUnit test.
To switch to run the test as Codeception, look at the top toolbar above the file tabs:
There you will be able to define various options:
Now under run you'll have additional options:
Choose the blue Codeception icon to run the test using Codeception instead of PHPUnit
I'm writing this because yii2 official documentation is still not complete and codeception documentation itself refer to yii2 official docs..:).
I have some questions:
In my yii2 app root there is a directory "tests/codeception", this means that codeception is already installed in my project?
in vendor/yiisoft there is another codeception directory "yii2-codeception" what is it?
the documentation say to create a yii2_basic_tests database and to run a migration, but migration script create only a "migration" table, is it correct?
the integration with yii2 provide some web interface or I must run the tests from console scripts?
Someone can explain me how to install and configure codeception in yii2 basic app step by step?
Thank you
Alessandro
I am doing some like that:
composer global require "codeception/codeception=2.0.*" "codeception/specify=*" "codeception/verify=*"
And next:
ln -s ~/.composer/vendor/bin/codecept /usr/local/bin/codecept
Then I am available to do globally
codecept run
First make sure you read this a couple of times http://www.yiiframework.com/doc-2.0/guide-test-environment-setup.html
Afterwards the actual tests are easy to set up. Make sure you make the codecept command work like it says on the last line of the link above. After you install an Yii app you have to go to the tests folder and run
codecept build
to initialise the tests. Then run
codecept run
to run the actual tests.
You can run
codecept run --coverage-html
to get the code coverage for your project.
I have never got the acceptance testing working with code coverage but I got acceptance working without coverage and unit&functional with coverage.