Overview
This article is meant to help provide troubleshooting steps to address issues the "Rerun failed tests" feature in CircleCI rerunning all tests, and not just failed tests.
Issue: All tests are still being rerun
If you find that all tests are being rerun after clicking "Rerun failed tests," please follow the steps below to resolve the issue.
Enable verbose mode: Make sure that the
--verbosesetting is enabled when invokingcircleci tests run. This will display the tests thatcircleci tests runis receiving during a rerun.
Upload JUnit XML: Use the
store_artifactscommand to upload the JUnit XML file containing the test results to CircleCI. This is the same file(s) that is being uploaded withstore_test_results.
Inspect JUnit XML: Manually inspect the newly uploaded JUnit XML via the Artifacts tab in CircleCI. Ensure that the XML file contains either a
fileattribute or aclassnameattribute. If neither attribute is present, unexpected behavior may occur when trying to rerun tests.
Check xargs is present: Ensure that
xargsis present in the--command=argument.
If you are using a common language or framework, you may be able to find an example in our documentation that outlines a standard implementation of this.
This can be useful in debugging errors seen in your pipelines, and finding niche settings that are required for certain frameworks, for example the junit_family=legacy settings required in pytest.