Example Project

See a complete working example: bistro-e2e — Playwright E2E tests with QA Sphere integration.

QA Sphere CLI Tool: Playwright Integration

This guide shows you how to integrate Playwright test results with QA Sphere using the QAS CLI tool.

The QAS CLI supports two methods for uploading Playwright results:

Playwright JSON upload (playwright-json-upload) — Uses Playwright's native JSON report format with support for test annotations (recommended)
JUnit XML upload (junit-upload) — Uses Playwright's JUnit reporter output

Installation

Prerequisites

Node.js version 18.0.0 or higher
Playwright test project

Installation Methods

Via NPX (Recommended) Simply run npx qas-cli. On first use, you'll need to agree to download the package. You can use npx qas-cli in all contexts instead of the qasphere command.

Verify installation:
```
npx qas-cli --version
```

Via NPM

npm install -g qas-cli

Verify installation:

qasphere --version

Configuration

The QAS CLI requires two environment variables to function properly:

QAS_TOKEN - QA Sphere API token
QAS_URL - Base URL of your QA Sphere instance (e.g., https://qas.eu2.qasphere.com)

You can create your API key under the QA Sphere settings.

To do that:

Go to Settings by clicking the gear icon in the top right corner.
Click API Keys
Click Create API Key and generate one

Api Key Screenshot

These variables can be defined in several ways:

Environment Variables: Export them in your shell:

export QAS_TOKEN=your_token_here
export QAS_URL=your_qas_url

.env File: Place a .env file in the current working directory:
```
# .env
QAS_TOKEN=your_token
QAS_URL=your_qas_url
```

Configuration File: Create a .qaspherecli file in your project directory or any parent directory:

# .qaspherecli
QAS_TOKEN=your_token
QAS_URL=your_qas_url

# Example with real values:
# QAS_TOKEN=tst0000001.1CKCEtest_JYyckc3zYtest.dhhjYY3BYEoQH41e62itest
# QAS_URL=https://qas.eu1.qasphere.com

Playwright Configuration

Option A: JSON Reporter (Recommended)

To generate JSON reports from Playwright, configure the JSON reporter in your playwright.config.js:

// playwright.config.js
const { defineConfig } = require('@playwright/test');

module.exports = defineConfig({
  testDir: './tests',
  timeout: 30000,

  // JSON reporter for QA Sphere integration
  reporter: [
    ['list'],  // Console output
    ['json', { outputFile: 'test-results/results.json' }]  // For QA Sphere
  ],

  use: {
    headless: true,
    screenshot: 'only-on-failure',
    video: 'retain-on-failure',
  },

  projects: [
    { name: 'chromium', use: { browserName: 'chromium' } },
    { name: 'firefox', use: { browserName: 'firefox' } },
    { name: 'webkit', use: { browserName: 'webkit' } },
  ],
});

Option B: JUnit XML Reporter

Alternatively, configure the JUnit reporter:

// playwright.config.js
const { defineConfig } = require('@playwright/test');

module.exports = defineConfig({
  testDir: './tests',
  timeout: 30000,

  // JUnit reporter for QA Sphere integration
  reporter: [
    ['list'],  // Console output
    ['junit', { outputFile: 'junit-results/results.xml' }]  // For QA Sphere
  ],

  use: {
    headless: true,
    screenshot: 'only-on-failure',
    video: 'retain-on-failure',
  },

  projects: [
    { name: 'chromium', use: { browserName: 'chromium' } },
    { name: 'firefox', use: { browserName: 'firefox' } },
    { name: 'webkit', use: { browserName: 'webkit' } },
  ],
});

Key Configuration Points

JSON Reporter (Option A): Generates a JSON file that supports Playwright test annotations for linking test cases
JUnit Reporter (Option B): Generates XML files using the standard JUnit format
Output File: Specify a consistent output path for your chosen format
Screenshots: Configure screenshot: 'only-on-failure' to capture screenshots for failed tests
Videos: Use video: 'retain-on-failure' to keep videos of failed tests

Uploading Results

Both the playwright-json-upload and junit-upload commands support two modes:

Create a new test run — when --run-url is not specified
Upload to an existing test run — when --run-url is specified

Command Options

Option	Description
`-r, --run-url`	Upload results to an existing test run URL
`--project-code`	Project code for creating a new test run (can also be auto-detected from markers)
`--run-name`	Name template for the new test run. Supports `{YYYY}`, `{MM}`, `{DD}`, `{HH}`, `{mm}`, `{ss}`, `{env:VAR}` placeholders
`--create-tcases`	Automatically create test cases in QA Sphere for results without valid markers
`--attachments`	Detect and upload attachments with test results (screenshots, videos, traces)
`--force`	Ignore API errors, invalid test cases, or attachment issues
`--ignore-unmatched`	Suppress individual unmatched test messages, show summary only
`--skip-report-stdout`	Control when to skip stdout blocks (`on-success` or `never`, default: `never`)
`--skip-report-stderr`	Control when to skip stderr blocks (`on-success` or `never`, default: `never`)

Using `playwright-json-upload` (Recommended)

# Create new test run automatically
npx qas-cli playwright-json-upload <path-to-json>

# Upload to existing test run
npx qas-cli playwright-json-upload -r <run-url> <path-to-json>

Examples:

Upload Playwright JSON results:

npx playwright test
npx qas-cli playwright-json-upload test-results/results.json

Upload to an existing test run:

npx qas-cli playwright-json-upload -r https://qas.eu1.qasphere.com/project/P1/run/23 test-results/results.json

Create a new run with a custom name:

npx qas-cli playwright-json-upload --project-code P1 --run-name "Playwright {YYYY}-{MM}-{DD} {HH}:{mm}" test-results/results.json

Auto-create test cases for unmatched results:

npx qas-cli playwright-json-upload --project-code P1 --create-tcases test-results/results.json

Using `junit-upload`

# Create new test run automatically
npx qas-cli junit-upload <path-to-junit-xml>

# Upload to existing test run
npx qas-cli junit-upload -r <run-url> <path-to-junit-xml>

Examples:

Upload JUnit XML results:

npx playwright test
npx qas-cli junit-upload junit-results/results.xml

Upload with attachments (screenshots, videos, traces):

npx qas-cli junit-upload --attachments junit-results/results.xml

Create a new run with a CI build name:

npx qas-cli junit-upload --project-code P1 --run-name "CI Build {env:BUILD_NUMBER} - {YYYY}-{MM}-{DD}" junit-results/results.xml

Suppress unmatched test noise in output:

npx qas-cli junit-upload --ignore-unmatched junit-results/results.xml

NPM Script Integration

Add convenient scripts to your package.json:

{
  "scripts": {
    "test": "playwright test",
    "test:upload": "playwright test && npx qas-cli playwright-json-upload --attachments test-results/results.json"
  }
}

Or using JUnit XML:

{
  "scripts": {
    "test": "playwright test",
    "test:upload": "playwright test && npx qas-cli junit-upload --attachments junit-results/results.xml"
  }
}

Then run:

npm run test:upload

Why this approach?

Always up to date: Using npx qas-cli ensures you always get the latest version. For CI/CD environments where stability is critical, you can pin a specific version (e.g., npx qas-cli@0.4.6)
Convenience: A simple npm run test:upload is easier to remember
CI/CD friendly: The script can be easily integrated into GitHub Actions, GitLab CI, or other pipelines
Team consistency: Everyone runs the same command with the same flags

Linking Tests to QA Sphere Test Cases

To map Playwright test results to QA Sphere test cases, you can use test annotations (for Playwright JSON) or test case markers in test names (for both JSON and JUnit XML).

Method 1: Test Annotations (Recommended for Playwright JSON)

Add a test annotation with the full QA Sphere test case URL. This is the most reliable approach and is the recommended method when using playwright-json-upload.

// tests/login.spec.ts
import { test, expect } from '@playwright/test';

test(
  'Login with valid credentials',
  {
    annotation: {
      type: 'test case',
      description: 'https://qas.eu1.qasphere.com/project/PRJ/tcase/312',
    },
  },
  async ({ page }) => {
    await page.goto('https://example.com/login');
    await page.fill('#username', 'user@example.com');
    await page.fill('#password', 'password123');
    await page.click('button[type="submit"]');
    await expect(page).toHaveURL(/dashboard/);
  }
);

The type must be "test case" (case-insensitive)
The description must be the full URL to the test case in QA Sphere

Method 2: Hyphenated Markers in Test Names (Both JSON and JUnit XML)

Include a PROJECT-SEQUENCE marker anywhere in the test name:

// tests/login.spec.ts
import { test, expect } from '@playwright/test';

test('PRJ-312: Login with valid credentials', async ({ page }) => {
  await page.goto('https://example.com/login');
  await page.fill('#username', 'user@example.com');
  await page.fill('#password', 'password123');
  await page.click('button[type="submit"]');
  await expect(page).toHaveURL(/dashboard/);
});

test('PRJ-313: Login with invalid credentials', async ({ page }) => {
  await page.goto('https://example.com/login');
  await page.fill('#username', 'invalid@example.com');
  await page.fill('#password', 'wrongpassword');
  await page.click('button[type="submit"]');
  await expect(page.locator('.error')).toBeVisible();
});

The marker format is PROJECT-SEQUENCE (e.g., PRJ-312), where:

PROJECT is your QA Sphere project code
SEQUENCE is the test case sequence number (minimum 3 digits, zero-padded if needed)

Examples of valid test names:

PRJ-312: Login with valid credentials
Login with invalid credentials: PRJ-313

note

When using playwright-json-upload, only test annotations and hyphenated markers are supported. Hyphenless marker formats (underscore-separated and CamelCase) are only available with junit-upload.

Attachment Support

Playwright automatically captures screenshots, videos, and traces for failed tests when configured. The --attachments flag will automatically detect and upload these files to QA Sphere.

Supported Attachments

Screenshots: Captured on test failure (when screenshot: 'only-on-failure' is configured)
Videos: Retained on failure (when video: 'retain-on-failure' is configured)
Traces: Playwright trace files for debugging

Uploading Attachments

# Upload results with all attachments
npx qas-cli junit-upload --attachments junit-results/results.xml

The CLI will automatically:

Detect attachment files referenced in the JUnit XML
Match them to the corresponding test cases
Upload them to QA Sphere alongside test results

Troubleshooting

Common Issues

JUnit XML not found
```
Error: File not found: junit-results/results.xml
```
Solution: Ensure Playwright is configured with the JUnit reporter and the output path matches your upload command.
Test cases not matching
- Ensure your test names include the marker format (e.g., PRJ-312: Test name)
- Verify the marker matches an existing test case in QA Sphere
- Check that the project code in the marker matches your QA Sphere project
Attachments not uploading
- Verify --attachments flag is included
- Check that screenshots/videos are being generated (configure in playwright.config.js)
- Ensure attachment paths in JUnit XML are relative to the working directory

Getting Help

For help with any command, use the -h flag:

npx qas-cli junit-upload -h
npx qas-cli playwright-json-upload -h

Complete Example Workflow

Here's a complete example of running Playwright tests and uploading to QA Sphere:

Using Playwright JSON (Recommended)

# 1. Set up environment variables (or use .env/.qaspherecli file)
export QAS_TOKEN=your_token_here
export QAS_URL=https://qas.eu1.qasphere.com

# 2. Run Playwright tests
npx playwright test

# 3. Upload results to QA Sphere
npx qas-cli playwright-json-upload --attachments test-results/results.json

Using JUnit XML

# 1. Set up environment variables (or use .env/.qaspherecli file)
export QAS_TOKEN=your_token_here
export QAS_URL=https://qas.eu1.qasphere.com

# 2. Run Playwright tests
npx playwright test

# 3. Upload results to QA Sphere
npx qas-cli junit-upload --attachments junit-results/results.xml

Or as a single command with an npm script:

npm run test:upload

By following these guidelines, you can effectively integrate your Playwright test results with QA Sphere, streamlining your test management process.

Installation​

Prerequisites​

Installation Methods​

Configuration​

Playwright Configuration​

Option A: JSON Reporter (Recommended)​

Option B: JUnit XML Reporter​

Key Configuration Points​

Uploading Results​

Command Options​

Using playwright-json-upload (Recommended)​

Using junit-upload​

NPM Script Integration​

Linking Tests to QA Sphere Test Cases​

Method 1: Test Annotations (Recommended for Playwright JSON)​

Method 2: Hyphenated Markers in Test Names (Both JSON and JUnit XML)​

Attachment Support​

Supported Attachments​

Uploading Attachments​

Troubleshooting​

Common Issues​

Getting Help​

Complete Example Workflow​

Using Playwright JSON (Recommended)​

Using JUnit XML​