Generating a Custom Report for Playwright with Charts

4 min readSep 28, 2024

In software testing, generating clear and informative reports is crucial for understanding test results and facilitating decision-making. Playwright, a powerful framework for browser automation, provides flexibility in customizing reports to suit your needs. In this article, we’ll walk through creating a custom HTML report for Playwright tests that include charts to represent test results visually.

Why do we need Custom Reports?

Custom reports allow you to:

Visualize Data: Use charts to convey complex data in an easy-to-understand format.
Summarize Results: Quickly assess the status of test runs.
Enhance Clarity: Present detailed test steps alongside visual summaries.
Sharablity: We can customize on whats required to share for the management and stakeholders.

Prerequisites

Before we dive into the implementation, ensure you have the following:

Node.js and npm are installed on your machine.
Playwright installed in your project.

To install Playwright, run:

npm install playwright

Setting Up Your Custom Reporter

You’ll implement a custom reporter in Playwright to create a custom report. This reporter will generate an HTML file containing your test results and charts.

Create a Custom Reporter File: Create a new file named custom-reporter.js in your project.
Implement the Reporter: Here’s a sample implementation of a custom reporter:

const fs = require('fs');

class CustomReporter {
    constructor() {
        this.testResults = [];
    }

    onTestEnd(test, result) {
        const steps = result.steps.map(step => step.title).join(', ');
        this.testResults.push({
            name: test.title,
            steps: steps,
            status: result.status,
            duration: (result.duration / 1000).toFixed(2) + 's',
        });
    }


    
    generateHTMLReport() {
        const testsSummary = this.testResults.map((test, index) => `
    <tr>
      <td>${index + 1}</td>
      <td>${test.name}</td>
      <td>${test.steps}</td>
      <td>${test.status.charAt(0).toUpperCase() + test.status.slice(1)}</td>
      <td>${test.duration}</td>
    </tr>
  `).join('');

        const passed = this.testResults.filter(test => test.status === 'passed').length;
        const failed = this.testResults.filter(test => test.status === 'failed').length;
        const skipped = this.testResults.filter(test => test.status === 'skipped').length;
        const total = passed + failed + skipped;

        const htmlContent = `
    <!DOCTYPE html>
    <html lang="en">
    <head>
      <meta charset="UTF-8">
      <meta name="viewport" content="width=device-width, initial-scale=1.0">
      <title>Playwright Test Report</title>
      <style>
        body {
          font-family: Arial, sans-serif;
          margin: 20px;
          background-color: #f9f9f9;
        }
        .header-container {
          text-align: center;
          color: #333; /* Darker text for better visibility */
          padding: 10px;
          margin-bottom: 20px;
        }
        .chart-container {
          width: 30%;
          margin: 0 auto;
        }
        canvas {
          max-width: 100%;
          height: auto;
        }
        .table-container {
          width: 100%;
          margin-top: 20px;
        }
        table {
          width: 100%;
          border-collapse: collapse;
          background-color: #fff;
          border-radius: 8px;
          overflow: hidden;
        }
        table, th, td {
          border: 1px solid #ddd;
          padding: 8px;
        }
        th {
          background-color: #f2f2f2;
          text-align: left;
        }
        tr:nth-child(even) {
          background-color: #f9f9f9;
        }
        tr:hover {
          background-color: #f1f1f1;
        }
      </style>
    </head>
    <body>

      <div class="header-container">
        <h1>Test Results Summary</h1>
        <p>Total Tests: ${total} | Passed: ${passed} | Failed: ${failed} | Skipped: ${skipped}</p>
        <div class="chart-container">
          <canvas id="donutChart"></canvas>
        </div>
      </div>

      <div class="table-container">
        <table>
          <thead>
            <tr>
              <th>S.No</th>
              <th>Test Name</th>
              <th>Steps</th>
              <th>Status</th>
              <th>Time Taken</th>
            </tr>
          </thead>
          <tbody>
            ${testsSummary}
          </tbody>
        </table>
      </div>

      <script src="https://cdn.jsdelivr.net/npm/chart.js"></script>
      <script>
        const ctx = document.getElementById('donutChart').getContext('2d');
        const donutChart = new Chart(ctx, {
          type: 'doughnut',
          data: {
            labels: ['Passed', 'Failed', 'Skipped'],
            datasets: [{
              label: 'Test Results',
              data: [${passed}, ${failed}, ${skipped}],
              backgroundColor: ['#4CAF50', '#F44336', '#FFC107'],
              hoverOffset: 4
            }]
          },
          options: {
            responsive: true,
            maintainAspectRatio: false
          }
        });
      </script>
    </body>
    </html>
  `;

        fs.writeFileSync('test-report.html', htmlContent);
    }


    onEnd() {
        this.generateHTMLReport();
    }
}

module.exports = CustomReporter;

Explanation:

onEnd(result): This method is called when the test run ends. It collects test results and calls generateHTMLReport().
generateHTMLReport(testsSummary): This function generates an HTML report with a donut chart that summarizes the test results.

Integrate the Custom Reporter

To use your custom reporter, update your Playwright configuration file (playwright.config.js):

import { defineConfig, devices } from '@playwright/test';

/**
 * Read environment variables from file.
 * https://github.com/motdotla/dotenv
 */

/**
 * See https://playwright.dev/docs/test-configuration.
 */
export default defineConfig({
  testDir: './tests',
  /* Run tests in files in parallel */
  fullyParallel: true,
  /* Fail the build on CI if you accidentally left test.only in the source code. */
  forbidOnly: !!process.env.CI,
  /* Retry on CI only */
  retries: process.env.CI ? 2 : 0,
  /* Opt out of parallel tests on CI. */
  workers: process.env.CI ? 1 : undefined,
  /* Reporter to use. See https://playwright.dev/docs/test-reporters */
  reporter: [
    ['./custom-reporter.js']
  ],
  /* Shared settings for all the projects below. See https://playwright.dev/docs/api/class-testoptions. */
  use: {
    /* Base URL to use in actions like `await page.goto('/')`. */
    // baseURL: 'http://127.0.0.1:3000',

    /* Collect trace when retrying the failed test. See https://playwright.dev/docs/trace-viewer */
    trace: 'on-first-retry',
  },

  /* Configure projects for major browsers */
  projects: [
    {
      name: 'chromium',
      use: { ...devices['Desktop Chrome'] },
    },

    {
      name: 'firefox',
      use: { ...devices['Desktop Firefox'] },
    },

  ],
  
});

Run Your Tests

Now that your custom reporter is set up, run your Playwright tests as you normally would:

npx playwright test

After the tests finish running, an HTML report named test-report.html will be generated in your project directory.

View Your Report

Open the test-report.html file in a web browser. You should see a summary of your test results, including:

Total tests run, passed, failed, and skipped.
A donut chart visually represents the results.
A detailed table listing each test case, its steps, status, and duration.

Conclusion

Custom reporting in Playwright enhances test results analysis by providing clear visualizations and summaries. This is an example that you get most of the data for the results.

By following this guide, you can implement a custom HTML report with charts, making your testing efforts more transparent and actionable. Customize further based on your project requirements to enhance clarity and communication within your team!

I have created a project on Gitlab and added the code here.

Feel free to hit clap if you like the content. Happy Automation Testing :) Cheers. 👏