Skip to main content

Documentation Index

Fetch the complete documentation index at: https://www.bugzy.ai/docs/llms.txt

Use this file to discover all available pages before exploring further.

Bugzy’s test-engineer subagent handles the full lifecycle of test code — generating new Playwright scripts, debugging failures, auto-fixing broken tests, and maintaining the suite as your application changes.

​
Test generation

When creating new tests, the test engineer:
  1. Reads test cases (from markdown files or an external TMS like Zephyr Scale)
  2. Generates Playwright TypeScript test scripts with proper assertions
  3. Creates Page Object classes for reusable interactions
  4. Adds appropriate waits, retries, and error handling
  5. Tags tests with @smoke, @regression, or custom tags
Generated tests follow your project’s existing patterns. If you have an established test suite, the agent reads your CLAUDE.md conventions and matches your style.

​
API testing

Bugzy supports testing REST API endpoints using Playwright’s built-in APIRequestContext — no additional framework needed. API tests use the { request } fixture instead of { page }, so no browser is launched (making them significantly faster). API testing capabilities include:
  • Endpoint discovery — during exploration, the agent looks for OpenAPI specs, route definitions, and API documentation
  • Test generation — produces API test cases alongside UI tests, using test_layer: api metadata
  • Authentication — supports Bearer tokens, API keys, and session-based auth patterns
  • Error validation — tests for proper 4xx/5xx responses, error payloads, and rate limiting
  • Mixed tests — combines API setup with UI verification in a single test (e.g., create data via API, verify in dashboard)
API tests use the same Playwright runner, Bugzy reporter, and test-runs/ output structure as UI tests.

​
Auto-fixing

When a test fails due to a test issue (not a product bug), the test engineer automatically:
1

Analyze the failure

Reads the error message, stack trace, and screenshots to identify the root cause.
2

Identify the fix

Determines whether the issue is a stale selector, a timing race, an outdated assertion, or a changed page structure.
3

Apply and validate

Modifies the test file, re-runs the test to confirm the fix works.
4

Retry if needed

If the first fix doesn’t work, tries an alternative approach — up to 3 total attempts.
5

Commit the fix

Once validated, commits the fix to the repo so the suite stays green.

​
Knowledge base

The test engineer builds and consults a knowledge-base.md file that captures:
  • Application structure and navigation patterns
  • Common selectors and interaction patterns
  • Known flaky areas and workarounds
  • Corrections from disputed findings
This knowledge base improves over time. Corrections from your team feed directly into future test generation and fixing decisions.

​
Server-side verification (SSH)

Bugzy can verify server-side state during test execution by connecting to your servers via SSH. This catches backend issues that browser-only testing misses — incorrect log entries, failed database mutations, misconfigured services, or missing file uploads. To enable SSH verification, add these environment variables to your project (Settings > Test Data > Environment Variables):
VariableValueSecret?
SSH_TEST_HOSTYour server hostname or IP (e.g., staging.example.com)No
SSH_TEST_USERSSH username (e.g., deploy)No
SSH_TEST_KEYYour SSH private key (full PEM block)Yes
SSH_TEST_PORTSSH port (optional, default: 22)No
  1. Create a dedicated read-only user on the target server (required): 
    # On your server (as root or sudo)
sudo useradd -m -s /bin/rbash bugzy-test
sudo mkdir -p /home/bugzy-test/.ssh
sudo chmod 700 /home/bugzy-test/.ssh
    Using rbash (restricted bash) as the shell prevents the user from running commands with / in the path, changing PATH, or using cd to navigate outside allowed directories.
  2. Restrict the user to read-only commands (required): 
    # Create a bin directory with only the commands Bugzy needs
sudo mkdir -p /home/bugzy-test/bin
sudo ln -s /usr/bin/cat /home/bugzy-test/bin/cat
sudo ln -s /usr/bin/grep /home/bugzy-test/bin/grep
sudo ln -s /usr/bin/tail /home/bugzy-test/bin/tail
sudo ln -s /usr/bin/head /home/bugzy-test/bin/head
sudo ln -s /usr/bin/ls /home/bugzy-test/bin/ls
sudo ln -s /usr/bin/df /home/bugzy-test/bin/df
sudo ln -s /usr/bin/free /home/bugzy-test/bin/free
sudo ln -s /usr/bin/uptime /home/bugzy-test/bin/uptime
sudo ln -s /usr/bin/curl /home/bugzy-test/bin/curl
sudo ln -s /usr/bin/psql /home/bugzy-test/bin/psql 2>/dev/null  # if installed
sudo ln -s /usr/bin/systemctl /home/bugzy-test/bin/systemctl 2>/dev/null

# Set this as the only PATH for the user
echo 'export PATH="$HOME/bin"' | sudo tee /home/bugzy-test/.bash_profile
sudo chmod 644 /home/bugzy-test/.bash_profile
sudo chown -R bugzy-test:bugzy-test /home/bugzy-test
    With rbash + a restricted PATH, the user can only run the commands you’ve symlinked. Commands like rm, kill, dd, or chmod are not available — the user literally cannot execute them.
  3. Generate a dedicated key pair for Bugzy testing: 
    # On your local machine
ssh-keygen -t ed25519 -C "bugzy-testing" -f bugzy_test_key -N ""
  4. Authorize the key on the server: 
    # Copy the public key to the server
cat bugzy_test_key.pub | ssh root@your-server \
  "sudo tee /home/bugzy-test/.ssh/authorized_keys && sudo chmod 600 /home/bugzy-test/.ssh/authorized_keys && sudo chown bugzy-test:bugzy-test /home/bugzy-test/.ssh/authorized_keys"
  5. Paste the private key (bugzy_test_key, not .pub) into the SSH_TEST_KEY environment variable in Bugzy. Include the full PEM block, from -----BEGIN OPENSSH PRIVATE KEY----- to -----END OPENSSH PRIVATE KEY-----.
  6. Verify the restriction works: 
    # These should work:
ssh -i bugzy_test_key bugzy-test@your-server "cat /var/log/app.log"
ssh -i bugzy_test_key bugzy-test@your-server "uptime"

# These should fail:
ssh -i bugzy_test_key bugzy-test@your-server "rm /tmp/test"       # command not found
ssh -i bugzy_test_key bugzy-test@your-server "/bin/rm /tmp/test"  # rbash blocks /
Do not skip steps 1-2. The Bugzy agent runs commands on your server autonomously. A restricted user with rbash and a limited PATH ensures the agent can only run read-only commands — even if it makes an unexpected decision. Never use root or an unrestricted user for SSH_TEST_USER.
  • Application logs (tail /var/log/app.log)
  • Database state (psql -c "SELECT * FROM orders ORDER BY id DESC LIMIT 1")
  • Service health (systemctl status myapp)
  • File uploads (ls -la /uploads/)
  • Internal API endpoints (curl http://localhost:3000/api/health)
  • System resources (df -h, free -m, uptime)
If your server is not reachable from the public internet, use mcp-tunnel to establish a secure connection from the Bugzy execution environment to your internal network.

​
File uploads in tests

Tests that involve file uploads (profile photos, CSV imports, PDF attachments) can use the Test Data file upload feature. Upload files through the dashboard’s Test Data page and reference them via environment variables: 
const filePath = process.env.PROFILE_PHOTO;
await page.locator('input[type="file"]').setInputFiles(filePath);
Files are stored securely in cloud storage and automatically downloaded to the test workspace before each run. Supported formats include images, PDFs, CSVs, spreadsheets, and text files (up to 10 MB each).

​
Test maintenance

As your application evolves, existing tests may break due to UI changes, new authentication flows, or restructured pages. The test engineer handles this during every test run:
  • Updates selectors when elements are moved or renamed
  • Adjusts assertions when expected behavior changes intentionally
  • Refactors Page Objects when page structure changes significantly

​
OAuth test authentication

Applications that use OAuth login (Google, GitHub, Microsoft) can be tested without automating the provider’s login page. Bugzy manages OAuth tokens automatically:
  1. Configure your OAuth app credentials in the Test Data page (client ID, client secret, scopes)
  2. Authorize with your test user account through a standard OAuth flow
  3. Run tests — Bugzy refreshes tokens automatically and injects them into the browser
The agent inspects your application to determine how it stores auth state (localStorage, cookies, or sessionStorage) and injects the token accordingly. If a token expires or is revoked, the dashboard prompts you to re-authorize. Supported providers: Google, GitHub, Microsoft. Multiple providers can be configured per project.

​
Learn more

Extend test coverage

The full test generation workflow from plan to automation.

Triage failures

How failures are classified and how auto-fix fits into triage.