Open
Conversation
In-memory SQLite database, httpx AsyncClient, and factory fixtures for all core models (commits, binaries, environments, runs, benchmark results, auth tokens, admin sessions). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Covers listing, pagination, get-by-id, 404 responses, python version filters, and binary-environment-commit relationships. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Covers benchmark name listing with filters, diff table with delta calculations, single and batch trend queries, and flamegraph retrieval. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Covers authenticated uploads, missing commit SHA, invalid binary and environment, configure flag mismatch, and memray failure reporting. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Covers Bearer and Token header formats, invalid tokens, missing tokens, and deactivated token rejection. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Public: maintainers listing, memray status (healthy and with failures). Logging: string/dict/list sanitization, token masking, safe log context. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add pytest-cov to dev dependencies. Configure coverage to use greenlet concurrency, fixing async coverage tracking. New upload tests: duplicate upload rejection, multi-binary uploads, memray failure update/ignore lifecycle, and upload clearing failures. Upload.py coverage: 22% -> 76%. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Compares changed files to detect backend modifications. On pushes to main, always runs if backend files changed. On PRs, compares against the base branch. Skips test setup entirely when only frontend or other files are modified. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Real benchmark data from the production database covering a deltablue_base memory regression (~10.5% high watermark increase) between two consecutive nogil commits, while json_dumps_base and nbody_base remain unchanged. Tests verify diff regression detection, previous commit metadata, alternative metrics, trends, batch trends, filtered benchmark names, and binary/environment relationships. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a backend testing harness and initial async API test suite for the FastAPI service (using httpx.AsyncClient + ASGI transport), along with coverage tooling and CI integration.
Changes:
- Introduces pytest/pytest-asyncio based tests for public, auth-protected, upload, and benchmark-related endpoints.
- Adds production-shaped fixtures and regression-focused tests for diff/trends behavior.
- Adds pytest configuration, coverage configuration, dev deps, docs updates, and a CI job to run backend tests with coverage.
Reviewed changes
Copilot reviewed 19 out of 20 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| backend/tests/conftest.py | Adds shared async fixtures (app, client, in-memory DB, sample models, auth/admin fixtures). |
| backend/tests/test_auth.py | Adds authentication-related tests for token handling. |
| backend/tests/test_upload.py | Adds upload + memray-failure endpoint tests (auth, validation, conflict cases). |
| backend/tests/test_public.py | Adds tests for public endpoints (maintainers, memray status). |
| backend/tests/test_production_data.py | Adds higher-fidelity tests using production-derived fixture shapes for diff/trends and filters. |
| backend/tests/production_fixtures.py | Provides production-derived fixture data for regression-oriented tests. |
| backend/tests/test_logging_utils.py | Adds unit tests for log sanitization/masking helpers. |
| backend/tests/test_health.py | Adds tests for /health endpoint shape/behavior. |
| backend/tests/test_environments.py | Adds tests for environment list/detail endpoints. |
| backend/tests/test_commits.py | Adds tests for commits endpoints and python-version filters. |
| backend/tests/test_binaries.py | Adds tests for binaries endpoints and related environment/commit listings. |
| backend/tests/test_benchmarks.py | Adds tests for benchmark name listing, diff, trends, batch trends, flamegraph. |
| backend/tests/init.py | Adds test package marker. |
| backend/pytest.ini | Configures pytest discovery and asyncio mode. |
| backend/.coveragerc | Configures coverage concurrency mode. |
| backend/requirements-dev.in | Adds pytest-cov to dev requirements input. |
| backend/requirements-dev.txt | Updates pinned dev lockfile to include coverage/pytest-cov. |
| backend/app/routers/upload.py | Broadens duplicate-run detection for IntegrityError handling. |
| README.md | Documents backend test/coverage commands and harness details. |
| .github/workflows/ci.yml | Adds a GitHub Actions job to run backend tests with coverage when backend changes. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
# Conflicts: # .github/workflows/ci.yml # README.md
Review fixes: - Use StaticPool for in-memory SQLite so all sessions share one connection, avoiding potential "no such table" errors. - Fix test_valid_bearer_token to actually test a protected endpoint with and without auth headers. - Assert ordering in trends test instead of building a dict. - Make health test explicit about the expected unhealthy DB status. - Add memray status assertion to test_upload_clears_memray_failure. Bug fix found by the new assertion: the upload endpoint's delete of memray failures was missing an await db.commit(), so the deletion was never persisted. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
5 tasks
This comment was marked as off-topic.
This comment was marked as off-topic.
Check e.orig.constraint_name first (available on asyncpg's UniqueViolationError), falling back to string matching for backends that don't expose it (e.g. SQLite in tests). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Well, they're "unit" to the extent to which hitting a service might be, but there's no separate running server. Everything's built around
httpx.AsyncClientand FastAPI's ASGI transport.Line coverage isn't perfect yet, but it's reasonable for the introduction of a testing harness.