---
title: "Run tests"
---

> Documentation Index
> Fetch the complete documentation index at: https://docs.evidential.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Run tests

This is a collection of random tips about testing. Most of the test configurations you will use are already defined
in the [Taskfile](https://github.com/agency-fund/evidential-be/blob/main/Taskfile.yml).

## Unit Tests {#unit-tests}

Run unit tests with:

```shell
task test
```

We run unittests with [pytest](https://docs.pytest.org/en/stable/). Run `task --summary test` to see examples of how to run subsets of tests.

The `task test` helper automatically creates a local Postgres instance for testing and creates
testing data warehouses named "dwh" and "wide".

[Various tests](https://github.com/agency-fund/evidential-be/blob/main/.github/workflows/test.yaml) are also run as part of our GitHub Action test workflow.

[conftest.py](https://github.com/agency-fund/evidential-be/blob/main/src/xngin/apiserver/conftest.py) defines fixtures used by many of the tests.

## Google Integration Tests {#google-integration-tests}

Some of our pytests have a test marked as 'integration'. These are only usually run in GHA but you can run them locally
by setting `GOOGLE_APPLICATION_CREDENTIALS` and running:

```shell
uv run pytest -m integration
```

## BigQuery Integration Tests {#bigquery-integration-tests}

Method 1:

1. Push the branch to GitHub (i.e. draft PR)
1. Use the GitHub CLI to trigger the BigQuery integration tests on a specific branch:

```shell
gh workflow run tests --ref your-branch-name -f run-bq-integration=true
```

`tests` refers to the name of the workflow containing the integration tests. `run-bq-integration` refers to the name
of the workflow_dispatch input that determines if the BigQuery integration tests are run. The branch must already be
pushed to GitHub.

Method 2: Trigger via GitHub UI

1. Push the branch to GitHub (i.e. draft PR)
1. Go to the [Testing workflow](https://github.com/agency-fund/evidential-be/actions/workflows/test.yaml)
1. Click "new workflow"
1. Select the PR's branch name
1. Tick the "bq integration" box.

## Showing additional test output {#showing-additional-test-output}

`task test -- -rA` to print out _all_ stdout from your tests; `-rx` for just those failing. (See
[docs](https://docs.pytest.org/en/latest/how-to/output.html#producing-a-detailed-summary-report) for more info.)

## Loading the testing DWH with a different schema? {#loading-the-testing-dwh-with-a-different-schema}

Use the CLI command:

```shell
uv run xngin-cli create-testing-dwh \
   --dsn=$DATABASE_URL \
   --password=$PASSWORD \
   --table-name=test_participant_type \
   --schema-name=alt
```

### How can I run the unittests against an arbitrary data warehouse? {#how-can-i-run-the-unittests-that-use-my-pgbq-instance-as-the-test-dwh}

Tests that exercise integration with a data warehouse do so via two mechanisms:

1. xngin.testing.settings.json defines some static configuration for stable databases.
1. The XNGIN_QUERIES_TEST_URI environment variable defines a SQLAlchemy URI that some of the apiserver.dwh tests are run
    against.

By default, `task test` will run all the tests against Postgres. See the bq-integration GHA job for an example of how
to run some of the tests against BigQuery.

## Testing environment for BigQuery as the Service Provider {#testing-environment-for-bigquery-as-the-service-provider}

You can create a test dataset on a project you've configured with bigquery
using our xngin-cli tool:

```shell
export GSHEET_GOOGLE_APPLICATION_CREDENTIALS=~/.config/gspread/service_account.json
xngin-cli create-testing-dwh --dsn 'bigquery://xngin-development-dc/ds'
```

The tool is also used for interacting with the BQ API directly
(i.e. `bigquery_dataset_set_default_expiration`).

These commands use the process defined in Google's
[Application Default Credentials](https://cloud.google.com/docs/authentication/application-default-credentials).
You do not need the gcloud CLI or related tooling installed.

:::note
The GHA service account has permissions to access the xngin-development-dc.ds dataset.
:::

Source: https://docs.evidential.dev/develop/guides/run-tests/index.mdx
