Skip to content

Integrate Evidential with your application

Documentation

The API reference documents every endpoint, with request samples and response examples. To try requests interactively, use the API’s own documentation at api.evidential.dev/docs.

Example Integration

Below describes steps typically taken for a client to integrate programmatically with Evidential as part of automating the execution of experiments in the client’s systems. Portion highlighted in light blue denote human interaction with the Evidential UI.

sequenceDiagram
  actor U as User
  participant E as Evidential
  participant CBE as Client Backend
  autonumber

  rect rgb(245, 255, 255)
      Note over E: via UI
      rect rgb(230, 255, 255)
          Note over U, E: initial setup
          U ->> E: Add Datasource + Participant Type
          U ->> E: add webhook
          U ->> E: add datasource APIKey
      end
      Note over U, E: experiment setup
      U ->> E: design and commit experiment
      alt for external assignment post-processing
          U ->> E: download CSV
      end

  end
  Note over E, CBE: Processing Webhook POST & Fetch Assignments
  E ->>+ CBE: notify via POST to webhook
  CBE -->>- E: status_code: 200
  CBE ->> E: fetch metadata <br> GET /v1/experiments/{experiment_id}
  alt bulk assignment fetch
      CBE ->> E: GET /v1/experiments/{experiment_id}/assignments
  else individual assignment fetch
      CBE ->> E: GET /v1/experiments/{experiment_id}/assignments/{participant_id}
      Note over E, CBE: for Online experiments, participants are assigned an arm if none yet
  end
  1. Configure a datasource and participant type

    Initial configuration of Datasource (data warehouse) and Participant Type (backed by a table in the data warehouse). One datasource configuration can be used by many experiments.

  2. Add a webhook

    Configuring a Webhook (in your Organization settings) is required to receive notifications of particular events. Your backend must expose an endpoint to which Evidential can POST data to, and should validate the Webhook-Token header contains a shared secret to ensure requests are legitimate.

  3. Create an API key

    Configuring an API Key (per Datasource; visit its details page) is required to make requests to Evidential’s backend to fetch experiment and assignment information. All API requests should have the custom headers Datasource-ID: and the generated X-API-Key: attached. These IDs are available via copy-to-clipboard buttons in the UI.

  4. Commit the experiment

    When a user commits an experiment, the client is notified via the configured webhook. For debugging purposes, the history of experiment.created webhooks is available in the Evidential UI.

  5. Export assignments as CSV (optional)

    For Preassigned experiments the user may immediately export a CSV version of the participant assignments and any strata for manual uploading to other systems or further post-processing or analysis. Other experiment types can also have their assignments exported this way as participants are added.

  6. Receive notifications

    The client’s backend system must have an “endpoint” that accepts a JSON payload POSTed to the registered webhook. A notification type’s body spec is described in the webhook’s details in the UI.

  7. Acknowledge the webhook

    The endpoint should return a HTTP response code indicating success (200-299). Users can look for webhook.sent in their Recent Events and see success/failure status. If the endpoint does not return a 2xx status code, Evidential will retry with an exponential backoff.

  8. Fetch experiment metadata

    It is recommended that the client system fetch the newly committed experiment’s information to facilitate execution on the client’s side. It is expected that at a minimum, the experiment’s start and end date could be used to signal when to enable the experiment and later shut it off. Furthermore, for an Online experiment, filters used to target eligible participants could be used to determine for whom to issue an assignment request.

  9. Look up assignments in bulk

    In addition to CSV exports from the UI, one can also export all assignments made so far via API as CSV or JSON. This allows a client’s backend to know which participant ids were (pre)assigned to which arms of an experiment. For example, prior to the start date of a Preassigned experiment, one could map the ids to phone numbers used in WhatsApp and mark those user profiles with the unique arm id assignments, so that the correct path is taken in the experiment’s chat flow.

  10. Look up individual assignments

    • For Preassigned experiments, one can look up an individual participant’s arm assignment given the experiment_id and the participant_id uniquely identifying it in your system. Requests for the same <experiment_id, participant_id> will return the same result.

    • For Online experiments, if the participant_id has not been seen yet, Evidential will randomly generate and return an arm assignment per the experiment’s design.

Additional developer notes

Request Encapsulation Middleware

Evidential’s API server includes middleware that allows clients to encapsulate their request payloads inside a wrapper object. This is useful when the client application you are using to send requests to Evidential has a fixed request format that cannot be changed to match Evidential’s expected schema.

For example, if the client application sends requests in the following format:

What your client sendsjson
{
  "data": {
    "payload": {
      "participant_id": "12345",
      "other_field": "value"
    }
  }
}

But the expected format by Evidential’s API is:

What Evidential expectsjson
{
  "participant_id": "12345",
  "other_field": "value"
}

The client can configure the middleware to extract the actual payload using a JSON Pointer path as follows:

path/to/endpoint?_unwrap=/data/payload

This will instruct the middleware to unwrap the request body and forward only the inner payload to the API endpoint.

Type to search…

↑↓ navigate↵ selectEsc close