List experiments on a data source
Authorization
Header parameters
Returns
200
Successful Response
- itemsarray ofobjectrequired
- design_specDesignSpecrequired
The type of assignment and experiment design.
}One of the following:
- AnyBanditDesignSpeccmab_online}
One of the following:
- CMABExperimentSpeccmab_online
- experiment_namestringrequiredmaxLength: 100
- descriptionstringrequiredmaxLength: 2000
- start_datestringrequiredformat: date-time
- end_datestringrequiredformat: date-time
- armsarray ofobjectrequired
- arm_namestringrequiredmaxLength: 100
- arm_idoptional
Unique ID of the arm. Leave unset when creating an arm.
}Any of the following:
- string
- null
- arm_descriptionoptional}
Any of the following:
- string
- null
- arm_weightoptional
Optional weight for this arm, used for unequal allocation. The weight must be greater than 0 and less than 100. If you set a weight, all arms must have weights, and they must sum to 100.
}Any of the following:
- ArmWeight
- null
- alpha_initoptional
Initial alpha parameter for the Beta prior.
}Any of the following:
- number
- null
- beta_initoptional
Initial beta parameter for the Beta prior.
}Any of the following:
- number
- null
- mu_initoptional
Initial mean parameter for the Normal prior.
}Any of the following:
- number
- null
- sigma_initoptional
Initial standard deviation parameter for the Normal prior.
}Any of the following:
- number
- null
- alphaoptional
Updated alpha parameter for the Beta prior.
}Any of the following:
- number
- null
- betaoptional
Updated beta parameter for the Beta prior.
}Any of the following:
- number
- null
- muoptional
Updated mean vector for the Normal prior.
}Any of the following:
- array<number>
- null
- covarianceoptional
Updated covariance matrix for the Normal prior.
}Any of the following:
- array<array<number>>
- null
- experiment_typeoptionalstringdefault: "cmab_online"
- design_urloptional
Optional URL to a more detailed experiment design doc.
}Any of the following:
- string
- null
- contextsoptional
Optional list of contexts that can be used to condition the assignment. Required for Contextual Multi-armed Bandit (CMAB) experiments.
}Any of the following:
- array<unknown>
- context_namestringrequiredmaxLength: 100
- context_idoptional
Unique identifier for the context. Do not set this when you create a new context.
}Any of the following:
- string
- null
- context_descriptionoptional}
Any of the following:
- string
- null
- value_typeoptionalbinaryorreal-valued
Type of value the context can take.
default: "binary"
- null
- prior_typeoptionalbetaornormal
The type of prior distribution for the arms.
default: "beta" - reward_typeoptionalbinaryorreal-valued
The type of outcome observed from the experiment.
default: "binary" - enable_autofailoptionalboolean
When true, Evidential records a failure for participants whose outcome does not arrive (via either push or pull) within a specified time window.
default: false - autofail_windowoptionalinteger
The time window in hours after which a participant is considered to have failed if no outcome arrives (via either push or pull). Default is 24 hours. Required if enable_autofail is true.
minimum: 1 · default: 24 - autofail_outcome_valueoptionalnumber
The outcome value to assign to participants who are automatically failed. Required if enable_autofail is true.
default: 0
- MABExperimentSpecmab_online
- experiment_namestringrequiredmaxLength: 100
- descriptionstringrequiredmaxLength: 2000
- start_datestringrequiredformat: date-time
- end_datestringrequiredformat: date-time
- armsarray ofobjectrequired
- arm_namestringrequiredmaxLength: 100
- arm_idoptional
Unique ID of the arm. Leave unset when creating an arm.
}Any of the following:
- string
- null
- arm_descriptionoptional}
Any of the following:
- string
- null
- arm_weightoptional
Optional weight for this arm, used for unequal allocation. The weight must be greater than 0 and less than 100. If you set a weight, all arms must have weights, and they must sum to 100.
}Any of the following:
- ArmWeight
- null
- alpha_initoptional
Initial alpha parameter for the Beta prior.
}Any of the following:
- number
- null
- beta_initoptional
Initial beta parameter for the Beta prior.
}Any of the following:
- number
- null
- mu_initoptional
Initial mean parameter for the Normal prior.
}Any of the following:
- number
- null
- sigma_initoptional
Initial standard deviation parameter for the Normal prior.
}Any of the following:
- number
- null
- alphaoptional
Updated alpha parameter for the Beta prior.
}Any of the following:
- number
- null
- betaoptional
Updated beta parameter for the Beta prior.
}Any of the following:
- number
- null
- muoptional
Updated mean vector for the Normal prior.
}Any of the following:
- array<number>
- null
- covarianceoptional
Updated covariance matrix for the Normal prior.
}Any of the following:
- array<array<number>>
- null
- experiment_typeoptionalstringdefault: "mab_online"
- design_urloptional
Optional URL to a more detailed experiment design doc.
}Any of the following:
- string
- null
- contextsoptional
Optional list of contexts that can be used to condition the assignment. Required for Contextual Multi-armed Bandit (CMAB) experiments.
}Any of the following:
- array<unknown>
- context_namestringrequiredmaxLength: 100
- context_idoptional
Unique identifier for the context. Do not set this when you create a new context.
}Any of the following:
- string
- null
- context_descriptionoptional}
Any of the following:
- string
- null
- value_typeoptionalbinaryorreal-valued
Type of value the context can take.
default: "binary"
- null
- prior_typeoptionalbetaornormal
The type of prior distribution for the arms.
default: "beta" - reward_typeoptionalbinaryorreal-valued
The type of outcome observed from the experiment.
default: "binary" - enable_autofailoptionalboolean
When true, Evidential records a failure for participants whose outcome does not arrive (via either push or pull) within a specified time window.
default: false - autofail_windowoptionalinteger
The time window in hours after which a participant is considered to have failed if no outcome arrives (via either push or pull). Default is 24 hours. Required if enable_autofail is true.
minimum: 1 · default: 24 - autofail_outcome_valueoptionalnumber
The outcome value to assign to participants who are automatically failed. Required if enable_autofail is true.
default: 0
- MABDwhExperimentSpecmab_online_dwh
- experiment_namestringrequiredmaxLength: 100
- descriptionstringrequiredmaxLength: 2000
- start_datestringrequiredformat: date-time
- end_datestringrequiredformat: date-time
- armsarray ofobjectrequired
- arm_namestringrequiredmaxLength: 100
- arm_idoptional
Unique ID of the arm. Leave unset when creating an arm.
}Any of the following:
- string
- null
- arm_descriptionoptional}
Any of the following:
- string
- null
- arm_weightoptional
Optional weight for this arm, used for unequal allocation. The weight must be greater than 0 and less than 100. If you set a weight, all arms must have weights, and they must sum to 100.
}Any of the following:
- ArmWeight
- null
- alpha_initoptional
Initial alpha parameter for the Beta prior.
}Any of the following:
- number
- null
- beta_initoptional
Initial beta parameter for the Beta prior.
}Any of the following:
- number
- null
- mu_initoptional
Initial mean parameter for the Normal prior.
}Any of the following:
- number
- null
- sigma_initoptional
Initial standard deviation parameter for the Normal prior.
}Any of the following:
- number
- null
- alphaoptional
Updated alpha parameter for the Beta prior.
}Any of the following:
- number
- null
- betaoptional
Updated beta parameter for the Beta prior.
}Any of the following:
- number
- null
- muoptional
Updated mean vector for the Normal prior.
}Any of the following:
- array<number>
- null
- covarianceoptional
Updated covariance matrix for the Normal prior.
}Any of the following:
- array<array<number>>
- null
- table_namestringrequired
Datasource table used to resolve target field metadata.
maxLength: 100 - primary_keystringrequired
Column name in table_name that uniquely identifies each participant.
maxLength: 300 · minLength: 1 - target_field_namestringrequired
Column name in table_name whose values this bandit optimises. Its data type is resolved from the DWH at create-time and used to validate subsequent outcome reports.
maxLength: 300 · minLength: 1 - experiment_typeoptionalstringdefault: "mab_online_dwh"
- design_urloptional
Optional URL to a more detailed experiment design doc.
}Any of the following:
- string
- null
- contextsoptional
Optional list of contexts that can be used to condition the assignment. Required for Contextual Multi-armed Bandit (CMAB) experiments.
}Any of the following:
- array<unknown>
- context_namestringrequiredmaxLength: 100
- context_idoptional
Unique identifier for the context. Do not set this when you create a new context.
}Any of the following:
- string
- null
- context_descriptionoptional}
Any of the following:
- string
- null
- value_typeoptionalbinaryorreal-valued
Type of value the context can take.
default: "binary"
- null
- prior_typeoptionalbetaornormal
The type of prior distribution for the arms.
default: "beta" - reward_typeoptionalbinaryorreal-valued
The type of outcome observed from the experiment.
default: "binary" - enable_autofailoptionalboolean
When true, Evidential records a failure for participants whose outcome does not arrive (via either push or pull) within a specified time window.
default: false - autofail_windowoptionalinteger
The time window in hours after which a participant is considered to have failed if no outcome arrives (via either push or pull). Default is 24 hours. Required if enable_autofail is true.
minimum: 1 · default: 24 - autofail_outcome_valueoptionalnumber
The outcome value to assign to participants who are automatically failed. Required if enable_autofail is true.
default: 0
- AnyFrequentistDesignSpecfreq_online}
One of the following:
- OnlineFrequentistExperimentSpecfreq_online
- experiment_namestringrequiredmaxLength: 100
- descriptionstringrequiredmaxLength: 2000
- start_datestringrequiredformat: date-time
- end_datestringrequiredformat: date-time
- armsarray ofobjectrequired
- arm_namestringrequiredmaxLength: 100
- arm_idoptional
Unique ID of the arm. Leave unset when creating an arm.
}Any of the following:
- string
- null
- arm_descriptionoptional}
Any of the following:
- string
- null
- arm_weightoptional
Optional weight for this arm, used for unequal allocation. The weight must be greater than 0 and less than 100. If you set a weight, all arms must have weights, and they must sum to 100.
}Any of the following:
- ArmWeight
- null
- table_namestringrequired
Data source table used to resolve participant field metadata.
maxLength: 100 - primary_keystringrequired
Column name in table_name that uniquely identifies each participant.
maxLength: 300 · minLength: 1 - strataarray ofobjectrequired
Optional fields to use for stratified assignment.
- field_namestringrequiredmaxLength: 300 · minLength: 1
- metricsarray ofobjectrequired
Primary and optional secondary metrics to target.
- field_namestringrequiredmaxLength: 300 · minLength: 1
- metric_pct_changeoptional
The smallest percent change, relative to metric_baseline, that you want to be able to detect. Cannot be set together with metric_target.
}Any of the following:
- number
- null
- metric_targetoptional
The absolute value you want to be able to detect. Cannot be set together with metric_pct_change.
}Any of the following:
- number
- null
- iccoptional
Intracluster correlation coefficient for cluster-randomized designs.
}Any of the following:
- number
- null
- avg_cluster_sizeoptional
Average number of participants per cluster.
}Any of the following:
- number
- null
- cvoptional
Coefficient of variation in cluster sizes (0 = equal sizes).
}Any of the following:
- number
- null
- metric_typeoptional
Inferred from the data warehouse column type.
}Any of the following:
- MetricType
- null
- metric_baselineoptional
Mean of the tracked metric.
}Any of the following:
- number
- null
- metric_stddevoptional
Standard deviation is set only for MetricType.NUMERIC metrics. Must be set for numeric metrics when available_n > 0.
}Any of the following:
- number
- null
- available_nonnull_noptional
The number of participants who meet the filtering criteria and have a non-null value for this metric.
}Any of the following:
- integer
- null
- available_noptional
The number of participants who meet the filtering criteria, whether or not this metric's value is NULL. Note: assignments are made from the available_n population. When available_n is greater than available_nonnull_n, some assigned participants may have a missing value for this metric.
}Any of the following:
- integer
- null
- use_one_time_metricoptionalboolean
Enable one-time metric mode: only assign participants with null values for this metric. Only allowed on the primary (first) metric of an individually-randomized preassigned experiment.
default: false
- filtersarray ofobjectrequired
Optional filters that narrow the eligible audience to the participants you want in the experiment.
- field_namestringrequiredmaxLength: 300 · minLength: 1
- relationincludesorexcludesorbetweenrequired
Defines operators for filtering values.
INCLUDES matches when the value matches any of the provided values, including null if explicitly specified. This is equivalent to NOT(EXCLUDES(values)).
EXCLUDES matches when the value does not match any of the provided values, including null if explicitly specified. If null is not explicitly excluded, we include nulls in the result.
BETWEEN matches when the value is between the two provided bounds (inclusive). Either bound can be null for an open-ended range. A null value matches only if a 3rd value of null is provided. BETWEEN applies to numeric, date and timestamp values.
- valueFilterValueTypesrequired}
Any of the following:
- array<unknown>}
Any of the following:
- integer
- null
- array<unknown>}
Any of the following:
- number
- null
- array<any of>}
Any of the following:
- string
- null
- array<any of>}
Any of the following:
- boolean
- null
- experiment_typeoptionalstringdefault: "freq_online"
- design_urloptional
Optional URL to a more detailed experiment design doc.
}Any of the following:
- string
- null
- desired_noptional
Desired number of individual participants. Required when creating individual-randomized preassigned experiments. For cluster-randomized preassigned experiment creation, use desired_n_clusters to control sampling; desired_n remains available for power calculations but is superseded by desired_n_clusters when both are set. When set, the power check also returns the minimum detectable effect for this size, along with the minimum sample size.
}Any of the following:
- integer
- null
- desired_nsoptional
Optional list of desired individual participant sample sizes. The power check returns the minimum detectable effect for each size (MetricPowerAnalysis.mde_curve), letting clients plot an MDE-vs-sample-size power curve from a single request. Superseded by desired_ns_clusters when both are set. Ignored when creating an experiment.
}Any of the following:
- array<integer>
- null
- poweroptionalnumber
The chance of detecting a real effect when one truly exists (1 - the false negative rate).
maximum: 1 · minimum: 0 · default: 0.8 - alphaoptionalnumber
The chance of a false positive: concluding there is an effect when there is none.
maximum: 1 · minimum: 0 · default: 0.05 - fstat_threshoptionalnumber
The p-value threshold for the balance check. When the balance check's p-value is above this threshold, we treat the assignment as "balanced".
maximum: 1 · minimum: 0 · default: 0.6
- PreassignedFrequentistExperimentSpecfreq_preassigned
- experiment_namestringrequiredmaxLength: 100
- descriptionstringrequiredmaxLength: 2000
- start_datestringrequiredformat: date-time
- end_datestringrequiredformat: date-time
- armsarray ofobjectrequired
- arm_namestringrequiredmaxLength: 100
- arm_idoptional
Unique ID of the arm. Leave unset when creating an arm.
}Any of the following:
- string
- null
- arm_descriptionoptional}
Any of the following:
- string
- null
- arm_weightoptional
Optional weight for this arm, used for unequal allocation. The weight must be greater than 0 and less than 100. If you set a weight, all arms must have weights, and they must sum to 100.
}Any of the following:
- ArmWeight
- null
- table_namestringrequired
Data source table used to resolve participant field metadata.
maxLength: 100 - primary_keystringrequired
Column name in table_name that uniquely identifies each participant.
maxLength: 300 · minLength: 1 - strataarray ofobjectrequired
Optional fields to use for stratified assignment.
- field_namestringrequiredmaxLength: 300 · minLength: 1
- metricsarray ofobjectrequired
Primary and optional secondary metrics to target.
- field_namestringrequiredmaxLength: 300 · minLength: 1
- metric_pct_changeoptional
The smallest percent change, relative to metric_baseline, that you want to be able to detect. Cannot be set together with metric_target.
}Any of the following:
- number
- null
- metric_targetoptional
The absolute value you want to be able to detect. Cannot be set together with metric_pct_change.
}Any of the following:
- number
- null
- iccoptional
Intracluster correlation coefficient for cluster-randomized designs.
}Any of the following:
- number
- null
- avg_cluster_sizeoptional
Average number of participants per cluster.
}Any of the following:
- number
- null
- cvoptional
Coefficient of variation in cluster sizes (0 = equal sizes).
}Any of the following:
- number
- null
- metric_typeoptional
Inferred from the data warehouse column type.
}Any of the following:
- MetricType
- null
- metric_baselineoptional
Mean of the tracked metric.
}Any of the following:
- number
- null
- metric_stddevoptional
Standard deviation is set only for MetricType.NUMERIC metrics. Must be set for numeric metrics when available_n > 0.
}Any of the following:
- number
- null
- available_nonnull_noptional
The number of participants who meet the filtering criteria and have a non-null value for this metric.
}Any of the following:
- integer
- null
- available_noptional
The number of participants who meet the filtering criteria, whether or not this metric's value is NULL. Note: assignments are made from the available_n population. When available_n is greater than available_nonnull_n, some assigned participants may have a missing value for this metric.
}Any of the following:
- integer
- null
- use_one_time_metricoptionalboolean
Enable one-time metric mode: only assign participants with null values for this metric. Only allowed on the primary (first) metric of an individually-randomized preassigned experiment.
default: false
- filtersarray ofobjectrequired
Optional filters that narrow the eligible audience to the participants you want in the experiment.
- field_namestringrequiredmaxLength: 300 · minLength: 1
- relationincludesorexcludesorbetweenrequired
Defines operators for filtering values.
INCLUDES matches when the value matches any of the provided values, including null if explicitly specified. This is equivalent to NOT(EXCLUDES(values)).
EXCLUDES matches when the value does not match any of the provided values, including null if explicitly specified. If null is not explicitly excluded, we include nulls in the result.
BETWEEN matches when the value is between the two provided bounds (inclusive). Either bound can be null for an open-ended range. A null value matches only if a 3rd value of null is provided. BETWEEN applies to numeric, date and timestamp values.
- valueFilterValueTypesrequired}
Any of the following:
- array<unknown>}
Any of the following:
- integer
- null
- array<unknown>}
Any of the following:
- number
- null
- array<any of>}
Any of the following:
- string
- null
- array<any of>}
Any of the following:
- boolean
- null
- experiment_typeoptionalstringdefault: "freq_preassigned"
- design_urloptional
Optional URL to a more detailed experiment design doc.
}Any of the following:
- string
- null
- desired_noptional
Desired number of individual participants. Required when creating individual-randomized preassigned experiments. For cluster-randomized preassigned experiment creation, use desired_n_clusters to control sampling; desired_n remains available for power calculations but is superseded by desired_n_clusters when both are set. When set, the power check also returns the minimum detectable effect for this size, along with the minimum sample size.
}Any of the following:
- integer
- null
- desired_nsoptional
Optional list of desired individual participant sample sizes. The power check returns the minimum detectable effect for each size (MetricPowerAnalysis.mde_curve), letting clients plot an MDE-vs-sample-size power curve from a single request. Superseded by desired_ns_clusters when both are set. Ignored when creating an experiment.
}Any of the following:
- array<integer>
- null
- poweroptionalnumber
The chance of detecting a real effect when one truly exists (1 - the false negative rate).
maximum: 1 · minimum: 0 · default: 0.8 - alphaoptionalnumber
The chance of a false positive: concluding there is an effect when there is none.
maximum: 1 · minimum: 0 · default: 0.05 - fstat_threshoptionalnumber
The p-value threshold for the balance check. When the balance check's p-value is above this threshold, we treat the assignment as "balanced".
maximum: 1 · minimum: 0 · default: 0.6 - cluster_keyoptional
Column name in table_name that identifies clusters for a cluster-randomized design. When set, per-metric icc, avg_cluster_size, and cv are either supplied on each metric or computed from this column at power_check time. When None, the design is assumed to be individual-randomized.
}Any of the following:
- string
- null
- desired_n_clustersoptional
Desired number of clusters to sample when creating a cluster-randomized preassigned experiment. Required and only valid when cluster_key is set. All eligible participants in each sampled cluster are included. In power checks, the minimum detectable effect is computed for this many clusters, converted to a per-metric sample size using each metric's avg_cluster_size; takes precedence over desired_n.
}Any of the following:
- integer
- null
- desired_ns_clustersoptional
Optional list of desired cluster counts. Only valid when cluster_key is set. The power check returns the minimum detectable effect for each count (MetricPowerAnalysis.mde_curve), converted to a per-metric sample size using each metric's avg_cluster_size; takes precedence over desired_ns. Ignored when creating an experiment.
}Any of the following:
- array<integer>
- null
- AnyFrequentistDesignSpecfreq_preassigned}
One of the following:
- OnlineFrequentistExperimentSpecfreq_online
- experiment_namestringrequiredmaxLength: 100
- descriptionstringrequiredmaxLength: 2000
- start_datestringrequiredformat: date-time
- end_datestringrequiredformat: date-time
- armsarray ofobjectrequired
- arm_namestringrequiredmaxLength: 100
- arm_idoptional
Unique ID of the arm. Leave unset when creating an arm.
}Any of the following:
- string
- null
- arm_descriptionoptional}
Any of the following:
- string
- null
- arm_weightoptional
Optional weight for this arm, used for unequal allocation. The weight must be greater than 0 and less than 100. If you set a weight, all arms must have weights, and they must sum to 100.
}Any of the following:
- ArmWeight
- null
- table_namestringrequired
Data source table used to resolve participant field metadata.
maxLength: 100 - primary_keystringrequired
Column name in table_name that uniquely identifies each participant.
maxLength: 300 · minLength: 1 - strataarray ofobjectrequired
Optional fields to use for stratified assignment.
- field_namestringrequiredmaxLength: 300 · minLength: 1
- metricsarray ofobjectrequired
Primary and optional secondary metrics to target.
- field_namestringrequiredmaxLength: 300 · minLength: 1
- metric_pct_changeoptional
The smallest percent change, relative to metric_baseline, that you want to be able to detect. Cannot be set together with metric_target.
}Any of the following:
- number
- null
- metric_targetoptional
The absolute value you want to be able to detect. Cannot be set together with metric_pct_change.
}Any of the following:
- number
- null
- iccoptional
Intracluster correlation coefficient for cluster-randomized designs.
}Any of the following:
- number
- null
- avg_cluster_sizeoptional
Average number of participants per cluster.
}Any of the following:
- number
- null
- cvoptional
Coefficient of variation in cluster sizes (0 = equal sizes).
}Any of the following:
- number
- null
- metric_typeoptional
Inferred from the data warehouse column type.
}Any of the following:
- MetricType
- null
- metric_baselineoptional
Mean of the tracked metric.
}Any of the following:
- number
- null
- metric_stddevoptional
Standard deviation is set only for MetricType.NUMERIC metrics. Must be set for numeric metrics when available_n > 0.
}Any of the following:
- number
- null
- available_nonnull_noptional
The number of participants who meet the filtering criteria and have a non-null value for this metric.
}Any of the following:
- integer
- null
- available_noptional
The number of participants who meet the filtering criteria, whether or not this metric's value is NULL. Note: assignments are made from the available_n population. When available_n is greater than available_nonnull_n, some assigned participants may have a missing value for this metric.
}Any of the following:
- integer
- null
- use_one_time_metricoptionalboolean
Enable one-time metric mode: only assign participants with null values for this metric. Only allowed on the primary (first) metric of an individually-randomized preassigned experiment.
default: false
- filtersarray ofobjectrequired
Optional filters that narrow the eligible audience to the participants you want in the experiment.
- field_namestringrequiredmaxLength: 300 · minLength: 1
- relationincludesorexcludesorbetweenrequired
Defines operators for filtering values.
INCLUDES matches when the value matches any of the provided values, including null if explicitly specified. This is equivalent to NOT(EXCLUDES(values)).
EXCLUDES matches when the value does not match any of the provided values, including null if explicitly specified. If null is not explicitly excluded, we include nulls in the result.
BETWEEN matches when the value is between the two provided bounds (inclusive). Either bound can be null for an open-ended range. A null value matches only if a 3rd value of null is provided. BETWEEN applies to numeric, date and timestamp values.
- valueFilterValueTypesrequired}
Any of the following:
- array<unknown>}
Any of the following:
- integer
- null
- array<unknown>}
Any of the following:
- number
- null
- array<any of>}
Any of the following:
- string
- null
- array<any of>}
Any of the following:
- boolean
- null
- experiment_typeoptionalstringdefault: "freq_online"
- design_urloptional
Optional URL to a more detailed experiment design doc.
}Any of the following:
- string
- null
- desired_noptional
Desired number of individual participants. Required when creating individual-randomized preassigned experiments. For cluster-randomized preassigned experiment creation, use desired_n_clusters to control sampling; desired_n remains available for power calculations but is superseded by desired_n_clusters when both are set. When set, the power check also returns the minimum detectable effect for this size, along with the minimum sample size.
}Any of the following:
- integer
- null
- desired_nsoptional
Optional list of desired individual participant sample sizes. The power check returns the minimum detectable effect for each size (MetricPowerAnalysis.mde_curve), letting clients plot an MDE-vs-sample-size power curve from a single request. Superseded by desired_ns_clusters when both are set. Ignored when creating an experiment.
}Any of the following:
- array<integer>
- null
- poweroptionalnumber
The chance of detecting a real effect when one truly exists (1 - the false negative rate).
maximum: 1 · minimum: 0 · default: 0.8 - alphaoptionalnumber
The chance of a false positive: concluding there is an effect when there is none.
maximum: 1 · minimum: 0 · default: 0.05 - fstat_threshoptionalnumber
The p-value threshold for the balance check. When the balance check's p-value is above this threshold, we treat the assignment as "balanced".
maximum: 1 · minimum: 0 · default: 0.6
- PreassignedFrequentistExperimentSpecfreq_preassigned
- experiment_namestringrequiredmaxLength: 100
- descriptionstringrequiredmaxLength: 2000
- start_datestringrequiredformat: date-time
- end_datestringrequiredformat: date-time
- armsarray ofobjectrequired
- arm_namestringrequiredmaxLength: 100
- arm_idoptional
Unique ID of the arm. Leave unset when creating an arm.
}Any of the following:
- string
- null
- arm_descriptionoptional}
Any of the following:
- string
- null
- arm_weightoptional
Optional weight for this arm, used for unequal allocation. The weight must be greater than 0 and less than 100. If you set a weight, all arms must have weights, and they must sum to 100.
}Any of the following:
- ArmWeight
- null
- table_namestringrequired
Data source table used to resolve participant field metadata.
maxLength: 100 - primary_keystringrequired
Column name in table_name that uniquely identifies each participant.
maxLength: 300 · minLength: 1 - strataarray ofobjectrequired
Optional fields to use for stratified assignment.
- field_namestringrequiredmaxLength: 300 · minLength: 1
- metricsarray ofobjectrequired
Primary and optional secondary metrics to target.
- field_namestringrequiredmaxLength: 300 · minLength: 1
- metric_pct_changeoptional
The smallest percent change, relative to metric_baseline, that you want to be able to detect. Cannot be set together with metric_target.
}Any of the following:
- number
- null
- metric_targetoptional
The absolute value you want to be able to detect. Cannot be set together with metric_pct_change.
}Any of the following:
- number
- null
- iccoptional
Intracluster correlation coefficient for cluster-randomized designs.
}Any of the following:
- number
- null
- avg_cluster_sizeoptional
Average number of participants per cluster.
}Any of the following:
- number
- null
- cvoptional
Coefficient of variation in cluster sizes (0 = equal sizes).
}Any of the following:
- number
- null
- metric_typeoptional
Inferred from the data warehouse column type.
}Any of the following:
- MetricType
- null
- metric_baselineoptional
Mean of the tracked metric.
}Any of the following:
- number
- null
- metric_stddevoptional
Standard deviation is set only for MetricType.NUMERIC metrics. Must be set for numeric metrics when available_n > 0.
}Any of the following:
- number
- null
- available_nonnull_noptional
The number of participants who meet the filtering criteria and have a non-null value for this metric.
}Any of the following:
- integer
- null
- available_noptional
The number of participants who meet the filtering criteria, whether or not this metric's value is NULL. Note: assignments are made from the available_n population. When available_n is greater than available_nonnull_n, some assigned participants may have a missing value for this metric.
}Any of the following:
- integer
- null
- use_one_time_metricoptionalboolean
Enable one-time metric mode: only assign participants with null values for this metric. Only allowed on the primary (first) metric of an individually-randomized preassigned experiment.
default: false
- filtersarray ofobjectrequired
Optional filters that narrow the eligible audience to the participants you want in the experiment.
- field_namestringrequiredmaxLength: 300 · minLength: 1
- relationincludesorexcludesorbetweenrequired
Defines operators for filtering values.
INCLUDES matches when the value matches any of the provided values, including null if explicitly specified. This is equivalent to NOT(EXCLUDES(values)).
EXCLUDES matches when the value does not match any of the provided values, including null if explicitly specified. If null is not explicitly excluded, we include nulls in the result.
BETWEEN matches when the value is between the two provided bounds (inclusive). Either bound can be null for an open-ended range. A null value matches only if a 3rd value of null is provided. BETWEEN applies to numeric, date and timestamp values.
- valueFilterValueTypesrequired}
Any of the following:
- array<unknown>}
Any of the following:
- integer
- null
- array<unknown>}
Any of the following:
- number
- null
- array<any of>}
Any of the following:
- string
- null
- array<any of>}
Any of the following:
- boolean
- null
- experiment_typeoptionalstringdefault: "freq_preassigned"
- design_urloptional
Optional URL to a more detailed experiment design doc.
}Any of the following:
- string
- null
- desired_noptional
Desired number of individual participants. Required when creating individual-randomized preassigned experiments. For cluster-randomized preassigned experiment creation, use desired_n_clusters to control sampling; desired_n remains available for power calculations but is superseded by desired_n_clusters when both are set. When set, the power check also returns the minimum detectable effect for this size, along with the minimum sample size.
}Any of the following:
- integer
- null
- desired_nsoptional
Optional list of desired individual participant sample sizes. The power check returns the minimum detectable effect for each size (MetricPowerAnalysis.mde_curve), letting clients plot an MDE-vs-sample-size power curve from a single request. Superseded by desired_ns_clusters when both are set. Ignored when creating an experiment.
}Any of the following:
- array<integer>
- null
- poweroptionalnumber
The chance of detecting a real effect when one truly exists (1 - the false negative rate).
maximum: 1 · minimum: 0 · default: 0.8 - alphaoptionalnumber
The chance of a false positive: concluding there is an effect when there is none.
maximum: 1 · minimum: 0 · default: 0.05 - fstat_threshoptionalnumber
The p-value threshold for the balance check. When the balance check's p-value is above this threshold, we treat the assignment as "balanced".
maximum: 1 · minimum: 0 · default: 0.6 - cluster_keyoptional
Column name in table_name that identifies clusters for a cluster-randomized design. When set, per-metric icc, avg_cluster_size, and cv are either supplied on each metric or computed from this column at power_check time. When None, the design is assumed to be individual-randomized.
}Any of the following:
- string
- null
- desired_n_clustersoptional
Desired number of clusters to sample when creating a cluster-randomized preassigned experiment. Required and only valid when cluster_key is set. All eligible participants in each sampled cluster are included. In power checks, the minimum detectable effect is computed for this many clusters, converted to a per-metric sample size using each metric's avg_cluster_size; takes precedence over desired_n.
}Any of the following:
- integer
- null
- desired_ns_clustersoptional
Optional list of desired cluster counts. Only valid when cluster_key is set. The power check returns the minimum detectable effect for each count (MetricPowerAnalysis.mde_curve), converted to a per-metric sample size using each metric's avg_cluster_size; takes precedence over desired_ns. Ignored when creating an experiment.
}Any of the following:
- array<integer>
- null
- AnyBanditDesignSpecmab_online}
One of the following:
- CMABExperimentSpeccmab_online
- experiment_namestringrequiredmaxLength: 100
- descriptionstringrequiredmaxLength: 2000
- start_datestringrequiredformat: date-time
- end_datestringrequiredformat: date-time
- armsarray ofobjectrequired
- arm_namestringrequiredmaxLength: 100
- arm_idoptional
Unique ID of the arm. Leave unset when creating an arm.
}Any of the following:
- string
- null
- arm_descriptionoptional}
Any of the following:
- string
- null
- arm_weightoptional
Optional weight for this arm, used for unequal allocation. The weight must be greater than 0 and less than 100. If you set a weight, all arms must have weights, and they must sum to 100.
}Any of the following:
- ArmWeight
- null
- alpha_initoptional
Initial alpha parameter for the Beta prior.
}Any of the following:
- number
- null
- beta_initoptional
Initial beta parameter for the Beta prior.
}Any of the following:
- number
- null
- mu_initoptional
Initial mean parameter for the Normal prior.
}Any of the following:
- number
- null
- sigma_initoptional
Initial standard deviation parameter for the Normal prior.
}Any of the following:
- number
- null
- alphaoptional
Updated alpha parameter for the Beta prior.
}Any of the following:
- number
- null
- betaoptional
Updated beta parameter for the Beta prior.
}Any of the following:
- number
- null
- muoptional
Updated mean vector for the Normal prior.
}Any of the following:
- array<number>
- null
- covarianceoptional
Updated covariance matrix for the Normal prior.
}Any of the following:
- array<array<number>>
- null
- experiment_typeoptionalstringdefault: "cmab_online"
- design_urloptional
Optional URL to a more detailed experiment design doc.
}Any of the following:
- string
- null
- contextsoptional
Optional list of contexts that can be used to condition the assignment. Required for Contextual Multi-armed Bandit (CMAB) experiments.
}Any of the following:
- array<unknown>
- context_namestringrequiredmaxLength: 100
- context_idoptional
Unique identifier for the context. Do not set this when you create a new context.
}Any of the following:
- string
- null
- context_descriptionoptional}
Any of the following:
- string
- null
- value_typeoptionalbinaryorreal-valued
Type of value the context can take.
default: "binary"
- null
- prior_typeoptionalbetaornormal
The type of prior distribution for the arms.
default: "beta" - reward_typeoptionalbinaryorreal-valued
The type of outcome observed from the experiment.
default: "binary" - enable_autofailoptionalboolean
When true, Evidential records a failure for participants whose outcome does not arrive (via either push or pull) within a specified time window.
default: false - autofail_windowoptionalinteger
The time window in hours after which a participant is considered to have failed if no outcome arrives (via either push or pull). Default is 24 hours. Required if enable_autofail is true.
minimum: 1 · default: 24 - autofail_outcome_valueoptionalnumber
The outcome value to assign to participants who are automatically failed. Required if enable_autofail is true.
default: 0
- MABExperimentSpecmab_online
- experiment_namestringrequiredmaxLength: 100
- descriptionstringrequiredmaxLength: 2000
- start_datestringrequiredformat: date-time
- end_datestringrequiredformat: date-time
- armsarray ofobjectrequired
- arm_namestringrequiredmaxLength: 100
- arm_idoptional
Unique ID of the arm. Leave unset when creating an arm.
}Any of the following:
- string
- null
- arm_descriptionoptional}
Any of the following:
- string
- null
- arm_weightoptional
Optional weight for this arm, used for unequal allocation. The weight must be greater than 0 and less than 100. If you set a weight, all arms must have weights, and they must sum to 100.
}Any of the following:
- ArmWeight
- null
- alpha_initoptional
Initial alpha parameter for the Beta prior.
}Any of the following:
- number
- null
- beta_initoptional
Initial beta parameter for the Beta prior.
}Any of the following:
- number
- null
- mu_initoptional
Initial mean parameter for the Normal prior.
}Any of the following:
- number
- null
- sigma_initoptional
Initial standard deviation parameter for the Normal prior.
}Any of the following:
- number
- null
- alphaoptional
Updated alpha parameter for the Beta prior.
}Any of the following:
- number
- null
- betaoptional
Updated beta parameter for the Beta prior.
}Any of the following:
- number
- null
- muoptional
Updated mean vector for the Normal prior.
}Any of the following:
- array<number>
- null
- covarianceoptional
Updated covariance matrix for the Normal prior.
}Any of the following:
- array<array<number>>
- null
- experiment_typeoptionalstringdefault: "mab_online"
- design_urloptional
Optional URL to a more detailed experiment design doc.
}Any of the following:
- string
- null
- contextsoptional
Optional list of contexts that can be used to condition the assignment. Required for Contextual Multi-armed Bandit (CMAB) experiments.
}Any of the following:
- array<unknown>
- context_namestringrequiredmaxLength: 100
- context_idoptional
Unique identifier for the context. Do not set this when you create a new context.
}Any of the following:
- string
- null
- context_descriptionoptional}
Any of the following:
- string
- null
- value_typeoptionalbinaryorreal-valued
Type of value the context can take.
default: "binary"
- null
- prior_typeoptionalbetaornormal
The type of prior distribution for the arms.
default: "beta" - reward_typeoptionalbinaryorreal-valued
The type of outcome observed from the experiment.
default: "binary" - enable_autofailoptionalboolean
When true, Evidential records a failure for participants whose outcome does not arrive (via either push or pull) within a specified time window.
default: false - autofail_windowoptionalinteger
The time window in hours after which a participant is considered to have failed if no outcome arrives (via either push or pull). Default is 24 hours. Required if enable_autofail is true.
minimum: 1 · default: 24 - autofail_outcome_valueoptionalnumber
The outcome value to assign to participants who are automatically failed. Required if enable_autofail is true.
default: 0
- MABDwhExperimentSpecmab_online_dwh
- experiment_namestringrequiredmaxLength: 100
- descriptionstringrequiredmaxLength: 2000
- start_datestringrequiredformat: date-time
- end_datestringrequiredformat: date-time
- armsarray ofobjectrequired
- arm_namestringrequiredmaxLength: 100
- arm_idoptional
Unique ID of the arm. Leave unset when creating an arm.
}Any of the following:
- string
- null
- arm_descriptionoptional}
Any of the following:
- string
- null
- arm_weightoptional
Optional weight for this arm, used for unequal allocation. The weight must be greater than 0 and less than 100. If you set a weight, all arms must have weights, and they must sum to 100.
}Any of the following:
- ArmWeight
- null
- alpha_initoptional
Initial alpha parameter for the Beta prior.
}Any of the following:
- number
- null
- beta_initoptional
Initial beta parameter for the Beta prior.
}Any of the following:
- number
- null
- mu_initoptional
Initial mean parameter for the Normal prior.
}Any of the following:
- number
- null
- sigma_initoptional
Initial standard deviation parameter for the Normal prior.
}Any of the following:
- number
- null
- alphaoptional
Updated alpha parameter for the Beta prior.
}Any of the following:
- number
- null
- betaoptional
Updated beta parameter for the Beta prior.
}Any of the following:
- number
- null
- muoptional
Updated mean vector for the Normal prior.
}Any of the following:
- array<number>
- null
- covarianceoptional
Updated covariance matrix for the Normal prior.
}Any of the following:
- array<array<number>>
- null
- table_namestringrequired
Datasource table used to resolve target field metadata.
maxLength: 100 - primary_keystringrequired
Column name in table_name that uniquely identifies each participant.
maxLength: 300 · minLength: 1 - target_field_namestringrequired
Column name in table_name whose values this bandit optimises. Its data type is resolved from the DWH at create-time and used to validate subsequent outcome reports.
maxLength: 300 · minLength: 1 - experiment_typeoptionalstringdefault: "mab_online_dwh"
- design_urloptional
Optional URL to a more detailed experiment design doc.
}Any of the following:
- string
- null
- contextsoptional
Optional list of contexts that can be used to condition the assignment. Required for Contextual Multi-armed Bandit (CMAB) experiments.
}Any of the following:
- array<unknown>
- context_namestringrequiredmaxLength: 100
- context_idoptional
Unique identifier for the context. Do not set this when you create a new context.
}Any of the following:
- string
- null
- context_descriptionoptional}
Any of the following:
- string
- null
- value_typeoptionalbinaryorreal-valued
Type of value the context can take.
default: "binary"
- null
- prior_typeoptionalbetaornormal
The type of prior distribution for the arms.
default: "beta" - reward_typeoptionalbinaryorreal-valued
The type of outcome observed from the experiment.
default: "binary" - enable_autofailoptionalboolean
When true, Evidential records a failure for participants whose outcome does not arrive (via either push or pull) within a specified time window.
default: false - autofail_windowoptionalinteger
The time window in hours after which a participant is considered to have failed if no outcome arrives (via either push or pull). Default is 24 hours. Required if enable_autofail is true.
minimum: 1 · default: 24 - autofail_outcome_valueoptionalnumber
The outcome value to assign to participants who are automatically failed. Required if enable_autofail is true.
default: 0
- AnyBanditDesignSpecmab_online_dwh}
One of the following:
- CMABExperimentSpeccmab_online
- experiment_namestringrequiredmaxLength: 100
- descriptionstringrequiredmaxLength: 2000
- start_datestringrequiredformat: date-time
- end_datestringrequiredformat: date-time
- armsarray ofobjectrequired
- arm_namestringrequiredmaxLength: 100
- arm_idoptional
Unique ID of the arm. Leave unset when creating an arm.
}Any of the following:
- string
- null
- arm_descriptionoptional}
Any of the following:
- string
- null
- arm_weightoptional
Optional weight for this arm, used for unequal allocation. The weight must be greater than 0 and less than 100. If you set a weight, all arms must have weights, and they must sum to 100.
}Any of the following:
- ArmWeight
- null
- alpha_initoptional
Initial alpha parameter for the Beta prior.
}Any of the following:
- number
- null
- beta_initoptional
Initial beta parameter for the Beta prior.
}Any of the following:
- number
- null
- mu_initoptional
Initial mean parameter for the Normal prior.
}Any of the following:
- number
- null
- sigma_initoptional
Initial standard deviation parameter for the Normal prior.
}Any of the following:
- number
- null
- alphaoptional
Updated alpha parameter for the Beta prior.
}Any of the following:
- number
- null
- betaoptional
Updated beta parameter for the Beta prior.
}Any of the following:
- number
- null
- muoptional
Updated mean vector for the Normal prior.
}Any of the following:
- array<number>
- null
- covarianceoptional
Updated covariance matrix for the Normal prior.
}Any of the following:
- array<array<number>>
- null
- experiment_typeoptionalstringdefault: "cmab_online"
- design_urloptional
Optional URL to a more detailed experiment design doc.
}Any of the following:
- string
- null
- contextsoptional
Optional list of contexts that can be used to condition the assignment. Required for Contextual Multi-armed Bandit (CMAB) experiments.
}Any of the following:
- array<unknown>
- context_namestringrequiredmaxLength: 100
- context_idoptional
Unique identifier for the context. Do not set this when you create a new context.
}Any of the following:
- string
- null
- context_descriptionoptional}
Any of the following:
- string
- null
- value_typeoptionalbinaryorreal-valued
Type of value the context can take.
default: "binary"
- null
- prior_typeoptionalbetaornormal
The type of prior distribution for the arms.
default: "beta" - reward_typeoptionalbinaryorreal-valued
The type of outcome observed from the experiment.
default: "binary" - enable_autofailoptionalboolean
When true, Evidential records a failure for participants whose outcome does not arrive (via either push or pull) within a specified time window.
default: false - autofail_windowoptionalinteger
The time window in hours after which a participant is considered to have failed if no outcome arrives (via either push or pull). Default is 24 hours. Required if enable_autofail is true.
minimum: 1 · default: 24 - autofail_outcome_valueoptionalnumber
The outcome value to assign to participants who are automatically failed. Required if enable_autofail is true.
default: 0
- MABExperimentSpecmab_online
- experiment_namestringrequiredmaxLength: 100
- descriptionstringrequiredmaxLength: 2000
- start_datestringrequiredformat: date-time
- end_datestringrequiredformat: date-time
- armsarray ofobjectrequired
- arm_namestringrequiredmaxLength: 100
- arm_idoptional
Unique ID of the arm. Leave unset when creating an arm.
}Any of the following:
- string
- null
- arm_descriptionoptional}
Any of the following:
- string
- null
- arm_weightoptional
Optional weight for this arm, used for unequal allocation. The weight must be greater than 0 and less than 100. If you set a weight, all arms must have weights, and they must sum to 100.
}Any of the following:
- ArmWeight
- null
- alpha_initoptional
Initial alpha parameter for the Beta prior.
}Any of the following:
- number
- null
- beta_initoptional
Initial beta parameter for the Beta prior.
}Any of the following:
- number
- null
- mu_initoptional
Initial mean parameter for the Normal prior.
}Any of the following:
- number
- null
- sigma_initoptional
Initial standard deviation parameter for the Normal prior.
}Any of the following:
- number
- null
- alphaoptional
Updated alpha parameter for the Beta prior.
}Any of the following:
- number
- null
- betaoptional
Updated beta parameter for the Beta prior.
}Any of the following:
- number
- null
- muoptional
Updated mean vector for the Normal prior.
}Any of the following:
- array<number>
- null
- covarianceoptional
Updated covariance matrix for the Normal prior.
}Any of the following:
- array<array<number>>
- null
- experiment_typeoptionalstringdefault: "mab_online"
- design_urloptional
Optional URL to a more detailed experiment design doc.
}Any of the following:
- string
- null
- contextsoptional
Optional list of contexts that can be used to condition the assignment. Required for Contextual Multi-armed Bandit (CMAB) experiments.
}Any of the following:
- array<unknown>
- context_namestringrequiredmaxLength: 100
- context_idoptional
Unique identifier for the context. Do not set this when you create a new context.
}Any of the following:
- string
- null
- context_descriptionoptional}
Any of the following:
- string
- null
- value_typeoptionalbinaryorreal-valued
Type of value the context can take.
default: "binary"
- null
- prior_typeoptionalbetaornormal
The type of prior distribution for the arms.
default: "beta" - reward_typeoptionalbinaryorreal-valued
The type of outcome observed from the experiment.
default: "binary" - enable_autofailoptionalboolean
When true, Evidential records a failure for participants whose outcome does not arrive (via either push or pull) within a specified time window.
default: false - autofail_windowoptionalinteger
The time window in hours after which a participant is considered to have failed if no outcome arrives (via either push or pull). Default is 24 hours. Required if enable_autofail is true.
minimum: 1 · default: 24 - autofail_outcome_valueoptionalnumber
The outcome value to assign to participants who are automatically failed. Required if enable_autofail is true.
default: 0
- MABDwhExperimentSpecmab_online_dwh
- experiment_namestringrequiredmaxLength: 100
- descriptionstringrequiredmaxLength: 2000
- start_datestringrequiredformat: date-time
- end_datestringrequiredformat: date-time
- armsarray ofobjectrequired
- arm_namestringrequiredmaxLength: 100
- arm_idoptional
Unique ID of the arm. Leave unset when creating an arm.
}Any of the following:
- string
- null
- arm_descriptionoptional}
Any of the following:
- string
- null
- arm_weightoptional
Optional weight for this arm, used for unequal allocation. The weight must be greater than 0 and less than 100. If you set a weight, all arms must have weights, and they must sum to 100.
}Any of the following:
- ArmWeight
- null
- alpha_initoptional
Initial alpha parameter for the Beta prior.
}Any of the following:
- number
- null
- beta_initoptional
Initial beta parameter for the Beta prior.
}Any of the following:
- number
- null
- mu_initoptional
Initial mean parameter for the Normal prior.
}Any of the following:
- number
- null
- sigma_initoptional
Initial standard deviation parameter for the Normal prior.
}Any of the following:
- number
- null
- alphaoptional
Updated alpha parameter for the Beta prior.
}Any of the following:
- number
- null
- betaoptional
Updated beta parameter for the Beta prior.
}Any of the following:
- number
- null
- muoptional
Updated mean vector for the Normal prior.
}Any of the following:
- array<number>
- null
- covarianceoptional
Updated covariance matrix for the Normal prior.
}Any of the following:
- array<array<number>>
- null
- table_namestringrequired
Datasource table used to resolve target field metadata.
maxLength: 100 - primary_keystringrequired
Column name in table_name that uniquely identifies each participant.
maxLength: 300 · minLength: 1 - target_field_namestringrequired
Column name in table_name whose values this bandit optimises. Its data type is resolved from the DWH at create-time and used to validate subsequent outcome reports.
maxLength: 300 · minLength: 1 - experiment_typeoptionalstringdefault: "mab_online_dwh"
- design_urloptional
Optional URL to a more detailed experiment design doc.
}Any of the following:
- string
- null
- contextsoptional
Optional list of contexts that can be used to condition the assignment. Required for Contextual Multi-armed Bandit (CMAB) experiments.
}Any of the following:
- array<unknown>
- context_namestringrequiredmaxLength: 100
- context_idoptional
Unique identifier for the context. Do not set this when you create a new context.
}Any of the following:
- string
- null
- context_descriptionoptional}
Any of the following:
- string
- null
- value_typeoptionalbinaryorreal-valued
Type of value the context can take.
default: "binary"
- null
- prior_typeoptionalbetaornormal
The type of prior distribution for the arms.
default: "beta" - reward_typeoptionalbinaryorreal-valued
The type of outcome observed from the experiment.
default: "binary" - enable_autofailoptionalboolean
When true, Evidential records a failure for participants whose outcome does not arrive (via either push or pull) within a specified time window.
default: false - autofail_windowoptionalinteger
The time window in hours after which a participant is considered to have failed if no outcome arrives (via either push or pull). Default is 24 hours. Required if enable_autofail is true.
minimum: 1 · default: 24 - autofail_outcome_valueoptionalnumber
The outcome value to assign to participants who are automatically failed. Required if enable_autofail is true.
default: 0
- power_analysesrequired}
Any of the following:
- PowerResponse
- analysesarray ofobjectrequired
- metric_specobjectrequired
Defines a metric to measure in an experiment with its baseline stats.
Adds the fields only the server can derive from the data, so request and response metrics stay separate types (and separate OpenAPI schemas).
- field_namestringrequiredmaxLength: 300 · minLength: 1
- metric_pct_changeoptional
Percent change target relative to the metric_baseline.
}Any of the following:
- number
- null
- metric_targetoptional
Absolute target value = metric_baseline * (1 + metric_pct_change).
}Any of the following:
- number
- null
- iccoptional
Intracluster correlation coefficient for cluster-randomized designs.
}Any of the following:
- number
- null
- avg_cluster_sizeoptional
Average number of participants per cluster.
}Any of the following:
- number
- null
- cvoptional
Coefficient of variation in cluster sizes (0 = equal sizes).
}Any of the following:
- number
- null
- metric_typeoptional
Inferred from the data warehouse column type.
}Any of the following:
- MetricType
- null
- metric_baselineoptional
Mean of the tracked metric.
}Any of the following:
- number
- null
- metric_stddevoptional
Standard deviation is set only for MetricType.NUMERIC metrics. Must be set for numeric metrics when available_n > 0.
}Any of the following:
- number
- null
- available_nonnull_noptional
The number of participants who meet the filtering criteria and have a non-null value for this metric.
}Any of the following:
- integer
- null
- available_noptional
The number of participants who meet the filtering criteria, whether or not this metric's value is NULL. Note: assignments are made from the available_n population. When available_n is greater than available_nonnull_n, some assigned participants may have a missing value for this metric.
}Any of the following:
- integer
- null
- use_one_time_metricoptionalboolean
Enable one-time metric mode: only assign participants with null values for this metric. Only allowed on the primary (first) metric of an individually-randomized preassigned experiment.
default: false - is_one_time_eligibleoptionalboolean
Set by the power check: True if the metric is eligible for one-time mode (it is the primary metric and its column has both null and non-null values).
default: false
- target_noptional
Minimum sample size needed to meet the design specs.
}Any of the following:
- integer
- null
- sufficient_noptional
Whether there are enough available participants to sample to reach target_n.
}Any of the following:
- boolean
- null
- target_possibleoptional
When the sample size is too small to reach the desired metric_target, this is the target that is possible given available_n, in the improvement (positive) direction from the baseline. It is the absolute counterpart of pct_change_possible. None when the sample size is large enough to detect the desired change, or (for BINARY metrics) when no value above the baseline is a large enough change to reach the requested power.
}Any of the following:
- number
- null
- pct_change_possibleoptional
When the sample size is too small to reach the desired metric_pct_change, this is the percent change that is possible given available_n. It is the relative counterpart of target_possible. None whenever target_possible is None.
}Any of the following:
- number
- null
- target_possible_loweroptional
The detectable metric value below the baseline. (target_possible is the metric value above the baseline.) Both detectable metric values above and below the baseline use the chosen confidence and power, and are always reported together, regardless of the direction of the requested change. For numeric metrics, this is the same distance below the baseline as target_possible is above it. However, for binary metrics, an increase and a decrease that are equally detectable can sit at different distances from the baseline. None when the sample size is large enough to detect the desired change, or (for BINARY metrics) when no value below the baseline is a large enough change to reach the requested power.
}Any of the following:
- number
- null
- pct_change_possible_loweroptional
The relative counterpart of target_possible_lower, as a percent change from the baseline (negative for a decrease). A value of -0.10 means a 10% decrease. None whenever target_possible_lower is None.
}Any of the following:
- number
- null
- pct_change_with_desired_noptional
The minimum detectable effect (MDE) achievable for the desired sample size at the chosen confidence and power, reported in the direction of the requested change (negative when the requested change is a decrease from the baseline). Present only when design_spec.desired_n or design_spec.desired_n_clusters is set (frequentist design specs). When desired_n_clusters is set, the desired sample size is desired_n_clusters times this metric's avg_cluster_size.
}Any of the following:
- number
- null
- mde_curveoptional
The MDE for each requested sample size, in the same order as design_spec.desired_ns (or desired_ns_clusters, which takes precedence). Present only when one of those is set. Each point is computed best-effort: a size where the calculation fails yields a null pct_change instead of failing the request.
}Any of the following:
- array<unknown>
- desired_nintegerrequired
Sample size in individual participants used for this point. For cluster-randomized designs this is desired_n_clusters times the metric's avg_cluster_size.
- desired_n_clustersoptional
The requested number of clusters for this point. None for individual-randomized requests.
}Any of the following:
- integer
- null
- pct_changeoptional
The minimum detectable effect (MDE) at this sample size, as a percent change relative to metric_baseline. None when the power calculation fails at this size (e.g. too small to solve).
}Any of the following:
- number
- null
- null
- msgoptional
A human-friendly summary of the power analysis results.
}Any of the following:
- MetricPowerAnalysisMessage
- null
- num_clusters_totaloptional
Total number of clusters needed across all arms.
}Any of the following:
- integer
- null
- clusters_per_armoptional
Number of clusters needed for each arm (one entry per arm).
}Any of the following:
- array<integer>
- null
- n_per_armoptional
Number of participants for each arm (one entry per arm).
}Any of the following:
- array<integer>
- null
- design_effectoptional
Design effect (DEFF), the clustering penalty multiplier.
}Any of the following:
- number
- null
- effective_sample_sizeoptional
Effective sample size accounting for clustering (total_n / DEFF).
}Any of the following:
- integer
- null
- null
- assign_summaryrequired}
Any of the following:
- AssignSummary
- sample_sizeintegerrequired
The number of participants across all arms in total.
- balance_checkoptional
Balance test results if available. Online experiments do not have balance checks.
}Any of the following:
- BalanceCheck
- null
- arm_sizesoptional
For each arm, the number of participants (and optionally clusters) assigned.
}Any of the following:
- array<unknown>
- armobjectrequired
Describes an experiment treatment arm.
- arm_namestringrequiredmaxLength: 100
- arm_idoptional
Unique ID of the arm. Leave unset when creating an arm.
}Any of the following:
- string
- null
- arm_descriptionoptional}
Any of the following:
- string
- null
- arm_weightoptional
Optional weight for this arm, used for unequal allocation. The weight must be greater than 0 and less than 100. If you set a weight, all arms must have weights, and they must sum to 100.
}Any of the following:
- ArmWeight
- null
- sizeoptionalintegerdefault: 0
- cluster_countoptional
The number of clusters assigned to this arm. Set only on cluster-randomized experiments.
}Any of the following:
- integer
- null
- null
- null
- impactoptionalhighormediumorlowornegativeorunclearor
Given the results across your tracked metrics and any other observed effects seen elsewhere, record an overall summary here. Do they agree or reject your hypotheses? Beyond the metrics tracked, did variations affect scalability, cost, feedback, or other aspects?
default: ""
400
The request is invalid. Check that the request body matches the required structure and that all required fields and headers are present.
403
You did not provide a valid API key, or your API key does not have permission for this operation.
Tip: Check that the API key in the X-API-Key header has access to the data source or experiment.
404
The resource was not found, or you do not have access to it.
Tip: Check that the API key in the X-API-Key header has access to the data source or experiment.