TestProject Forum

API : Why does a data source override specified data parameters? [SOLVED]

In our organisation, we use Terraform to create ephemeral QA environments, where the application URLs on that QA environment include the JIRA ticket number, to keep each QA environment’s application URLs unique.
Each QA environment has several web applications, each with its own application URL.
One QA environment would be:

Another QA environment would be:

Each of these applications has its own test data, and we use a separate TestProject data source to accommodate for each application.

We use Jenkins jobs to trigger our TestProject job, which in turn, executes a suite of tests for each application, on each QA environment.

Because TestProject doesn’t know in advance what the QA environment application URL will be (it’s not a fixed URL and changes with each QA environment), we cannot specify it as a default value on a test scenario, nor within a data source. Instead, we need to specify the Application URL in Jenkins’ execution parameters as a property of data, which will override any existing Application URL set on the test scenario or in the data source.
We also need to specify the dataSourceId in the execution parameters, so each application (primary, secondary, …) knows what additional sets of test data to use.

I would expect my execution parameters in Jenkins to look something like this:

    "testParameters": [
            "dataSourceId": "<MY_DATA_SOURCE_ID>",
            "data": [
                    "ApplicationURL": "<ENV_VARIABLE_THAT_POPULATES_WHICH_URL_TO_USE>"

The problem is, setting the ApplicationURL in the data parameter does not override the value in the data source or on the test scenario:

  • dataSourceId (optional) : A GUID of a data source to be used for this execution (will be applied to tests based on testId and testPosition ). If provided, will override the data property. If some parameters are missing from the file, defaults will be used.

In fact, it is the opposite. providing dataSourceId in the API request will override any property provided in data. Even if I don’t specify an application URL in the data source, it still ignores what I provide in data, and relies on the default value in the test scenario (which will only ever run the tests for one URL).

Why is it this way? It doesn’t make sense to me. Surely, it should be the other way around, where any properties provided in data will override those that are either contained within the data source (which should be used as a fallback option in case they are not provided within data) and override the default parameters specified within the test scenario.

This is by design, when executing a test with a data source it will only take into consideration the data source (as overriding values), what you are asking to do is to override the same parameter twice, and this logic is not in-place currently.
If you need to override the value of a specific parameter you can easily pass this by using a project parameter as project parameters are not affected by a data source. you can also include a step in your test that takes the project parameter and sets it in a test parameter value this test parameter can also be included in the data source file, with a simple if condition logic you can then decide whether you want to use the value inputted from the project parameter into the test parameter or to use the default value the data source assign to the parameter by default, it can look something like this.

Assigning the value:

if condition is met:


Now testURI will have a value from the data source that you can override from the job API only if a certain condition is met.