Test parameter on test scenario overriding the same parameter in a data source

I have a test scenario that uses test parameters for assertions.
I usually leave the test parameter default values empty, as I know these will be populated by a data source when the test scenario is executed.
I use multiple data sources for different test environments.

However, I decided to populate the test parameters on the test scenario as a fallback option, so the tests can at least run successfully for one environment if the data sources don’t work.
The default values I have provided will work for one particular test environment, but not for others as all environments contain different data (hence the use of data sources).

When I execute the test, I select the data source for the tests to use.
I have noticed that one test parameter is causing a failure in the test, if it is populated on the scenario as well as in the data source. Both values are different, with the correct data being in the data source.
When I empty the value on the test scenario, the test passes.

This shouldn’t be happening, because, as long as the test parameter is specified in the data source, it doesn’t matter what the test parameter on the scenario contains. The data source is the first source of test data the test looks at. This is true, because there are other test parameter values on both the same test scenario and data source (again, both with different data) and another step in the test passes before the one in question fails.
I can see this in the test results screenshot.
Also, if the test parameter was incorrectly configured in the data source, the test wouldn’t pass even when the parameter on the scenario is empty.

It may also be worth noting that the test scenario in question is a nested step within another test scenario, but that shouldn’t make any difference.

1 Like

I think I have identified the cause of the bug.

The test parameter exists on the test scenario, but it does not exist on the encapsulating test scenario that contains the former as a nested step.

The test parameter I described which works correctly, exists on the test scenario (populated), encapsulating test scenario (empty) and the data source (populated).

So I think the bug is, when an encapsulating scenario does not have the same test parameter imported from the actual test scenario for whatever reason (e.g. accidentally removed).
It should not be prioritising the parameter on the actual test scenario over the data source.

1 Like

Hi @james.storey, thanks for sharing this. It sounds like you’re reporting a potential bug relating to test fails and parameters. I noticed this is in general questions. I don’t think there is a forum category specific for bugs, so I will move this to site feedback which is the closest thing. It may be worth passing this on directly to TestProject to log on their system (if they don’t get around to seeing this).

Hi Philip,

What’s the best way to directly inform the TestProject team of the bug?

1 Like

I normally contact them intercom, you should see a button at the bottom right of TestProject:

We have an open improvement for this (TP-16429).
This is a feature request because the current logic in place is to not refactor all the parameters in the main tests and it’s not a bug.
I will let you know here in-case this will get resolved.

This topic was automatically closed after 180 days. New replies are no longer allowed.