TestProject Forum

Tests passed but BrowserStack results are "unmarked"

I created tests using smart recorder for iOS app.
The are executing with success on BrowserStack. On TestProject dashboard tests are marked as “passed” but on BrowserStack they are marked as “unmarked”.
What should I do to pass the results to BrowserStack?

image

@katarzynaj

It should have been updated.
Looks like there is a bug and we will look into why it doesn’t work correctly.

Thanks. In TestProject dashboard I have:
image

Hi @katarzynaj,

Can you please provide more information about your tests?
Did you execute them inside a job or separately?
What is the agent version that you used?

Thanks,
Amit

Yes, they were executed inside a job.
I have virtual agent 3.0.5 + BrowserStack integration.

Hello TestProject team, @amit.yahav

I’ve observed the same issue in BrowserStack where TestProject Report is all tests in a job are Passed and BrowserStack status is “Unmarked” meaning, status is NOT updated to Passed. This is an intermittent issue as it has worked before and after the problem was observed.

Using AWS Workspace configured with the browserstack.local option. Execute a job in TestProject. Job is finished, TestProject correctly reports Passed; however, BrowserStack status is not updated.

2 hours later, perform the exact same steps, BrowserStack status is updated as expected.

My TestProject Agent version 3.3.0.

Please do let me know if I can provide additional information that would be helpful.

Staci

Hi @staci,

Thank you for providing us with this valuable information.
Can you please elaborate on the environment that you are using?
For example, what agent are you using? Is it a virtual agent or a physical one?

Are you executing recorded tests or coded tests?
Any special configuration that you are using with BrowserStack?

One last thing I wanted to ask you, could you please provide us the agent logs? You can send it directly in our in-app chat.

Thanks,
Amit

HI @amit.yahav

Sure! Following are more environment details.

  • Using AWS Workspace with a Windows 10 Skin (I believe AWS skins MS Data Center 2016 or similar)
  • Agent Version 3.3.0 is installed on the AWS Workspace and was the agent used when I observed the behavior. This means it would be physical agent (not using the BrowserStack virtual agent in this scenario)
  • I was executing recorded tests that are very simple, provide data to a registration page, click register, check a few things and add some additional configuration data to the new registered account. Meaning, nothing in the recorded test was rocket science, very straight forward.
  • The configuration I was using to run the tests is the browserstack.local option because my test environment is behind a firewall and not accessible with publicly by BrowserStack

I used the following guides to configure the Test Project and BrowserStack integration

  1. BrowserStack Integration - TestProject Documentation
  2. Run TestProject tests with Automate | BrowserStack Docs

Staci

Hi @staci,

Is it possible to get the agent logs? I need the logs on the date that it happened.
Also, the coded test would help me a lot.

You can send it directly in our chat.

Thanks,
Amit

@amit.yahav

I made a mistake in my first response. Tests were REcorded.

I am happy to provide the logs, but looks like I can only select last 5 days where this problem was 7 days ago

.

In the meantime, I’ll try to reproduce again.

Hi @staci,

If you can reproduce it, please send me the logs.

Thanks,
Amit