TestProject Forum

I am building a automation framwork and I want to know if this is right for me

  1. Does Test Project have an easy local runtime?
    its really key that we can run/debug the tests locally, if we can only run them in the cloud that makes support and debugging largely unsustainable.
  2. Does Test Project integration with github since we are looking to land on circleCi thats quickly becoming a must as well ?
  3. ls Test Project light weight? if it takes a half hour for it to spin up thats a killer, we need to be able to run the tests quickly.

Thank you

Along with these questions Pros and Cons would be nice. I am new to the community and the project so anything helps. Thanks.

Hello @ernst

Happy to have you among our community! Great questions – let me elaborate on each:

1) Yes, You can execute your recorded tests locally, the TestProject Agent will detect the browsers/devices which are installed/connected on your machine so you’ll be able to configure which browser/device the test will be running on, in addition using TestProject’s OpenSDK you can develop, run and debug your coded tests locally. You will automatically receive all the real-time test reports in the TestProject Reports Dashboard without needing to upload anything to the cloud.

2) We have an integration with GitHub actions and as well with CircleCI. You can also read more here on how to use the TestProject OpenSDK within your CI/CD flow.

3) Yes, TestProject is lightweight. All you need to download & install the TestProject Agent (here’s a quick video about it, also explaining more on the Agent’s internals) – it’s a small & local executable that takes care of all the heavy lifting for you, bundles up best of breed Selenium & Appium for you – taking care of all the maintenance and setup for you, always up-to-date with the latest & stable drivers and make sure your tests run quickly. So you really do not need to worry about anything, only focus on creating tests :blush:

4) As for Pros and Cons – That’s a great question to ask your fellow community members here :wink: From our end, TestProject is full of pros! Here are a couple to name a few:

  • Cross platform support for Windows, macOS, Linux and Docker, and support for all targets: web, Android, iOS and API testing all within one single platform. Even iOS on Windows, without any Xcode dependencies. And for mobile testing, the support is for both physical devices as well as Emulators and Simulators (and with built-in integrations for device/browser farms as well, such as: Sauce Labs and BrowserStack).
  • Single open source OpenSDK that supports Java, Python and C# with 100% compatibility with Selenium and Appium, including execution of existing scripts developed in either framework. Also supporting BDD testing with plugins for Cucumber, SpecFlow and Behave.
  • One-click agent install and update, downloading and configuring all dependencies automatically, including Selenium, Appium, and other required components.
  • Out-of-the-box beautiful reporting, dashboards, and test analytics, with execution history and with automatic sharing via email and Slack. You can even export the reports into PDF and have RESTful API support.
  • Complete team collaboration experience with our Share Center, so you can easily collaborate with your entire team, wherever you are.
  • Centralized test framework including page object model, test data management, test storage, and more.
  • Easy integration with your DevOps toolchain, including integrations to Jenkins, CircleCI, TeamCity, Azure DevOps, GitHub, qTest, BrowserStack, Sauce Labs, and a fully documented RESTful API.
  • Quick and simple learning curve for testers who are just getting started, by using our AI-powered Test Recorder (with self healing, adaptive wait and an advanced Automation Assistant), and at the same time a perfect fit for expert automation developers since they can leverage our OpenSDK for local execution and simple deployment.

I also recommend exploring more over on our various channels:

And of course, the entire team and I are always here to help!
Feel free to reach out anytime.

3 Likes

Think you’ll be pleasantly surprised how powerful TestProject is.

I’ve a months+ experience with TP (we decided to use it as our chosen framework, after evaluated several others, including Functionize, Mabl, TestCafé Studio & Cypress) - and find it to be a really capable framework. And just keeps getting better (with AI, expanding SDK support, and upcoming Git support for recorded tests).

In regards to your questions:

  1. Tests are run on your own hardware using agents (which make handling selenium and appium drivers a cake-walk). You can deploy as many agents as you want, either as a locally installed executables or via Docker.
    It is also possible to forego agents, and run tests using BrowserStack / SauceLabs if you’re willing to pay for testruns in their cloud-based infrastructure.
  2. Git integration is supported for tests that are exported to, or created with, an SDK. If you’re looking to leverage recorded tests, currently Git support is only for exported tests. There is no way (yet) to record a test, export it, and then re-import it (f.ex. to use the recorder to maintain the test) - but this feature is slated to be released in Feb 2021.
  3. TestProject tests run quickly (at least the web-based one’s I’ve created)! They’re basically just selenium tests and run just about as quickly (only caveat is when using plugins - I notice that test steps using certain plugins can take a bit longer - but we’re talking about fractions of a second here, so not a deal breaker). I don’t have any experience with Appium (Android / iOS) test yet - but other forum members can certainly provide more info as to that.

Some key features: (very subjective - YMMV)
Element Library: Recorded tests build up a library of elements. When used prudently, this can allow an element used in many tests to be maintainable from a single object.
Recorder: The recorder is one of the better recorders I have tried - and allows you to debug tests “on the fly”. It also has useful tools for interacting with and exploring elements in the application under test.
Plugins: There are many useful plugins to choose from. There is also support for creating your own plugins.
Parameters: Easy to setup test parameters (per test and also shared for a whole project).
Nested tests: A test can consume another test (which itself can also include other nested tests)
Reports: The reports generated for tests are really nice. Easy to use and pretty informative
SDK: Test-Project has good SDK support. Java, C# & Python are supported (other languages are also underway).
Live Support: The level of support offered for a “Free Test Platform” is commendable. When necessary, support techs have helped me directly on-line (via Zoom). Kudos to them for this!

As to negatives (also fairly subjective)
Issues you may experience with TestProject are rather dependent on what your needs are. The main niggle I have with TestProject is this:
Documentation: There are lots of guides, but many are incomplete (some even out-dated). While amazingly simple to get started with, there are certain features that are not so intuitive. Documentation is pretty basic, and some features are poorly documented, if at all. The documentation also lags feature releases somewhat. If you’re gonna use TestProject, be prepared to dive into the Forum, and to use their Live Support. The community is an important part of the experience.

3 Likes

Thank you @nathan for sharing such detailed, informative, and kind feedback, it really means a lot to us!! You really are a TestProject expert :muscle:

As for the test execution speed while using plugins (Addons), it sometimes might add another second or so to the run time but we are working on making sure that won’t occur.

And regarding our Documentation,
appreciate the feedback – indeed the Forum here, the live support chat and our entire community is a great part of the experience and happy you are such an active part of it too!

In addition, the team indeed is continuously working on maintaining the Docs, making sure we keep it as up-to-date as possible, along with our fast release cycles :wink: Be sure to stay tuned as we continue improving and adding more and more informative content and guides to it – a lot is planned to be released and updated within this Q1 :rocket:

1 Like

Thank you for the input this was very helpful.