Not sure what went over worse, this decision or Elon gutting Twitter.
You know what would make this tool better? If everyone on my team of 12 could only use it less than twice a month.
-no one ever
4 Likes
Postman is a great tool for beginners looking to get into automation.
The basics can be easily grasped by non technical team members too and it has a good helpful community.
The local collection runner is a very important feature and itās easy to go through 25 runs in a few minutes.
The enterprise licenses are extremely expensive for smaller organizations and teams to make a case for. Additionally ,with a change like this, we donāt know what else could be limited in the future.
I hope this decision could be overturned somehow or at least the limits scaled up to more reasonable numbers because it feels it will otherwise alienate a lot of users.
10 Likes
Iāve been using Postman for 3 years. Great tool. Iāve just joined a new company and got them to start using Postman last week. Then Iāve discovered this new collection run limit that is going to be imposed. This will make Postman virtually unusable for any test driven development unless you have the top tier license. As such, its no longer an option and Iāve been told to consider alternatives. Insomnia here I comeā¦
2 Likes
@cryosat-cosmologist @docking-module-part1 If possible, I think @norrispostman Code-First is the ultimate solution, If you donāt want to be locked in or forced to accept āgreat ideasā, after all something free is probably the most expensive thing, everything that is free will eventually become something commercial if they can, no one can always give selflessly all the time including open source project that I understand, just the rationality of commercializationā¦
This is such a shame. I was on the brink of adoption, but this limit and being in a small business just means we have to seek out alternatives now. I am already running 35 basic tests in the matter of 1 minute, so this limit makes the app sadly unusable. I hope Postman rethink this decision.
@swatchbookemma This limit is not on running individual tests but on running entire collections. If you run a collection that contains 35 tests it counts as 1 run.
In case you need to run these tests on a regular basis, have you looked at using Scheduled Collection Runs?
1 Like
Oh I understand. Thank you for clarifying that.
1 Like
But i think the bigger problem for us is that by the nature of nested collections with many folders, if you want to run any subset folder of a collection that utilizes any setNextRequest or shared parent pre-request, the collection runner is the easiest and most straight forward way to do it.
I have hardly ever run my entire collection from the top level. But while debugging or creating a new subset of tests in that collection, its used often.
Also can you confirm that the collection runner run count is shared across the entire team, not for individual users? Iām currently on a team with 121 people on it (company wide). If thatās the case then even if everyone had professional it would be useless. We would have to break up our team in to very narrow silos to gain any value.
And maybe this is the way its intended to be used. I wasnāt there when the reqs were made, but iām guessing Postman helped guide us to our current configuration. it will be a mess to reorg.
For us this waters down the benefit of having workspaces stored in the cloud, if we canāt see each others workspaces because we are siloed in different teams.
Also is it true that users with different plans (basic, professional, enterprise) cannot be on the same team?
If so, Test Engineers would likely want more runs, but a dev engineer may never hit the button at all. Why make us all be the same level for different user needs?
1 Like
Thanks for the suggested tool. SoapUI is great as well.
I am also not a fan of the new license restrictions. We have a team where 4 testers have a Basic license, and 6 developers a free account. The testers create and change the collections, the devās only run them (using the collection runner). Itās mostly used to prepare test data for follow-up manual testing.
Our estimated use is about 200 runs/month per tester, 50 runs/month per dev. Itās clear that this single metric of ānumber of collection runs per month for the entire teamā will require us to upgrade from Basic to Enterprise, AND to require 6 additional licenses. Going from $576 a year to $11,880 a year is quite the increaseā¦ Especially when you consider we donāt need any of the additional functionalities or other metrics (API calls, custom domain, integrations, ā¦). Itās just this single metric that kills the licensing.
We donāt mind paying for useful tools. I would totally understand if we need to get a Basic license for all our developers. I would totally understand if we need to upgrade to Pro for our heavy users (testers). However, both will not work because every single colleague on its own exceeds the allowed collection runs.
With these tight restrictions, we are unable to find a suitable licensing plan within reasonable budget. We have used Postman for years with great satisfaction, and have many collections that we canāt just replace in a few weeks. So weāve decided to take up the offer āupgrade to Pro and get unlimited collection runs the first yearā. But during this year we will be migrating to a different tool, Postman is unfortunately no longer dependable to provide the services we need.
8 Likes
@mathijs_h The new limits will only kick in when your renewal is up. So if you renew for a year before 15th of March then youāll still get year worth of unlimited runs, even on the basic plan.
At first, I thought I misunderstood the announcement about the local runner - but itās seriousā¦ The local execution of the Runner will be counted. I donāt understand why they defined limits which are so low. 25 local runs for basic is a joke, 250 runs for prof. could become scarce during intensive development phases. This are bad news for us developers who use Postman to test the application during the development. And I find it unreasonable to limit a local execution.
We will now try to change the subscription (basic to prof). But the taste is so bitter, that we plan to leave postman this year. Iam sad about subscription decsion of postman.
4 Likes
@sebastian.strefel.d As mentioned in the blog, if you renew your plan before the 15th of March youāll get unlimited runs until the next renewal date.
e.g. if you renew on the 10th of March 2023 for a year of Basic, then youāll get unlimited runs until 10th of March 2024.
All this does is delay the inevitable.
I guess it might give teams a bit more time to plan their exit.
3 Likes
Or give them more time to set up scheduled collection runs, add the Postman CLI/Newman to their CI pipeline etc!
1 Like
I work at Postman and to summarize the status of this change.
We are packaging our plans to capture the value that automation brings - thereās still Postman CLI or Newman for folks who want to try it out or build their own solutions. Running collections in the runner is something most frequently used by teams building business-critical test suites in a professional setting, who ultimately align with our professional or enterprise plans. Not everyone agrees with the change, but we are trying to do whatās best for the community and the company in the long term so that we can grow together.
Even with these changes, we do not want to block users who want to adopt or use our collection runner in a non-professional setting. We will help users in these cases via our support channel
help@postman.com.
Please continue providing your feedback, and we will continue investing in the collection runner to deliver more value to our community.
My team at Postman also works on the Basic plan, and the change was sudden and impacts our usual workflows. But thereās still a lot of options (Newman, Postman CLI, Scheduled runs, etc) that we can use. I highly recommend reaching out to the support channel to talk 1v1 with our Product team to work out the specifics for your team.
1 Like
So what alternative software do you recommend us using for local runs?
Local runs are integral to building tests (to see if it works) before setting them as scheduled runs or via newman on your CI.
@kyle-revio, You can run your tests locally to see if they work, you can run individual calls, you just wonāt be able to run the whole collection via the collection runner.
1 Like
Ah I didnt know that. Thats useful thanks
What about testing with postman.setNextRequest()?
It has no effect on individual calls and now if i want to test the workflow of my collection i have to export it and run it via newman. Finetuning has become tedious since i have to do those extra steps every time i make an ever so little change.
Itās either this or the enterprise option. Is that correct? That hardly seems reasonable to me.
1 Like