Moving towards the next-generation URL processor

Hello Postman community! :wave:

We have introduced an improved URL processing system within Postman. This provides you the flexibility and compatibility with the ever-evolving landscape of HTTP clients and servers you requested us to help better manage. Additionally, it fixes a number of limitations of the present URL processor around the aspects of parsing and encoding.

Since the new URL processing system is a ground-up replacement of how Postman parses and encodes your URLs, we would love that you check whether your existing issues around URL parsing and encoding are addressed all while not creating any unintended side-effects.

If you have any feedback to provide, please post it on our GitHub issue tracker!

How to start using the new URL processor.

This feature is already enabled in Postman Canary since version 7.20.0-canary03 or can be turned on from Postman 7.20 through the settings.

What do you get when you use the new processor?

Address major encoding related concerns
A host of issues around encoding, such as disparity of expectations in URL encoding, is closer to being resolved. Encoding of the URL path and query parameters were the most contentious of all and almost all of them are now waiting for your feedback to be marked as resolved.

Seamless experience matching popular HTTP clients and browsers
The new URL parser matches the encoding behavior of most prominent HTTP clients and browsers. Mainly, it adopts maximum parts of the WHATWG (URL) standards that the world has come to like and agree upon. The new processor leverages the knowledge of your intents that it can gather from the user interface and provides the least ambiguous and forgiving URL parsing for the many obscure URLs that we have encountered. It also attempts to bridge the gap between what URL you enter in Postman UI and what is actually sent across the wire.

More about URL encoding

URLs are permitted to only contain a certain range of characters. Any character falling outside of that range must be [encoded](https://en.wikipedia.org/wiki/Percent-encoding). How such characters are encoded, and which characters to encode depends entirely on where the character is located within the structure of the URL.

URL Component Characters to Encode
Path SPACE, ", <, >, #, ?, {, }, `
Query SPACE, ", #, &, ', <, =, >
User info SPACE, ", <, >, #, ?, {, }, /, :, ;, =, @, [, \, ], ^, |, `

Handling internationalized domain names

Make requests to exotic API endpoints
Equipped with full control of URL processing, we can now provide the unmatched ability to completely turn off encoding algorithms on URL path and query parameters. This is available as a setting alongside your requests and, as such, opens up the possibility for you to send requests to virtually the most stubborn or dated API servers and debug endpoints that have encoding related issues.

Making the best out of “variables” feature in Postman
This new URL processor is aware of Postman Variables (which look like {{this}}) at its core. As such variable names that conflict with URL parsing are no longer an issue. Even the usage of nested variable names are now enabled in URLs.

  • Resolves variables with reserved characters like {{host#1}}
  • Resolves nested variables like {{api-{{version}}}}

Some interesting new features that got added

  • Better support for IPv6 in URLs
  • Support for IP shorthand such as 127.1 for 127.0.0.1
  • Support for \\ in the path for Windows compatibility
  • Safer handling of multi-byte characters (emojis) and other multi-lingual character sets.
  • Better handling of newlines and whitespace characters.
References

Cheers :tada:

2 Likes