Error 401, "unauthorized" while trying to connect to Twitter's API v2 with Academic access

New to APIs/Postman
1

Hello everyone,

I know this is probably a silly question, and that the answer is surely out there, but I have looked at many tutorials and guides on the Twitter Dev platforms, Youtube as well as threads here in this forum, but couldn’t find an answer I managed to understand as the solution I need. I am kind of a newbie with APIs and Postman, and surely do not yet have the technical knowledge allowing me to understand the subtleties linked to my issues.

Here is the situation. I have access to the relatively new Twitter Academic Research track. I have my project and app created, and consequently I have my own tokens and keys. I am interested in collecting historical tweets containing specific keywords (e.g. #globalwarming, among others) using the full-archive search, yet, every time I try to launch a collection, I get an error 401. I have also tried to simply launch Single tweet lookups, with the same error. Here is what I did to setup my Postman environment:

I have added the Twitter Postman collection I need (i.e. the Twitter API v2) as indicated in the Twitter doc here, and the Twitter API v2 now appears both in my Postman Environments and Collections.

Still following the aforementioned guide, I have also added my tokens and keys in the Twitter API v2 Environment. The keys and tokens I am given thanks to my Twitter app on the Academic track are an API Key, and API Secret Key, a Bearer Token, an Access Token, and an Access Token Secret. Yet, within Postman, the fields I am asked to fill are consumer_key, consumer_secret, access_token, token_secret, bearer_token. Here is my first question then: Do I have to understand that the API Key and API Secret correspond to the Consumer Key and Consumer Secret respectively?

My second question is: despite these different variables, I am unable to start any kind of collection. These are the only steps I took. Still following the guide I mentioned, these should be (as far as I understand at least), the only steps needed regarding the parameters and authorizations, apart from the obvious information you need to provide for the specific collections you need (e.g. query information regarding the ‘#globalwarming’ hashtag I gave as an example).
Am I then missing something else that I have not done or that I did improperly preventing me from starting a collection?.

Thanks in advance for any help you could provide. Again, I am a bit at a loss, not being extremely proficient with these types of tools and programming as a whole, so I apologize in advance if the answers were already provided somewhere else.

Do not hesitate to tell me if you need any additional information or screenshots to help you understand my issue, I will try to be as reactive as possible.

2 Likes
2

Hey @mitchbuchanon ! First off – there’s no shame here in the Just getting started channel, and there is no such thing as a silly question when you’re asking for help :slight_smile:

First question - At the collection level, in the Authorization tab, I suggest changing the type to Bearer Token, and using the one you’ve been provided.

More thoughts:

Second question - When you say “starting a collection”, what do you mean?

3

Hello @hannah.neil , and thanks a lot for your answer (I kind of lost hope… ^^)!

I’m trying to quickly answer your question here, but I will definitely check the links you provided as soon as possible, it seems very promising!

To answer your second question, I mean start collecting tweets thanks to Postman. As mentioned previously, my project would involve collecting historical tweets containing specific hashtags (e.g. #globalwarming) and analyzing them and various metadata associated to them. Ideally I would like to go as far as 2010, and analyze the discourse associated to global warming and other environmental questions thanks to the study of the geolocation, the date etc, hence my need for other metadata.

Do you think that it is possible to collect such a corpus thanks to Postman?

Thanks in advance, and do not hesitate to ask for further details if anything is unclear. : )

1 Like
4

Hi @mitchbuchanon! I see you’re working with my good friend - the Twitter API!

What you’re looking to do should absolutely be possible within Postman! In order to do this, you’ll have to have some custom JavaScript code that continuously calls for older tweets if they’re available. This can be done by capturing the “next token” of a request and using that for ensuing requests.

I wrote in more detail about how to achieve this in this post here. Hope that helps!

5

Hi @sean.keegan , and thanks for your response!

I’ve looked at the resources you’ve provided, and they’re very interesting, but I still can’t get past that “status 401, unauthorized” issue I mentioned in my original post. Any idea what this is?

Thanks in advance!

Best

6

Hi @mitchbuchanon ,

Sorry to hear you’re still having trouble! You have the right idea in terms of getting started. A 401 error usually mean there was a problem authenticating your request. From the Twitter docs, they specifically say that “A 401 status code means that there was a problem authenticating your request, such as an invalid consumer key or secret.”

My guess is maybe you either entered in the credentials incorrectly or in the wrong spot. You are correct in matching the environment variable names with what Twitter calls them: “API Key” from Twitter developer dashboard goes with consumer_key environment variable and “API Secret Key” from Twitter developer dashboard goes with the consumer_secret environment variable.

The values for the access_token and token_secret environment variables can be retrieved from the “Access Token and Secret” section of your project/app. See the screen shot for where to get this info.

Screen Shot 2021-06-07 at 3.57.50 PM
Screen Shot 2021-06-07 at 3.57.50 PM1908×874 102 KB

Once all those are entered, you should be good to start making requests! The last thing to check would be that you have your environment saved AND selected. The red oval should have the “Twitter API v2” environment selected. If the environment is not selected and you go to make a request, it will give you that 401 error.

Screen Shot 2021-06-07 at 4.02.42 PM
Screen Shot 2021-06-07 at 4.02.42 PM1913×939 171 KB

Lastly - you should actually be able to send a successful request if you ONLY use the bearer token. I just tried and was able to make requests with only my bearer token info entered. I would try both methods of sending a request.

If you’re still struggling to get a request working, perhaps some screenshots or a public workspace may help further debug your issues :slight_smile:

Best of luck!

7

Thanks a lot, I’ll take a look at all that when I have time (not right now unfortunately), and keep you posted if I have any other issue.

Thanks again!

M

1 Like
8

Best of luck @mitchbuchanon! Hope you’re able to get something working :slight_smile:

9

I have the same issue here (mine is academic account), I try to scrape by python and it works find but somehow I cannot use the postman to get the data. Any suggestion? I did take a look at youtube video: [https://www.youtube.com/watch?v=guHH51GDDI0] (https://www.youtube.com/watch?v=guHH51GDDI0) but it did not work as it was showed in the youtube. Thanks in advance for any help you may provide.

Postman
Postman1140×527 52.9 KB

10

Hi!

I’m sorry to hear that you’re experiencing similar issues… Personally (using the Academic track as well), I realized that Postman wasn’t the solution I was hoping for, and instead switched to another tool. I decided to use twarc (command line tool usable with Linux/Mac), which is very easy to use, and fits all of my needs. In case you’d like to take a look, here is the twarc2 (twarc2 is the version you need to use for the Academic track) page to get an idea of how this works:

In case you need any help, the community on Github is extremely reactive and friendly. I’d also be happy to help you if I could, so don’t hesitate to get back to me here, or in private.

Good luck!

11

Hi @mitchbuchanon ,

Thanks for sharing the experiences and offering helps :grinning:. I used two packages in Python, twarc and tweepy, to scrape the data from the twitter. I got confused at the first place after receiving the information as JSON format. Now, I learn how to convert the JSON structure to .csv format. Both stackoverflow and Github comunities are extremely helpful.

Regards,

12

Hi guys,

I’ve also followed the earlier replies here and youtube vids but keep getting 401s.

image
image893×236 17.8 KB

I’ve tried regenerating the tokens too but to no avail.

Just to be sure, I share screenshots of Authorization and Variable page details.

image
image857×479 30.2 KB

image
image838×452 27.3 KB

Many thanks in advance!

13

If it is not too late to respond :), I got the same error and I fixed it by going to Authorization tab, selecting bearer auth and pasting my bearer key into the field. That fixed the error for me.

14

@joint-operations-a11

On the authorisation page, the Token needs to point to the variable where you are storing the bearer token. Usually {{bearer_token}} not “Token”.

On your variables page, you then have the bearer_token but you have the value set to the variable name, which is incorrect. It needs to be the actual bearer key.

15

Hey, no worries, it can definitely be confusing when you’re just starting out with APIs and tools like Postman!

To answer your first question, yes, the API Key and API Secret Key correspond to the Consumer Key and Consumer Secret, respectively, in Postman.

As for the 401 error, it’s usually related to authorization issues. Double-check that you’ve set up the Bearer Token correctly in your headers for the requests that require it, and also ensure that your tokens haven’t expired. It’s important that you include all the necessary keys and tokens in the right places.

If you’re still having trouble getting data from Twitter, you might also want to consider alternative approaches for scraping data. For example, this guide on how to scrape data from Twitter might help you bypass some API limitations and explore different ways to collect the information you need.

Let me know if you need any more guidance!