Python requests not working or hanging when trying to access site

Question

I'm having a weird issue recently with python (3.9) requests when accessing the NASDAQ stock screener. I want to get the list of companies on the NASDAQ, NYSE, and AMEX. There is a link for each exchange where the request returns a json that has all the information I want. The following is the link for the NYSE for example: https://api.nasdaq.com/api/screener/stocks?tableonly=true&limit=25&exchange=NYSE&download=true

If I just use the above link in a browser, it loads the json file and all the information just fine. My issue is that whenever I try to use requests to load the json file, it doesn't do anything. It doesn't return any response or response code...it just seems to hang for an indefinite amount of time.

Even with something as simple as the following, nothing happens for me:

import requests
requests.get("https://api.nasdaq.com/api/screener/stocks?tableonly=true&limit=25&exchange=NYSE&download=true")

Does anyone have any ideas as to what is going on? I didn't find any cookies being passed or anything like that. And before whenever there was a cookie that was required for other sites, I remember always having a failed response being returned anyways.

Thanks in advance for any help.

Edit: To show what I mean by literally nothing happening, this is what it looks like for me when I try to run the sample code:

Python shell running sample code

It just stays like that forever. No errors, doesn't exit, no response, nothing for me to use to figure out what is going on.

question from:https://stackoverflow.com/questions/65874351/python-requests-not-working-or-hanging-when-trying-to-access-site

Answer 1

you should try ton change your headers. Add parameter to your requests by using headers as headers={'Referer': referlink on the website, 'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/57.36 (KHTML, like Gecko) Chrome/4.96 Safari/57.36', } you can see your headers by using inspect element on chrome, look the requests you've made and the headers should be in.