I am trying to grab the data table in url = 'https://quotes.wsj.com/index/HK/XHKG/HSI/historical-prices/download?num_rows=150&range_days=150&endDate=02/29/2020'. I clicked on the link and a csv file will be download. Therefore, the link is correct. However, from the response.content, there is message saying "The request could not be satisfied.Request blocked." Can server distinguish manual click and python Request and block it? Any way to work around it?
My codes:
import requests
url = 'https://quotes.wsj.com/index/HK/XHKG/HSI/historical-prices/download?num_rows=150&range_days=150&endDate=02/29/2020'
response = requests.get(url)
print(response.content)
open('wsj.csv', 'wb').write(response.content)
Related
i am wondering how can i access the response headers of the initial request to the react server which returns the html of my website. there is a token which is sent in the response headers of the html static server.
i have already printed the window object and did not see anything special about that.
for instance this one :
I'm exploring Bitso API (Bitso is a mexican crypto exchange).
The docs of the API is well explained at some languages such as Python and Ruby for its use. The problem here is that there are no examples using straight URLs for request.
What I'm planning to do is to create the URL that the code is creating on its requests function.
There is a request for balance account, that is the data I'd like to get.
According documentation, this is a private request that need some headers at the request (Key, nonce and signature), you can take a look here.
The code to make this request in Python is the following one:
import time
import hmac
import hashlib
import requests
bitso_key = "BITSO_KEY"
bitso_secret = "BITSO_SECRET"
nonce = str(int(round(time.time() * 1000)))
http_method = "GET"
request_path = "/v3/balance/"
json_payload = ""
# Create signature
message = nonce+http_method+request_path+json_payload
signature = hmac.new(bitso_secret.encode('utf-8'),
message.encode('utf-8'),
hashlib.sha256).hexdigest()
# Build the auth header
auth_header = 'Bitso %s:%s:%s' % (bitso_key, nonce, signature)
# Send request
response = requests.get("https://api.bitso.com/v3/balance/", headers={"Authorization": auth_header})
print(response.content)
So based in this I could say that the URL is something like this:
https://api.bitso.com/v3/balance/Bitso%20<Key>:<nonce>:<signature>
I'm sure that I'm wrong with that supposition, I understand that headers={"Authorization": auth_header} seems to be a JSON object used as header in the URL, but I'd like to know how that JSON object is translated at the URL to make a request. I'd like to copy-paste that URL at the browser and get the data as response.
I need that URL so I could use it to connect the service to a Business Intelligence tool.
Thanks!
According to the documentation this Authorization is a header in the request. You can try using postman but you still need hash the destination URL with your api key and the nonce to avoid replay attacks.
I am trying to send some JSON data to an Odoo controller, but when I send the request, I always get 404 as response.
This is the code of my controller:
import openerp.http as http
import logging
_logger = logging.getLogger(__name__)
class Controller(http.Controller):
#http.route('/test/result', type='json', auth='public')
def index(self, **args):
_logger.info('The controller is called.')
return '{"response": "OK"}'
Now, I type the URL (http://localhost:8069/test/result) on the browser to check if it is available, and I get function index at 0x7f04a28>, /test/result: Function declared as capable of handling request of type 'json' but called with a request of type 'http'. This way I know that the controller is listening at that URL and is expecting JSON data.
So I open a Python console and type:
import json
import requests
data = {'test': 'Hello'}
data_json = json.dumps(data)
r = requests.get('http://localhost:8069/test/result', data=data_json)
When I print r in the console, it returns <Response [404]>, and I cannot see any message in the log (I was expecting The controller is called.).
There is a similar question here, but it is not exactly the same case:
OpenERP #http.route('demo_json', type="json") URL not displaying JSON Data
Can anyone help me? What am I doing wrong?
I have just solved the problem.
Firstly, as #techsavvy told, I had to modify the decorator, to write type='http' instead of type='json'.
And after that, the request from the console returned a 404 error because it did not know which database it was sending data to. In localhost:8069 I had more than one database. So I tried to have only one at that port. And that is, now it works great!
To manage that without removing any of the other databases, I have just modified the config file to change the parameter db_filter and put there a regular expression which only included my current database.
I have just gone through your issue and I noticed that you have written JSON route which is call from javascript. if you want to call it from browser url hit then you have to define router with type="http" and auth="public" argument in route:
#http.route('/', type='http', auth="public", website=True)
I am writing code to search the New York Times API for news articles and to print information from those articles.
I have been able to get the URLs for the news articles, but when I try to read the information from them, I am getting the following error:
urllib.error.HTTPError: HTTP Error 303: The HTTP server returned a redirect error that would lead to an infinite loop.
The last 30x error message was:
See Other
My code is pasted below and I commented the line where I am getting the error. The thing that confuses me is that urlopen worked fine the first time I used it to read information from NYT's API, but resulted in an error the second time when I try to read information from the URL of the specific article.
from urllib.request import urlopen
import json
url = 'http://api.nytimes.com/svc/search/v2/articlesearch.json?q=olympics&api-key=mykey'
request = urlopen(url) #no problems here
request = request.readall().decode('utf-8')
json_obj = json.loads(request)
for i, item in enumerate(json_obj):
title = json_obj.get('response').get('docs')[i].get('headline').get('main')
web_url = json_obj.get('response').get('docs')[i].get('web_url')
print(i+1)
print(title)
print(web_url)
print()
request = urlopen(web_url) #This is where I am getting the error
request = request.read().decode('utf-8')
print (request)
I am using portable python 3.2.5.1
I am also new to using python. Most of my programming experience is with java.
I have written a ac3 script to upload files to remote servers and it is working as expected with all the events.
From the server side script i am echoing some text according to the upload status and i want that status to be received in flash like response text in ajax.
Is it possible to receive response text from a server script after uploading a file.
Or it doesn't have to be a file upload yet i wanted to have the string echoed in the server side script as response text in flash.
If that is not possible like ajax or normal request and response then is there another ways of achieving that?
Here is the code which gets executed when upload is complete.
fr.addEventListener(Event.COMPLETE, function(e:Event): void
{
lblPer.text = 'Completed';
// ExternalInterface.call("uploadError",e.target.data);
})
where fr is FileReference object.
Try listening for DataEvent.UPLOAD_COMPLETE_DATA. That should have a data property on it that you can read the response from.