I have a json file that I provide via flask, and I'd like to parse it from a requests object, but I can't get the full data back. I get a broken json file. What I get is about one third of the way through the file to the end.
I get different answers whether I use r.json() or json.loads(r.text)from the simple get request:
r = requests.get("url/api")
If I stream it manually, I can see the start of the file
r= requests.get("url/api", stream=True)
r.raw.read(1000).decode('utf-8')
And I successfully save down the file as here how to parse big requests object, but the problem persists when I try to then upload the file using json.load(local_filename)
The shape of my json file is:
[{"key1":"value1_1", "key2":"value2_1"},{"key1":"value1_2", "key2":"value2_2"}.......]
This is how I provide the json from flask
#app.route('/api', methods=['GET'])
def provide_json():
with open('record_file.json') as f:
record_file = json.load(f)
return json.dumps(record_file)
I've checked that my json file is valid, and I'm also assuming that my flask code with throw an exception is I wasn't delivering a valid json file because I've used json.dumps. Is that right? I've also tried wrapping the json file in flask with return json.dumps({"data_file":record_file}) but it doesn't change the problem.
You are returning list which is not a valid response for flask. Try this
from flask import jsonify
#app.route('/api', methods=['GET'])
def provide_json():
with open('record_file.json') as f:
record_file = json.load(f)
return jsonify({"data": record_file})
Related
whenever I create an endpoint to return a list, it returns a number-keyed dictionary instead.
from flask import Flask
app = Flask(__name__)
#app.route('/somelist')
def somelist():
return ['a', 'b', 'c']
When I go to view the endpoint, I get a dictionary like this:
What's going on here? I just want a list!
It is not a dictionary. The 0:, for example is not part of the response that you've returned.
The browser JSON extension you are using is showing the list along with its indicies.
Instead, use curl or Postman and inspect the raw response without browser response parsing. Or click on the Raw Data tab...
I am trying to send to my flask app json data and having it return a CSV file. My ajax request is sending JSON data to the view via POST request and then the view is supposed to return back a csv file. However, it fails to return the csv file in the browser as a download. I'm not sure how to make it work or if its even possible. Thanks!
// AJAX - Send data over to python and return csv
$("#export").click(
function(){
$.ajax({
url: "/dbCSV",
type: "POST",
contentType: 'application/json;charset=UTF-8',
dataType:"json",
data: JSON.stringify(datam)
});
event.preventDefault();
}
);
#analyzers.route("/dbCSV", methods=["GET","POST"])
def dbCSV():
if request.method=="POST":
data = pd.DataFrame(request.get_json())
resp = make_response(data.to_csv())
resp.headers["Content-Disposition"] = "attachment; filename=export.csv"
resp.headers["Content-Type"] = "text/csv"
return resp
return jsonify({"msg":"Could not generate CSV File"})
I'd recommend using send_file(...) with a BytesIO (file stream) object:
from io import BytesIO
from flask import send_file
...
response_stream = BytesIO(data.to_csv().encode())
return send_file(
response_stream,
mimetype="text/csv",
attachment_filename="export.csv",
)
Keep in mind that you will not be able to open the download prompt when sending a POST request using AJAX. Instead, you will simply receive the file as an AJAX response. To solve this issue, you will have to take a look at this question:
download file using an ajax request
Maybe your code was already working and this was your problem – I can not tell from looking at it.
I finally figure it out. Basically I can store the user input using the session object available from the flask library. This allows different functions to access this rather then having to worry about creating global variables or passing them around via functions or objects.
Note 1- If the amount of user data that has to be saved is extensive then using Redis or some other type of in memory data storage would be a better choice in this case.
Save the csv file in static path and then use the that static path csv URL to get csv file in download form from browser.
I have downloaded my Facebook data. I got the data in the form of JSON files.
Now I am trying to read these JSON files into NetworkX. I don't find any function to read graph from JSON file into NetworkX.
In another post, found the info related to reading a graph from JSON, where the JSON file was earlier created from NetworkX using json.dump().
But here in my case I have downloaded the data from Facebook. Is there any function to read graph from JSON file into NetworkX?
Unlike Pandas tables or Numpy arrays, JSON files has no rigid structure so one can't write a function to convert any JSON file to Networkx graph. If you want to construct a graph based on JSON, you should pick all needed info yourself. You can load a file with json.loads function, extract all nodes and edges according to your rules and then put them into your graph with add_nodes_from and add_edges_from functions.
For example Facebook JSON file you can write something like it:
import json
import networkx as nx
with open('fbdata.json') as f:
json_data = json.loads(f.read())
G = nx.DiGraph()
G.add_nodes_from(
elem['from']['name']
for elem in json_data['data']
)
G.add_edges_from(
(elem['from']['id'], elem['id'])
for elem in json_data['data']
)
nx.draw(
G,
with_labels=True
)
And get this graph:
I am trying to send some JSON data to an Odoo controller, but when I send the request, I always get 404 as response.
This is the code of my controller:
import openerp.http as http
import logging
_logger = logging.getLogger(__name__)
class Controller(http.Controller):
#http.route('/test/result', type='json', auth='public')
def index(self, **args):
_logger.info('The controller is called.')
return '{"response": "OK"}'
Now, I type the URL (http://localhost:8069/test/result) on the browser to check if it is available, and I get function index at 0x7f04a28>, /test/result: Function declared as capable of handling request of type 'json' but called with a request of type 'http'. This way I know that the controller is listening at that URL and is expecting JSON data.
So I open a Python console and type:
import json
import requests
data = {'test': 'Hello'}
data_json = json.dumps(data)
r = requests.get('http://localhost:8069/test/result', data=data_json)
When I print r in the console, it returns <Response [404]>, and I cannot see any message in the log (I was expecting The controller is called.).
There is a similar question here, but it is not exactly the same case:
OpenERP #http.route('demo_json', type="json") URL not displaying JSON Data
Can anyone help me? What am I doing wrong?
I have just solved the problem.
Firstly, as #techsavvy told, I had to modify the decorator, to write type='http' instead of type='json'.
And after that, the request from the console returned a 404 error because it did not know which database it was sending data to. In localhost:8069 I had more than one database. So I tried to have only one at that port. And that is, now it works great!
To manage that without removing any of the other databases, I have just modified the config file to change the parameter db_filter and put there a regular expression which only included my current database.
I have just gone through your issue and I noticed that you have written JSON route which is call from javascript. if you want to call it from browser url hit then you have to define router with type="http" and auth="public" argument in route:
#http.route('/', type='http', auth="public", website=True)
I have a URL on my Raspberry Pi device server. How can I parse it?
I have used the urllib2 module to get the contents of the URL from server.
I want to use JSON for parsing the url.
so if i understad properly , you have a json file with a URL that you want to pass to your python module
used this a while back
import json
with open('yourjasonfilewithurl.jason') as json_data_file:
data = json.load(json_data_file)
The use the data function like this, the below is dependant on the number of elements you have set up in your json config file
myurl = (data["details"][0]["url"])
hope this helps