I have managed to get the google charts plugin (http://www.web2pyslices.com/slice/show/1721/google-charts-plugin) to work with my web2py application. Using the JSON example data (data hard coded into default.py).
I am struggling with using my own data. The chart does not work with my JSON data which returns the information as:
{"data": [["2014-03-28", 1000], ["2014-03-25", 1100]]}
When I hardcode the data with the titles, the chart works:
data = [['Date','Sales'],["2014-03-28",1000],["2014-03-25",1100]]
This returns JSON as:
{"data": [["Date", "Sales"], ["2014-03-28", 1000], ["2014-03-25", 1100]]}
The code for this is:
def return_data():
data = [['Date','Sales'],["2014-03-28",1000],["2014-03-25",1100]]
return dict(data=data)
Below is the code I am using in default.py to return the information from the database, the query works, it's the chart that doesn't!:
def return_data():
sales = db().select(db.sales.quantity, db.sales.date)
data = [[row.date,row.quantity] for row in sales]
return dict(data=data)
Somehow, I think I need to add the 'date' and 'sales' labels to the start of the json data but I have not managed to do this - I think I need to do some sort of encode? - do I need to use the simplejson or can this be done without??
Many thanks
Related
I am trying to turn a data frame into some nested json and have been struggling a bit. Here is an example I created. The use case is to split a document for each guest at a hotel chain, with the guest at the top, the hotel details under the visit data, and the daily charges under the 'measurements' info.
The dataframe:
Here is an example of how I am trying to get the JSON to look
I have tried creating a multilevel index and using to_json:
Is there a way to do this using to_json() or will I need to build some nested loops to create nested dictionaries? This is the best I have been able to get:
I would recommend a programming approach. pandas.DataFrame.groupby can be useful.
def hotel_data_to_json(df):
return [
person_data_to_json(person_df)
for person_id, person_df
in df.groupby('person_id')
]
def person_data_to_json(df):
row = df.iloc[0]
return {
'person_id': row['person_id'],
'personal_name': row['personal_name'],
'family_name': row['family_name'],
'visits': [
visit_data_to_json(visit_df)
for visit_id, visit_df
in df.groupby('visit_id')
]
}
def visit_data_to_json(df):
row = df.iloc[0]
# and so on
Ok so I don't necessarily understand this or how to do this at all but I am either looking for something that will show me how to sit there and split this portion of MYSQL json into either separate rows or just a way to export as either csv or json and then split one portion off from the others
Example of the JSON:
[{"id":2, "identifier":"IDENTIFIER:", "license":"LICENSE:", "firstname":"FIRSTNAME", "lastname":"LASTNAME", "accounts":"{"money":9595,"bank":9595}"},
{"id":2, "identifier":"IDENTIFIER", "license":"LICENSE", "firstname":"FIRSTNAME", "lastname":"LASTNAME", "accounts":"{"black_money":9595,"bank":9595,"money":9595}"}]
I want to be able to separate the three things in the JSON array called accounts this is all held in a mysql DB and I want to either be able to run something and have a exportable table that can be imported into google sheets or something of that sort so I can sort them if need be.
I expect that your json is probably like
[{"id":2, "identifier":"IDENTIFIER:", "license":"LICENSE:", "firstname":"FIRSTNAME", "lastname":"LASTNAME", "accounts":"{\"money\":9595,\"bank\":9595}"}, {"id":2, "identifier":"IDENTIFIER", "license":"LICENSE", "firstname":"FIRSTNAME", "lastname":"LASTNAME", "accounts":"{\"black_money\":9595,\"bank\":9595,\"money\":9595}"}]
but when copying / pasting the backslashes disappeared. Your json is obviously special since it includes other json. Try
function myFunction() {
var json = SpreadsheetApp.getActiveSpreadsheet().getActiveSheet().getRange('A1').getValue()
var data = JSON.parse(json)
var result = []
result.push(['account.black_money','account.bank','account.money'])
data.forEach(function(elem){
var account = JSON.parse(elem.accounts)
result.push([account.black_money,account.bank,account.money])
})
return result
}
take a copy of : https://docs.google.com/spreadsheets/d/1NwSUF7hRNjcLRbr2HP_mjj-fPfbKJz7BNLYFr184P4o/copy
I am trying to scrape the pokemon API and create a dataset for all pokemon. So I have written a function which looks like this:
import requests
import json
import pandas as pd
def poke_scrape(x, y):
'''
A function that takes in a range of pokemon (based on pokedex ID) and returns
a pandas dataframe with information related to the pokemon using the Poke API
'''
#GATERING THE DATA FROM API
url = 'https://pokeapi.co/api/v2/pokemon/'
ids = range(x, (y+1))
pkmn = []
for id_ in ids:
url = 'https://pokeapi.co/api/v2/pokemon/' + str(id_)
pages = requests.get(url).json()
# content = json.dumps(pages, indent = 4, sort_keys=True)
if 'error' not in pages:
pkmn.append([pages['id'], pages['name'], pages['abilities'], pages['stats'], pages['types']])
#MAKING A DATAFRAME FROM GATHERED API DATA
cols = ['id', 'name', 'abilities', 'stats', 'types']
df = pd.DataFrame(pkmn, columns=cols)
The code works fine for most pokemon. However, when I am trying to run poke_scrape(229, 229) (so trying to load ONLY the 229th pokemon), it gives me the JSONDecodeError. It looks like this:
So far I have tried using json.loads() instead but that has not solved the issue. What is even more perplexing is that specific pokemon has loaded before and the same issue was with another ID - otherwise I could just manually enter the stats for the specific pokemon that is unable to load into my dataframe. Any help is appreciated!
Because of the way the PokeAPI works, some links to the JSON data for each pokemon only load when the links end with a '/' (such as https://pokeapi.co/api/v2/pokemon/229/ vs https://pokeapi.co/api/v2/pokemon/229 - first link will work and the second will return not found). However, others will respond with a response error because of the added '/' so fixed the issue with a few if statements right after the for loop in the beginning of the function
I am trying to write my custom API view and I am struggling a bit with querysets and JSON. It shouldn't be that complicated but I am stuck still. Also I am confused by some strange behaviour of the loop I coded.
Here is my view:
#api_view()
def BuildingGroupHeatYear(request, pk, year):
passed_year = str(year)
building_group_object = get_object_or_404(BuildingGroup, id=pk)
buildings = building_group_object.buildings.all()
for item in buildings:
demand_heat_item = item.demandheat_set.filter(year=passed_year).values('building_id', 'year', 'demand')
print(demand_heat_item)
print(type(demand_heat_item)
return Response(demand_heat_item))
Ok so this actually gives me back exactly what I want. Namely that:
{'building_id': 1, 'year': 2019, 'demand': 230.3}{'building_id': 1, 'year': 2019, 'demand': 234.0}
Ok, great, but why? Shouldn't the data be overwritten each time the loop goes over it?
Also when I get the type of the demand_heat_item I get back a queryset <class 'django.db.models.query.QuerySet'>
But this is an API View, so I would like to get a JSON back. SHouldn't that throw me an error?
And how could I do this so I get the same data structure back as a JSON?
It tried to rewrite it like this but without success because I can't serialize it:
#api_view()
def BuildingGroupHeatYear(request, pk, year):
passed_year = str(year)
building_group_object = get_object_or_404(BuildingGroup, id=pk)
buildings = building_group_object.buildings.all()
demand_list = []
for item in buildings:
demand_heat_item = item.demandheat_set.filter(year=passed_year).values('building_id', 'year', 'demand')
demand_list.append(demand_heat_item)
json_data = json.dumps(demand_list)
return Response(json_data)
I also tried with JSON Response and Json decoder.
But maybe there is a better way to do this?
Or maybe my question is formulated clearer like this: How can I get the data out of the loop, and return it as a JSON
Any help is much appreciated. Thanks in advance!!
Also, I tried the following:
for item in buildings:
demand_heat_item = item.demandheat_set.filter(year=passed_year).values('building_id', 'year', 'demand')
json_data = json.dumps(list(demand_heat_item))
return Response(json_data)
that gives me this weird response that I don't really want:
"[{\"building_id\": 1, \"year\": 2019, \"demand\": 230.3}, {\"building_id\": 1, \"year\": 2019, \"demand\": 234.0}]"
I'm using wget to fetch several dozen JSON files on a daily basis that go like this:
{
"results": [
{
"id": "ABC789",
"title": "Apple",
},
{
"id": "XYZ123",
"title": "Orange",
}]
}
My goal is to find row's position on each JSON file given a value or set of values (i.e. "In which row XYZ123 is located?"). In previous example ABC789 is in row 1, XYZ123 in row 2 and so on.
As for now I use Google Regine to "quickly" visualize (using the Text Filter option) where the XYZ123 is standing (row 2).
But since it takes a while to do this manually for each file I was wondering if there is a quick and efficient way in one go.
What can I do and how can I fetch and do the request? Thanks in advance! FoF0
In python:
import json
#assume json_string = your loaded data
data = json.loads(json_string)
mapped_vals = []
for ent in data:
mapped_vals.append(ent['id'])
The order of items in the list will be indexed according to the json data, since the list is a sequenced collection.
In PHP:
$data = json_decode($json_string);
$output = array();
foreach($data as $values){
$output[] = $values->id;
}
Again, the ordered nature of PHP arrays ensure that the output will be ordered as-is with regard to indexes.
Either example could be modified to use a mapped dictionary (python) or an associative array (php) if needs demand.
You could adapt these to functions that take the id value as an argument, track how far they are into the array, and when found, break out and return the current index.
Wow. I posted the original question 10 months ago when I knew nothing about Python nor computer programming whatsoever!
Answer
But I learned basic Python last December and came up with a solution for not only get the rank order but to insert the results into a MySQL database:
import urllib.request
import json
# Make connection and get the content
response = urllib.request.urlopen(http://whatever.com/search?=ids=1212,125,54,454)
content = response.read()
# Decode Json search results to type dict
json_search = json.loads(content.decode("utf8"))
# Get 'results' key-value pairs to a list
search_data_all = []
for i in json_search['results']:
search_data_all.append(i)
# Prepare MySQL list with ranking order for each id item
ranks_list_to_mysql = []
for i in range(len(search_data_all)):
d = {}
d['id'] = search_data_all[i]['id']
d['rank'] = i + 1
ranks_list_to_mysql.append(d)