My dataframe is
'col1' , 'col2'
A , 89
A , 232
C , 545
D , 998
and would like to export as follow :
{
'A' : [ 89, 232 ],
'C' : [545],
'D' : [998]
}
However, all the to_json does not fit this format (orient='records', ...).
Is there a way to ouput like this ?
Use groupby for convert to list and then to_json:
json = df.groupby('col1')['col2'].apply(list).to_json()
print (json)
{"A":[89,232],"C":[545],"D":[998]}
Detail:
print (df.groupby('col1')['col2'].apply(list))
col1
A [89, 232]
C [545]
D [998]
Name: col2, dtype: object
Related
I have list of objects as JSON. Each object has two properties: id(string) and arg(number).
When I use pandas.read_json(...), the resulting DataFrame has the id interpreted as number as well, which causes problems, since information is lost.
import pandas as pd
json = '[{ "id" : "1", "arg": 1 },{ "id" : "1_1", "arg": 2}, { "id" : "11", "arg": 2}]'
df = pd.read_json(json)
I'd expect to have a DataFrame like this:
id arg
0 "1" 1
1 "1_1" 2
2 "11" 2
I get
id arg
0 1 1
1 11 2
2 11 2
and suddenly, the once unique id is not so unique anymore.
How can I tell pandas to stop doing that?
My search so far only yielded results, where people where trying to achive the opposite - having columns of string beeing interpreted as numbers. I totally don't want to achive that in this case!
If you set the dtype parameter to False, read_json will not infer the types automatically:
df = pd.read_json(json, dtype=False)
Use dtype parameter for preventing cast id to numbers:
df = pd.read_json(json, dtype={'id':str})
print (df)
id arg
0 1 1
1 1_1 2
2 11 2
print (df.dtypes)
id object
arg int64
dtype: object
I am wondering how to transfer the dataframe to json format.
name ㅣ type ㅣ count
'james'ㅣ 'message'ㅣ 4
'kane' ㅣ 'text' ㅣ 3
'james'ㅣ 'text' ㅣ 2
'kane' ㅣ 'message'ㅣ 3
----------------------------result--------------------------------
dataframe to json fomat
data = [
{name : 'james', 'message' : 4, 'text; : 2}, {'name' : 'kane', 'message' :3, 'text' : 3}
]
How to change dataframe to json data?
You can use to_json and collect_list functions.
import pyspark.sql.functions as f
df1 = df.withColumn('json', f.struct('name', 'type', 'count')) \
.groupBy().agg(f.collect_list('json').alias('data')) \
.withColumn('data', f.to_json(f.struct(f.col('data')))) \
.show(10, False)
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|data |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|{"data":[{"name":"james","type":"message","count":4.0},{"name":"kane","type":"text","count":3.0},{"name":"james","type":"text","count":2.0},{"name":"kane","type":"message","count":3.0}]}|
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
I want to select rows information from table in MySQL. Rather than getting as it is, I want to get this information as JSON array list type.
For example,
A B C
=========
1 2 3
2 3 4
I want it to become data like the followings.
{ rows: [
{ "A" : 1, "B" : 2, "C" : 3},
{ "A" : 2, "B" : 3, "C" : 4}
]}
In MySQL you can achieve as below:
SELECT CONCAT('{ rows:[', result, ']}') AS result1 FROM
(
SELECT GROUP_CONCAT('{', jsondata, '}' SEPARATOR ',') AS result FROM
(
SELECT
CONCAT
(
'"A":' , A , ','
'"B":', B, ','
'"C":' , C
) AS jsondata
FROM test
) AS jsondata
) AS jsonsdata1;
SQL Demo link: http://sqlfiddle.com/#!9/95bbbc/2
I try to convert a json object to R dataframe, here is the json object:
json <-
'[
{"Name" : "a", "Age" : 32, "Occupation" : "凡达"},
{"Name" : "b", "Age" : 21, "Occupation" : "打蜡设计费"},
{"Name" : "c", "Age" : 20, "Occupation" : "的拉斯克奖飞"}
]'
then I use fromJSON, mydf <- jsonlite::fromJSON(json), the result is
Name Age Occupation
1 a 32 <U+51E1><U+8FBE>
2 b 21 <U+6253><U+8721><U+8BBE><U+8BA1><U+8D39>
3 c 20 <U+7684><U+62C9><U+65AF><U+514B><U+5956><U+98DE>
I was wondering how this happens, and is there any solution?
Using the package rjson can solve the problem, but the output is a list, but I want a dataframe output.
Thank you.
I've tried Sys.setlocale(locale = "Chinese"), well the characters are indeed Chinese,but the results are still weird like below:
Name Age Occupation
1 a 32 ·²´ï
2 b 21 ´òÀ¯Éè¼Æ·Ñ
3 c 20 µÄÀ˹¿Ë½±·É
I am working on converting XML to j son string using the PostgreSQL.
we have attributecentric XML and would like to know how to convert it to j son.
Example XML:
<ROOT><INPUT id="1" name="xyz"/></ROOT>
Need to get the j son as follows:
{ "ROOT": { "INPUT": { "id": "1", "name": "xyz" }}}
got the above json format from an online tool.
any help or guidance will be appreciated.
Regards
Abdul Azeem
Basically, breakdown of this problem is the following:
extract values from given XML using xpath() function
generate and combine JSON entries using json_object_agg() function
The only tricky thing here is to combine key,value pairs together from xpath()/json_object_agg() results, which are{ "id": "1"} and { "name" : "xyz"}.
WITH test_xml(data) AS ( VALUES
('<ROOT><INPUT id="1" name="xyz"/></ROOT>'::XML)
), attribute_id(value) AS (
-- get '1' value from id
SELECT (xpath('//INPUT/#id',test_xml.data))[1] AS value
FROM test_xml
), attribute_name(value) AS (
-- get 'xyz' value from name
SELECT (xpath('//INPUT/#name',test_xml.data))[1] AS value
FROM test_xml
), json_1 AS (
-- generate JSON 1 {"id": "1"}
SELECT json_object_agg('id',attribute_id.value) AS payload
FROM attribute_id
), json_2 AS (
-- generate JSON 2 {"name" : "xyz"}
SELECT json_object_agg('name',attribute_name.value) AS payload FROM attribute_name
), merged AS (
-- Generate INPUT result - Step 1 - combine JSON 1 and 2 as single key,value source
SELECT key,value
FROM json_1,json_each(json_1.payload)
UNION ALL
SELECT key,value
FROM json_2,json_each(json_2.payload)
), input_json_value AS (
-- Generate INPUT result - Step 2 - use json_object_agg to create JSON { "id" : "1", "name" : "xyz" }
SELECT json_object_agg(merged.key,merged.value) AS data
FROM merged
), input_json AS (
-- Generate INPUT JSON as expected { "INPUT" : { "id" : "1", "name" : "xyz" } }
SELECT json_object_agg('INPUT',input_json_value.data) AS data
FROM input_json_value
)
-- Generate final reult
SELECT json_object_agg('ROOT',input_json.data)
FROM input_json;