from a Dataframe, I want to have a JSON output file with one key having a list:
Expected output:
[
{
"model": "xx",
"id": 1,
"name": "xyz",
"categories": [1,2],
},
{
...
},
]
What I have:
[
{
"model": "xx",
"id": 1,
"name": "xyz",
"categories": "1,2",
},
{
...
},
]
The actual code is :
df = pd.read_excel('data_threated.xlsx')
result = df.reset_index(drop=True).to_json("output_json.json", orient='records')
parsed = json.dumps(result)
jsonfile = open("output_json.json", 'r')
data = json.load(jsonfile)
How can I achive this easily?
EDIT:
print(df['categories'].unique().tolist())
['1,2,3', 1, nan, '1,2,3,6', 9, 8, 11, 4, 5, 2, '1,2,3,4,5,6,7,8,9']
You can use:
df = pd.read_excel('data_threated.xlsx').reset_index(drop=True)
df['categories'] = df['categories'].apply(lambda x: [int(i) for i in x.split(',')] if isinstance(x, str) else '')
df.to_json('output.json', orient='records', indent=4)
Content of output.json
[
{
"model":"xx",
"id":1,
"name":"xyz",
"categories":[
1,
2
]
}
]
Note you can also use:
df['categories'] = pd.eval(df['categories'])
Related
I'm trying to parse json file to csv. Could you help?
example json:
{
"expand": "schema,names",
"startAt": 0,
"maxResults": 50,
"total": 21,
"issues": [
{
"expand": "operations,versionedRepresentations",
"id": "217580",
"self": "issue/217580",
"key": "ART-4070",
"fields": {"summary": "#[ART] Pre.3 Verification \\"S\\""}
},
{
"expand": "operations,versionedRepresentations",
"id": "217579",
"self": "issue/217579",
"key": "ART-4069",
"fields": {"summary": "Verification \\"C\\""}
},
{
"expand": "operations,versionedRepresentations",
"id": "217577",
"self": "issue/217577",
"key": "ART-4068",
"fields": {"summary": "#[ART] Enum type"}
}
]
}
result csv should be like:
key;summary
ART-4070;#[ART] Pre.3 Verification \"S\"
ART-4069;Verification \"C\"
ART-4068;#[ART] Enum type
I've tried such a code:
import groovy.json.*
def jsonSlurper = new JsonSlurper()
def json = '''
{
"expand": "schema,names",
"startAt": 0,
"maxResults": 50,
"total": 21,
"issues": [
{
"expand": "operations,versionedRepresentations",
"id": "217580",
"self": "issue/217580",
"key": "ART-4070",
"fields": {"summary": "#[ART] Pre.3 Verification \\"S\\""}
},
{
"expand": "operations,versionedRepresentations",
"id": "217579",
"self": "issue/217579",
"key": "ART-4069",
"fields": {"summary": "Verification \\"C\\""}
},
{
"expand": "operations,versionedRepresentations",
"id": "217577",
"self": "issue/217577",
"key": "ART-4068",
"fields": {"summary": "#[ART] Enum type"}
}
]
}
'''
def obj = jsonSlurper.parse(json)
def columns = obj.issues*.keySet().flatten().unique()
// remove nulls
def encode = { e -> e == null ? '' : e }
// Print all the column names
println columns.collect { c -> encode( c ) }.join( ';' )
// create all the rows
println obj.issues.collect { row ->
// A row at a time
columns.collect { colName -> encode( row[ colName ] ) }.join( ';' )
}.join( '\n' )
but result is wrong:
expand;id;self;key;fields
operations,versionedRepresentations;217580;issue/217580;ART-4070;[summary:#[ART] Pre.3 Verification "S"]
operations,versionedRepresentations;217579;issue/217579;ART-4069;[summary:Verification "C"]
operations,versionedRepresentations;217577;issue/217577;ART-4068;[summary:#[ART] Enum type]
how can i extract only what i want from json file? I need only two columns:key,summary and values for them.
You want to extract only specific information from your list of issues
and you need different strategies to extract those. So I'd use
a "configuration" to describe the extraction (see the map config
below). Then the code is quite close to your original one (extracted
some common code etc)
import groovy.json.*
def config = [ // header -> extractor
"key": { it.key },
"summary": { it.fields.summary }
]
def encode(e) { // help with nulls; quote the separator
(e ?: "").replaceAll(";", "\\;")
}
def csvLine(items) { // write items as "CSV"
println(items.collect{ encode it }.join(";"))
}
// main
def obj = new JsonSlurper().parse("data.json" as File)
csvLine(config.keySet())
obj.issues.each{ issue ->
csvLine(config.values().collect{ f -> f issue })
}
I want to create a map of data with the pid value as the key and the name as the array for a json array. Here is the json structure:
{
"measurements": [
{
"pid": 6691,
"name": "lung",
"measurement": "qualityFactor",
},
{
"pid": 1106,
"name": "kidney",
"measurement": "qualityFactor",
},
{
"id": 119,
"name": "pancreas",
"measurement": "qualityFactor",
},
]
}
Here is my attempt with Groovy but I'm stuck:
def jsonSlurper= new JsonSlurper()
Object objs=jsonSlurper.parseText(jsonData)
List pp =objs.data
Map<String,String> m=new HashMap()
pp.each{ it ->
it.collect{Map mm ->
println "Map m is ${mm}"
}
}
I want the map to look like
["6691" : "lung" , "1106" :"kidney" ....] etc.
Hiw can I accomplish this?
As a full answer with corrected JSON:
def json = '''{
"measurements": [
{
"pid": 6691,
"name": "lung",
"measurement": "qualityFactor",
},
{
"pid": 1106,
"name": "kidney",
"measurement": "qualityFactor",
},
{
"pid": 119,
"name": "pancreas",
"measurement": "qualityFactor",
}
]
}
'''
import groovy.json.*
def parsed = new JsonSlurper().parseText(json)
def map = map.measurements.collectEntries { [it.pid.toString(), it.name] }
I have a json in the below format.
[
{"id": 1,
"name": "peter" },
{"id": 2,
"name": "john" },
{"id": 3,
"name": "justin" }
.
.
{"id": 500,
"name": "david" },
]
I am trying to create an array in batches of 10 in the below format
[
{
{"id": 1,
"name": "peter" },
.
.
{"id": 10,
"name": "nixon" },
},
{
{"id": 11,
"name": "nancy" },
.
.
{"id": 20,
"name": "underwood" },
}
.
.
]
I tried using reduce and tried for loop to loop through it, but was unsuccessful
Here's a demo.
const str = "abcdefghigklmnopqrstuvwxyz";
let data = [];
for(let i = 0; i < 26; i++){
data.push({id : i, name: str.charAt(i)});
}
let res = data.reduce((acc, d) => {
let groupId = Math.floor(d.id / 10);
acc[groupId] = acc[groupId] || {};
acc[groupId][d.id] = d;
return acc;
}, {});
console.log(Object.values(res));
If you can ensure that id is the same sequence as their position in array, i think simply slice will better.
I want to generate JSON from TOML files. The JSON structure should be something like this, with arrays of objects within arrays of objects:
{
"things": [
{
"a": "thing1",
"b": "fdsa",
"multiline": "Some sample text."
},
{
"a": "Something else",
"b": "zxcv",
"multiline": "Multiline string",
"objs": [ // LOOK HERE
{ "x": 1},
{ "x": 4 },
{ "x": 3 }
]
},
{
"a": "3",
"b": "asdf",
"multiline": "thing 3.\nanother line"
}
]
}
I have some TOML that looks like the example below, but it doesn't seem to work with the objs section.
name = "A Test of the TOML Parser"
[[things]]
a = "thing1"
b = "fdsa"
multiLine = """
Some sample text."""
[[things]]
a = "Something else"
b = "zxcv"
multiLine = """
Multiline string"""
[[things.objs]] # MY QUESTION IS ABOUT THIS PART
x = 1
[[things.objs]]
x = 4
[[things.objs]]
x = 3
[[things]]
a = "3"
b = "asdf"
multiLine = """
thing 3.
another line"""
Is there a way to do it in TOML? JSON to TOML converters don't seem to work with my example. And does it work with deeper nesting of arrays of arrays/tables?
As per the PR that merged this feature in the main TOML repository, this is the correct syntax for arrays of objects:
[[products]]
name = "Hammer"
sku = 738594937
[[products]]
[[products]]
name = "Nail"
sku = 284758393
color = "gray"
Which would produce the following equivalent JSON:
{
"products": [
{ "name": "Hammer", "sku": 738594937 },
{ },
{ "name": "Nail", "sku": 284758393, "color": "gray" }
]
}
I'm not sure why it wasn't working before, but this seems to work:
name = "A Test of the TOML Parser"
[[things]]
a = "thing1"
b = "fdsa"
multiLine = """
Some sample text."""
[[things]]
a = "Something else"
b = "zxcv"
multiLine = """
Multiline string"""
[[things.objs]]
x = 1
[[things.objs]]
x = 4
[[things.objs]]
x = 7
[[things.objs.morethings]]
y = [
2,
3,
4
]
[[things.objs.morethings]]
y = 9
[[things]]
a = "3"
b = "asdf"
multiLine = """
thing 3.
another line"""
JSON output:
{
"name": "A Test of the TOML Parser",
"things": [{
"a": "thing1",
"b": "fdsa",
"multiLine": "Some sample text."
}, {
"a": "Something else",
"b": "zxcv",
"multiLine": "Multiline string",
"objs": [{
"x": 1
}, {
"x": 4
}, {
"x": 7,
"morethings": [{
"y": [2, 3, 4]
}, {
"y": 9
}]
}]
}, {
"a": "3",
"b": "asdf",
"multiLine": "thing 3.\\nanother line"
}]
}
Let's assume that I have GET request with URL domain/search/?value=123 and data in JSON:
[
{
"id": 1,
"value": 123,
"value2": 123123
},
{
"id": 2,
"value": 1,
"value2": 214
}
]
I would like to get data where value = 123. In this case:
[
{
"id": 1,
"value": 123,
"value2": 123123
}
]
I have found information how to capture parameters from URL in this post. I wonder what I should do now to find best solution in Django. Thanks in advance.
How can I use it in views.py:
if request.method == 'GET':
myObject = myObjectClass.objects.all()
serializer = myObjectSerializer(myObject, many=True)
return Response(serializer.data)
when data from JSON is not only an integer.
This is how to get it in python:
data = [
{
"id": 1,
"value": 123,
"value2": 123123
},
{
"id": 2,
"value": 1,
"value2": 214
}
]
result = None
for item in data:
if item['value'] == 123:
result = [item]
break
print(result)
Here is a javascript code to do this :-
function test()
{
//alert("Hi");
var text = '[{"id":1,"value":123,"value2":123123},{"id":2,"value":1,"value2":214}]'
json = JSON.parse(text);
var result = [];
for(var i = 0; i < json.length; i++) {
var obj = json[i];
if(obj['value'] == 123){
result.push(obj);
}
}
console.log(result);
}
In views.py , you can get the value of the URL through request object
data = [
{
"id": 1,
"value": 123,
"value2": 123123
},
{
"id": 2,
"value": 1,
"value2": 214
}
]
print [ item for item in data if item['value'] == request.GET.get('value') ]