Import json file into postgres table with single json column - json

I have been trying to load a json file in postgres as a single json column.
Table:
create table book(values json);
The file looks like this:
[
{
"isbn": "846896359-3",
"title": "Jungle Book 2, The",
"price": 22.05,
"date": "12/28/2017",
"authors": [
{
"first": "Marlène",
"last": "Ashley",
"age": 38
},
{
"first": "Miléna",
"last": "Finley",
"age": 37
},
{
"first": "Stévina",
"last": "Bullus",
"age": 44
}
],
"publisher": {
"name": "Youspan",
"address": {
"street": "Iowa",
"number": "853",
"city": "München",
"country": "Germany"
},
"phone": "361-191-8111"
}
},
{
"isbn": "558973823-7",
"title": "Star Trek III: The Search for Spock",
"price": 36.58,
"date": "4/19/2017",
"authors": [
{
"first": "Uò",
"last": "Ibel",
"age": 26
},
{
"first": "Mélys",
"last": "Grasner",
"age": 36
},
{
"first": "Mylène",
"last": "Laven",
"age": 40
},
{
"first": "Pò",
"last": "Lapsley",
"age": 37
}
],
"publisher": {
"name": "Chatterbridge",
"address": {
"street": "Dennis",
"number": "1",
"city": "São Tomé",
"country": "Sao Tome and Principe"
},
"phone": "845-226-0017"
}
}
]
Tried the copy command but it throws a 'Token "" is invalid.' error. I have also tried a number of other solutions

So, I finally got it to work.
First I removed new line characters from the file using
tr -d '\n' < yourfile.txt
Then I ran the following script:
create table Bookstemp(values text);
copy Bookstemp from 'BOOKS_DATANew.json';
create table books(valjson json);
Insert into books
select values
from (
select json_array_elements(replace(values,'\','\\')::json) as values
from Bookstemp
) a;

Related

I need to retrieve the index(number 1) information in mongodb compass

I just need to query all information of the doctor in the first index.
Here is my sample XML data
Here is my sample JSON data:
Both XML and JSON data are same. I just converted the XML to JSON format.
I have more documents like this.
I tried this query in this image
this query showing all information but I just need first doctors information.
I also tried in command prompt.Here is the query done in cmd
Here is my sample JSON code block: but I have more similar documents
{
"doctors": {
"doctor":[
{
"ssn": "257-79-xxxx",
"name": "Mavis Bxx",
"address": "xxxx Rusk Drive",
"country": "France",
"email": "",
"phone": "",
"patients": {
"patient": [
{
"gender": "Male",
"name": "Itch xxxx",
"address": "xx Cottonwood Avenue",
"revenue": "254",
"_country": "Spain",
"_id": "27"
},
{
"gender": "Male",
"name": "Damon xxxxx",
"address": "xx David Trail",
"revenue": "370",
"_country": "Germany",
"_id": "21"
}
]
},
"_id": "6"
},
{
"ssn": "179-45-xxxx",
"name": "Tobie Conxxxx",
"address": "x Comanche Center",
"country": "Spain",
"email": "tconringh#xxx.xxx",
"phone": "+86 998 262 xxxx",
"patients": {
"patient": {
"gender": "Male",
"name": "Vergil Tome",
"address": "x Melody Drive",
"revenue": "254",
"_country": "Germany",
"_id": "15"
}
},
"_id": "18"
},
{
"ssn": "777-59-xxxx",
"name": "Gertrud Macxxxx",
"address": "x Buell Drive",
"country": "USA",
"email": "gmaclaig2#xxx.com",
"phone": "+62 975 394 xxxx",
"patients": {
"patient": [
{
"gender": "Non-binary",
"name": "Dre Skxxxx",
"address": "x Becker Circle",
"revenue": "400",
"_country": "Germany",
"_id": "20"
},
{
"gender": "Female",
"name": "Arleyne Lestxxxx",
"address": "xx Farragut Court",
"revenue": "225",
"_country": "France",
"_id": "22"
}
]
},
"_id": "3"
}
You needs to change your collect data format. Your current format is single big size data. Not a quarriable format.
So
From current format
{
"doctors": {
"docker": [
{
...
"patients": {
"patient": [
{
...
},
{
...
}
]
},
},
{
...
}
]
}
}
To this format
[
{
...
"patients": [
{
...
},
{
...
}
]
},
{
...
}
]
Test JSON
[
{
"ssn": "2xx-7x-4xxx",
"name": "Mavis Bxxxed",
"address": "9xxx Rusk Drive",
"country": "France",
"email": "",
"phone": "",
"patients": [
{
"gender": "Male",
"name": "Itch Txxx",
"address": "5xx Cottonwood Avenue",
"revenue": "25x",
"_country": "Spain",
"_id": "27"
},
{
"gender": "Male",
"name": "Damon Wxxx",
"address": "09xx Dxxxx Trail",
"revenue": "370",
"_country": "Germany",
"_id": "21"
}
],
"_id": "6"
},
{
"ssn": "19-45-xxxx",
"name": "Tobie Coxxxx",
"address": "8x Comxxxx Center",
"country": "Spain",
"email": "tconxxxxx#usa.gov",
"phone": "",
"patients": [
{
"gender": "Male",
"name": "Some one",
"address": "1 Mexxxx Drive",
"revenue": "254",
"_country": "Germany",
"_id": "15"
}
],
"_id": "18"
}
]
In Compass View after add data upper data
Find doctor by name query
{ name: "Mavis Bxxxed" }
Find doctor by patient name
{ "patients.name": "Some one" }
If same search with this query
{ "patients.name": "Itch Txxx" }
OR
{ "patients.name": "Damon Wxxx" }
will same result to find the first docker (Mavis Bxxxed)
Due to it's patients start [ and end ]. it is array type a single size of data.
You can filter by project option
{
patients: {
$filter:
{
input: "$patients",
cond: { $eq: [ "$$patient.name", "Itch Txxx"] },
as: "patient"
}
}
}
OR shows only address
{ "patients.address" : 1 }

Moving a JSON array which have another array elements to the same LEVEL

I have the following JSON which have an element array_within_array in another array. So I want to pull both array_within_array element and upper array element(event_array) to root level using jq.
So here there are 3 events.
meal_selection
login
placed_order
Event placed_order have two sub-events in array. So after conversion there should be 4 events(1 from meal_selection, 1 from login and 2 from placed_order). these all should be on the same level.
Here is the JSON
{
"region": "USA",
"user_id": "123",
"event_array": [{
"event_attributes": {
"date": "2021-08-17",
"category": "lunch",
"location": "office"
},
"event_name": "meal_selection",
"created_at": "2021-08-13 01:28:57"
},
{
"event_name": "login",
"created_at": "2021-08-13 01:29:02"
},
{
"event_attributes": {
"array_within_array": [
{
"date": "2021-08-17",
"category": "lunch",
"location": "office"
},
{
"date": "2021-08-18",
"category": "dinner",
"location": "home"
}
]
},
"event_name": "placed_order",
"created_at": "2021-08-13 01:28:08"
}
]
}
and I want to convert to the below one
{
"region": "USA",
"user_id": "123",
"event_attributes": {
"date": "2021-08-17",
"category": "lunch",
"location": "office"
},
"event_name": "meal_selection",
"created_at": "2021-08-13 01:28:57"
}
{
"region": "USA",
"user_id": "123",
"event_name": "login",
"created_at": "2021-08-13 01:29:02"
}
{
"region": "USA",
"user_id": "123",
"event_attributes": {
"date": "2021-08-17",
"category": "lunch",
"location": "office"
},
"event_name": "placed_order",
"created_at": "2021-08-13 01:28:08"
}
{
"region": "USA",
"user_id": "123",
"event_attributes": {
"date": "2021-08-18",
"category": "dinner",
"location": "home"
},
"event_name": "placed_order",
"created_at": "2021-08-13 01:28:08"
}
Here's a straightforward solution that keeps things simple by using two steps:
{ region, user_id} + (.event_array[] )
| if .event_attributes|has("array_within_array")
then .event_attributes.array_within_array as $a
| .event_attributes = $a[]
else .
end

Find a record in json Object if the record has specific key in python

I have a JSON object which has 100000 records. I want a select a record which has specific value to the one of the key
Eg:
[{
"name": "bindu",
"age": "24",
"qualification": "b.tech"
},
{
"name": "naveen",
"age": "23",
"qualification": "b.tech"
},
{
"name": "parvathi",
"age": "23",
"qualification": "m.tech"
},
{
"name": "bindu s",
"status": "married"
},
{
"name": "naveen k",
"status": "unmarried"
}]
now I want to combine the records which are having the name with 'bindu' and 'bindu s. We can achieve this by iterating on the JSON object but since the size is more it is taking more time. Is there any way to make this easy.
I want the output like
[{
"name": "bindu",
"age": "24",
"qualification": "b.tech",
"status": "married"
},
{
"name": "naveen",
"age": "23",
"qualification": "b.tech",
"status": "unmarried"
},
{
"name": "parvathi",
"age": "23",
"qualification": "m.tech"
"status": ""
},
This will rename and merge your objects by first name.
jq 'map(.name |= split(" ")[0]) | group_by(.name) | map(add)'

How do I create transformed nested JSON elements with jq?

I have a JSON file with nested elements that I'm trying to manipulate into an unnested JSON file. How do I do this?
Using js, I have tried to isolate the percentages, which I was able to do. I'm not sure how to rename the percentage field. I've seen instances where it seems like value.gender or value.grade should work, but I'm also not sure how to combine it.
jq '.data[] | .id as $id | (.demographics[] | .percentage as $percentage | .gender as $gender | .grade as $grade | {"id":$id, "percentage":$percentage})' test2.json
From here, I want to be able to rename the percentage field to the gender and grades values. I then want to group by id.
Here is the original JSON file (test2.json):
{
"data": [{
"id": "abc",
"students": "elementary",
"demographics": [{
"grade": "K-2",
"percentage": "0.1",
"gender": "unspecified"
},
{
"grade": "K-2",
"gender": "male",
"percentage": "0.5"
},
{
"gender": "female",
"percentage": "0.4",
"grade": "K-2"
},
{
"grade": "3-6",
"percentage": "0.3",
"gender": "male"
},
{
"percentage": "0.2",
"gender": "unspecified",
"grade": "3-6"
},
{
"grade": "3-6",
"gender": "female",
"percentage": "0.5"
}
],
"neighborhood_name": [{
"percentage": "0.5",
"neighborhood": "atwood"
},
{
"region": "bluff",
"percentage": "0.5"
}
]
},
{
"id": "def",
"students": "midhigh",
"demographics": [{
"grade": "7-9",
"percentage": "0.2",
"gender": "unspecified"
},
{
"grade": "7-9",
"gender": "male",
"percentage": "0.2"
},
{
"gender": "female",
"percentage": "0.6",
"grade": "7-9"
},
{
"grade": "10-12",
"percentage": "0.1",
"gender": "male"
},
{
"percentage": "0.1",
"gender": "unspecified",
"grade": "10-12"
},
{
"grade": "10-12",
"gender": "female",
"percentage": "0.8"
}
],
"neighborhood_name": [{
"percentage": "0.2",
"neighborhood": "atwood"
},
{
"region": "bluff",
"percentage": "0.8"
}
]
}
]
}
Here is what I expect:
{
"id": "abc",
"students": "elementary",
"demo_K-2_unspecified": "0.1",
"demo_K-2_male": "0.5",
"demo_K-2_female": "0.4",
"demo_3-6_male": "0.3",
"demo_3-6_unspecified": "0.6",
"demo_3-6_female": "0.5",
},
{
"id": "def",
"students": "midhigh",
"demo_7-9_unspecified": "0.2",
"demo_7-9_male": "0.2",
"demo_7-9_female": "0.6",
"demo_10-12_male": "0.1",
"demo_10-12_unspecified": "0.1",
"demo_10-12_female": "0.8",
}
With the sample data, the following filter produces the desired output:
.data[]
| {id, students} as $ix
| .demographics
| map( {"demo_\(.grade)_\(.gender)": .percentage} )
| $ix + add
The main idea here is to use map to create a list of the key-value pairs, so that the composite object can then easily be created using add.
As a one-liner
jq '.data[] | {id,students} + (.demographics | map( {"demo_\(.grade)_\(.gender)": .percentage} ) | add)' test2.json

Problem retrieving data from json output

I am a noob when it comes to json. The more I use it, I am starting to like it. I have a output that looks like this
[
{
"product": {
"id": "6",
"category": "Books",
"created": "2010-04-13 15:15:18",
},
"availability": [
{
"country": "United Kingdom",
"price": "$13.99",
},
{
"country": "Germany",
"price": "$13.99",
}
]
}
]
Actually I have listed only one product, But output has many products listed. I want to loop through this json output and get all the product's category, countries where they are available and price using fbjs.
I appreciate any help.
Thanks.
If your JSON is like this,
var products = [
{
"product": {
"id": "6",
"category": "Books",
"created": "2010-04-13 15:15:18",
},
"availability": [
{
"country": "United Kingdom",
"price": "$13.99",
},
{
"country": "Germany",
"price": "$13.99",
}
]
},
{
"product": {
"id": "7",
"category": "Books",
"created": "2010-04-13 15:15:18",
},
"availability": [
{
"country": "United Kingdom",
"price": "$13.99",
},
{
"country": "Germany",
"price": "$13.99",
}
]
}
]
You can iterate through:
for(var index = 0; index < products.length; index++) {
alert(products[index].product.category);
}