Having problems entering JSON into MySQL database - mysql

I am trying to develop a system to handle data from a third party site. I will enter their data in my MySQL DB. Part of their data is a JSON string. I am new to JSON in MySQL. I have read and watched a tutorial on how to query JSON. I have done so successfully with JSON strings that I have entered. The problem is I cannot get their string to insert into my table. I get an error that says,"#3140 - Invalid JSON text: "The document root must not follow by other values." at position 11 in value for column 'ordered_shed.items'. Below is the JSON I received from the company. I have looked at it and cannot figure out what the problem is. I am using PHPMyAdmin to insert the code. The structure is.
id int(11) AUTO_INCREMENT
items JSON
"lineItems":[
{ "description": "details-item-size",
"productKey": "size",
"quantity": 1
},
{ "description": "details-item-style",
"model": "",
"price": 2229,
"productKey": "style",
"quantity": 1,
"value": ""
},
{ "description": "details-item-flooring",
"price": 0,
"productCategory": "flooring-interior",
"quantity": 1
},
{ "description": "details-item-floor-joist",
"productCategory": "flooring-interior",
"quantity": 1
},
{ "description": "details-item-roof-overhang",
"productKey": "RoofOverhang",
"quantity": 1
},
{ "description": "details-item-sidewall-height",
"productKey": "wall-height",
"quantity": 1
},
{ "description": "standard",
"productCategory": "structure",
"quantity": 1
},
{ "description": "Pressure Treated Skids",
"productCategory": "structure",
"quantity": 1
},
{ "description": "details-item-loft",
"productCategory": "flooring-interior",
"quantity": 1
},
{ "description": "details-item-roof-material",
"productKey": "roof-material",
"quantity": 1
},
{ "description": "details-item-siding-color",
"quantity": 1,
"productKey": "siding-color"
},
{ "description": "details-item-siding",
"productKey": "siding",
"quantity": 1
},
{ "description": "details-item-roof-color",
"quantity": 1,
"productKey": "roof-color"
},
{ "description": "details-item-trim-color",
"quantity": 1,
"productKey": "trim-color"
}]
Any help would be greatly appreciated.

The JSON you've provided is not valid. You can always check here.
I think you just need your JSON to look like this:
{
"lineItems": [{
"description": "details-item-size",
"productKey": "size",
"quantity": 1
},
{
"description": "details-item-style",
"model": "",
"price": 2229,
"productKey": "style",
"quantity": 1,
"value": ""
},
{
"description": "details-item-flooring",
"price": 0,
"productCategory": "flooring-interior",
"quantity": 1
},
{
"description": "details-item-floor-joist",
"productCategory": "flooring-interior",
"quantity": 1
},
{
"description": "details-item-roof-overhang",
"productKey": "RoofOverhang",
"quantity": 1
},
{
"description": "details-item-sidewall-height",
"productKey": "wall-height",
"quantity": 1
},
{
"description": "standard",
"productCategory": "structure",
"quantity": 1
},
{
"description": "Pressure Treated Skids",
"productCategory": "structure",
"quantity": 1
},
{
"description": "details-item-loft",
"productCategory": "flooring-interior",
"quantity": 1
},
{
"description": "details-item-roof-material",
"productKey": "roof-material",
"quantity": 1
},
{
"description": "details-item-siding-color",
"quantity": 1,
"productKey": "siding-color"
},
{
"description": "details-item-siding",
"productKey": "siding",
"quantity": 1
},
{
"description": "details-item-roof-color",
"quantity": 1,
"productKey": "roof-color"
},
{
"description": "details-item-trim-color",
"quantity": 1,
"productKey": "trim-color"
}
]
}

You can use
SELECT json_valid(<jscol>) as "isValid?"
to check whether json column is valid. If returns 1 then it's valid. Invalid if it returns 0 (zero), and you can insert like this :
insert into tab(items)
select case when json_valid(<jscol>)=1 then
<jscol>
end;
Demo

Related

How should I structure this nested JSON database to import into firebase firestore?

I have a database for an eCommerce website in this structure:
{
"hats": {
"id": 1,
"title": "Hats",
"routeName": "hats",
"items": [
{
"id": 1,
"name": "Brown Brim",
"imageUrl": "https://i.ibb.co/ZYW3VTp/brown-brim.png",
"price": 25
},
{
"id": 2,
"name": "Blue Beanie",
"imageUrl": "https://i.ibb.co/ypkgK0X/blue-beanie.png",
"price": 18
}
]
},
"sneakers": {
"id": 2,
"title": "Sneakers",
"routeName": "sneakers",
"items": [
{
"id": 10,
"name": "Adidas NMD",
"imageUrl": "https://i.ibb.co/0s3pdnc/adidas-nmd.png",
"price": 220
}
]
},
"jackets": {
"id": 3,
"title": "Jackets",
"routeName": "jackets",
"items": [
{
"id": 18,
"name": "Black Jean Shearling",
"imageUrl": "https://i.ibb.co/XzcwL5s/black-shearling.png",
"price": 125
},
{
"id": 19,
"name": "Blue Jean Jacket",
"imageUrl": "https://i.ibb.co/mJS6vz0/blue-jean-jacket.png",
"price": 90
}
]
}
}
I'm trying to find a way to import it to firestore without too much hassle.
As you can see the items field should be a new sub-collection, but whenever I try to import it to firestore, I get this error:
{
status: false,
message: 'Value for argument "data" is not a valid Firestore document. Input is not a plain JavaScript object.'
}
In the meantime I managed to solve this issue, by importing my JSON file to Realtime Database, editing there, then exporting it to Firestore!

How can I join models in ExtJS?

In ExtJS 6.02, I would like to join 2 or more models into a single store.
1 to 1 relationship:
Given users that can only be associated with a single user detail row in DB:
"users": [
{
"id": 1,
"name": "Philip J. Fry",
"userDetail": 1
},
{
"id": 2,
"name": "Hubert Farnsworth",
"userDetail": 2
},
{
"id": 3,
"name": "Turanga Leela",
"userDetail": 3
},
{
"id": 4,
"name": "Amy Wong",
"userDetail": 4
}
]
"usersDetails": [
{
"id": 1,
"email": "jfry#a.com",
"sex": "m"
},
{
"id": 2,
"email": "hubert#a.com",
"sex": "m"
},
{
"id": 3,
"email": "leela#a.com",
"sex": "f"
},
{
"id": 4,
"email": "amy#a.com",
"sex": "m"
}
]
I would like to have this on the store:
"users": [
{
"id": 1,
"name": "Philip J. Fry",
"email": "jfry#a.com",
"sex": "m"
},
{
"id": 2,
"name": "Hubert Farnsworth",
"email": "hubert#a.com",
"sex": "m"
},
{
"id": 3,
"name": "Turanga Leela",
"email": "leela#a.com",
"sex": "f"
},
{
"id": 4,
"name": "Amy Wong",
"email": "amy#a.com",
"sex": "m"
}
]
1 to many relationship:
Given users that can have multiple posts in BD:
"users": [
{
"id": 1,
"name": "Philip J. Fry",
"userDetail": 1
},
{
"id": 2,
"name": "Hubert Farnsworth",
"userDetail": 2
},
{
"id": 3,
"name": "Turanga Leela",
"userDetail": 3
},
{
"id": 4,
"name": "Amy Wong",
"userDetail": 4
}
]
"posts": [
{
"id": 1,
"user": 1,
"title": "Post 1 title",
"body": "Post 1 body"
},
{
"id": 2,
"user": 1,
"title": "Post 2 title",
"body": "Post 2 body"
},
{
"id": 3,
"user": 2,
"title": "Post 3 title",
"body": "Post 3 body"
},
{
"id": 4,
"user": 3,
"title": "Post 4 title",
"body": "Post 4 body"
}
]
I would like to have a store containing:
"items": [
{
"id": 1,
"name": "Philip J. Fry",
"posts": [
{
"id": 1,
"title": "Post 1 title",
"body": "Post 1 body"
},
{
"id": 2,
"title": "Post 2 title",
"body": "Post 2 body"
}
]
},
{
"id": 2,
"name": "Hubert Farnsworth",
"posts": [
{
"id": 3,
"user": 2,
"title": "Post 3 title",
"body": "Post 3 body"
}
]
},
{
"id": 3,
"name": "Turanga Leela",
"userDetail": 3,
"posts": [
{
"id": 4,
"user": 3,
"title": "Post 4 title",
"body": "Post 4 body"
}
]
},
{
"id": 4,
"name": "Amy Wong",
"posts": []
}
]
Many to 1 relationship:
This is actually my actual issue, I have a DB table with multiple items that each can only match a single item in another table.
Should be same as 1 to many.
Many to many relationship:
Basicly same as 1 to many I guess?
I have seem this solution: https://fiddle.sencha.com/#fiddle/1hs9&view/editor it doesn't cover everything and maybe there's a better option.
According to Sencha support:
The concept of joining records doesn't really exist within ExtJS but
you can, by making use of renderers, access and print record
association data.
I have a simple fiddle of your example above showing you how you may
achieve this. Please note that dynamic loading of association (which
you have in your example) makes using renderers a bit tricky, but not
impossible, as the loading of the association info is asynchronous in
nature and the renderer cannot wait for the response. The work around
is to refresh the entire grid when all association stores have
finished loading.
https://fiddle.sencha.com/#fiddle/3d1v&view/editor

How to set alias instead of id , i need category_id without using forloop

$categoryData = $this->compCatObj->find()->select(['CompanyCategory.id', 'CompanyCategory.name','CompanyCategory.restricted'])->contain(['CompanyItems'=>['fields'=>['CompanyItems.id','CompanyItems.company_category_id','CompanyItems.name']]])->toArray();
Response
{
"status": "success",
"message": "List of company categories",
"data": [
{
"id": 1,
"name": "Breakfast gdfgedfgdf",
"restricted": "no",
"company_items": []
},
{
"id": 2,
"name": "Breakfast",
"restricted": "yes",
"company_items": []
}
]
}
i need category_id instead of id. Is there any way to do this without using forloop.
yes you can do this by a small change in your code,
$categoryData = $this->compCatObj->find()->select(['category_id' => 'CompanyCategory.id', 'CompanyCategory.name','CompanyCategory.restricted'])->contain(['CompanyItems'=>['fields'=>['CompanyItems.id','CompanyItems.company_category_id','CompanyItems.name']]])->toArray();
Response
{
"status": "success",
"message": "List of company categories",
"data": [
{
"category_id": 1,
"name": "Breakfast gdfgedfgdf",
"restricted": "no",
"company_items": []
},
{
"category_id": 2,
"name": "Breakfast",
"restricted": "yes",
"company_items": []
}
]
}

Convert nested json to csv to sheets json api

I'm want to make my json to csv so that i can upload it on google sheets and make it as json api. Whenever i have change data i will just change it on google sheets. But I'm having problems on converting my json file to csv because it changes the variables whenever i convert it. I'm using https://toolslick.com/csv-to-json-converter to convert my json file to csv.
What is the best way to convert json nested to csv ?
JSON
{
"options": [
{
"id": "1",
"value": "Jumbo",
"shortcut": "J",
"textColor": "#FFFFFF",
"backgroundColor": "#00000"
},
{
"id": "2",
"value": "Hot",
"shortcut": "D",
"textColor": "#FFFFFF",
"backgroundColor": "#FFFFFF"
}
],
"categories": [
{
"id": "1",
"order": 1,
"name": "First Category",
"active": true
},
{
"id": "2",
"order": 2,
"name": "Second Category",
"shortcut": "MT",
"active": true
}
],
"products": [
{
"id": "03c6787c-fc2a-4aa8-93a3-5e0f0f98cfb2",
"categoryId": "1",
"name": "First Product",
"shortcut": "First",
"options": [
{
"optionId": "1",
"price": 23
},
{
"optionId": "2",
"price": 45
}
],
"active": true
},
{
"id": "e8669cea-4c9c-431c-84ba-0b014f0f9bc2",
"categoryId": "2",
"name": "Second Product",
"shortcut": "Second",
"options": [
{
"optionId": "1",
"price": 11
},
{
"optionId": "2",
"price": 20
}
],
"active": true
}
],
"discounts": [
{
"id": "1",
"name": "S",
"type": 1,
"amount": 20,
"active": true
},
{
"id": "2",
"name": "P",
"type": 1,
"amount": 20,
"active": true
},
{
"id": "3",
"name": "G",
"type": 2,
"amount": 5,
"active": true
}
]
}
Using python, this can be easily done or almost done. Maybe this code will help you in some way to understand that.
import json,csv
data = []
with open('your_json_file_here.json') as file:
for line in file:
data.append(json.loads(line))
length = len(data)
with open('create_new_file.csv','w') as f:
writer = csv.writer(f)
writers = csv.DictWriter(f, fieldnames=['header1','header2'])
writers.writeheader()
for iter in range(length):
writer.writerow((data[iter]['specific_col_name1'],data[iter]['specific_col_name2']))
f.close()

Export mysql data to JSON in a nested and recursivelty way starting from a table?

I have seen several examples to export MySQL tables to JSON, however such examples export data in an aggregated way. For example, take a database where you have two tables "Invoice head" and "Invoice details". "Invoice details" is a child of "Invoice head".The data in JSON is usually represented like:
{
"invoice_head": [
{
"number": 1,
"cliente": "Carlos",
"date": "2016-12-12"
},
{
"number": 2,
"cliente": "Fernando",
"date": "2017-01-01"
}
],
"invoice_details": [
{
"headnumber": 1,
"lineno": 1,
"product": "Shoes",
"quantity": 2
},
{
"headnumber": 1,
"lineno": 2,
"product": "Socks",
"quantity": 1
},
{
"headnumber": 2,
"lineno": 1,
"product": "Laptop",
"quantity": 1
}
],
}
I need to export data in a nested way:
{
"invice_head": [
{
"number": 1,
"cliente": "Carlos",
"date": "2016-12-12",
"invice_details": [
{
"headnumber": 1,
"lineno": 1,
"product": "Shoes",
"quantity": 2
},
{
"headnumber": 1,
"lineno": 2,
"product": "Socks",
"quantity": 1
}
]
},
{
"number": 2,
"cliente": "Fernando",
"date": "2017-01-01"
"invice_details": [
{
"headnumber": 2,
"lineno": 1,
"product": "Laptop",
"quantity": 1
}
]
}
]
}
For this I guess one needs to start in a table and recursively go through its childs for each record.
Does anybody knows if there is anything that does it? I don't want to reinvent the wheel.