MySQL 5.7.8 JSON merge new data - mysql

I'm trying to make a notes / comments system for an admin area with the new MySQL JSON support. Comments need to be editable and I wanted to add support for other things in the future, maybe file attachments (would store the filepath in JSON only not the file itself!).
{
"comments": [
{
"comment": "This is a comment",
"user_id": 5,
"datecreated": "2016-03-19"
},
{
"comment": "This is a comment",
"user_id": 1,
"datecreated": "2016-03-19"
"comments": [
{
"comment": "This is a sub-comment",
"user_id": 4,
"datecreated": "2016-03-19"
},
{
"comment": "This is a sub-comment",
"user_id": 4,
"datecreated": "2016-03-19"
}
]
}
]
}
I thought there would be a way to merge in new data similar to array_merge() without needing to target a particular key every time.
This query works but it targets only one thing, the comment's text content. If I wanted to add/edit tags, image or file attachments etc. then I'd need a very long query or several queries.
UPDATE shared_notes SET json = JSON_REPLACE(json, "$.comments[1].comment", "This is a test comment") WHERE note_id = :note_id
I tried using JSON_REPLACE and JSON_SET functions with JSON_OBJECT but it overwrites all keys that aren't specified, which means user_id, datecreated and any sub comments get overwritten.
UPDATE shared_notes SET json = JSON_REPLACE(json, "$.comments[1]", JSON_OBJECT("comment", "This is a test comment") ) WHERE note_id = :note_id
This frankenstein of a query almost works but it actually concatenates the updated comment onto the end of the old one:
UPDATE shared_notes SET json = JSON_SET(json, "$.comments[1]", JSON_MERGE(JSON_EXTRACT(json, "$.comments[1]"), CAST('{"comment":"Test"}' AS JSON) ) ) WHERE note_id = :note_id
So is there a better way to easily / dynamically update the JSON using MySQL or is targeting $.comments[1].comment, $.comments[1][0].user_id etc. the only way?

For future reference json_merge has been deprecated since 5.7.22 in favour of json_merge_preserve or json_merge_patch
so adding to #Adam Owczarczyk's answer:
{...}
select json_merge_preserve('{"comments" : {"comment" : "This is a test comment" }}', comments)
from sampl_test;

This is a very late answer, but still - you can do it like this:
create table sampl_test(id int, comments json);
insert into sampl_test values(1,
'{
"comments": [
{
"comment": "This is a comment",
"user_id": 5,
"datecreated": "2016-03-19"
},
{
"comment": "This is a comment",
"user_id": 1,
"datecreated": "2016-03-19",
"comments": [
{
"comment": "This is a sub-comment",
"user_id": 4,
"datecreated": "2016-03-19"
},
{
"comment": "This is a sub-comment",
"user_id": 4,
"datecreated": "2016-03-19"
}
]
}
]
}
')
;
select json_merge('{"comments" : {"comment" : "This is a test comment" }}', comments)
from sampl_test;

Related

Spring - Ignore JSON properties in request but not in response

This is a valid Entry object retrieved from a GET request http://domain/entry/{id}:
{
"id": 1,
"description": "ML Books",
"dueDate": "2017-06-10",
"paymentDate": "2017-06-10",
"note": "Distribution de lucros",
"value": 6500,
"type": "INCOME",
"category": {
"id": 2,
"name": "Food"
},
"person": {
"id": 3,
"name": "User",
"active": true,
"address": {
// properties suppressed for better reading
}
}
}
In a POST request I want to save the foreing objects Category and Person just sending the respective Id's, like this:
{
"description": "NEW ENTRY",
"dueDate": "2019-06-22",
"paymentDate": "2019-06-22",
"note": "Coloured pens",
"value": 10,
"type": "INCOME",
"categoryId": 5,
"personId": 5
}
To save the objects without Spring saying the person and category objects were null, I've added #JsonIgnore to them in the model, and followed this thread.
It partially worked:
now it saves de object just with the Id
but not retrieves the object in GET requests
Now, when retrieving a Entry with the same GET request http://domain/entry/{id}:
{
"id": 23,
"description": "Pens",
"dueDate": "2019-06-22",
"paymentDate": "2019-06-22",
"note": "Coloured pens",
"value": 10,
"type": "INCOME",
"categoryId": null, // It supposed to bring the entire object
"personId": null // It supposed to bring the entire object
}
PS: categoryId and personId are marked with #Transient, that's why it are null.
So as the title states, I want to ignore the properties Category and Person just in POST request (saving them), not in GET requests (retrieving them).
Any help will be welcome.
Thanks in advance
Jackson have added READ_ONLY and WRITE_ONLY annotation arguments for JsonProperty. So you could also do something like:
#JsonProperty(access = JsonProperty.Access.READ_ONLY)
#Id
#GeneratedValue
private Long id;

How to query deep nested json array from couchbase?

How to query deep nested json array from couchbase? I have the following documents in the couchbase bucket. I need to query to list all apps who have Permissions "android.permission.BATTERY_STATS"
How to query to list all apps with permissions from nested json array?
My Json Documents,
Document:1
{
"data": {
"com.facebook.katana": {
"studioId": "Facebook",
"screenshotUrls": [
"https://lh3.googleusercontent.com/JcPdPqplBxgG6dEQuxvuhO4jvE64AzxOCGWe8w55dMMeXU4rZs2MwpfGQTWvv6QR-g",
"https://lh3.googleusercontent.com/w0kSYY7jlPjGDd3KEVgDTpzUf4k67G7rfELOf6qj1SSC7n6Ege44vp8QkeX57ZM6bFU"
],
"primaryCategoryName": "Social",
"studioName": "Facebook",
"description": "Keeping up with friends is faster and easier than ever. Share updates and photos, engage with friends and Pages, and stay connected to communities important to you"
"starRatings": {
"1": 9706642,
"2": 3384344,
"3": 7224416,
"4": 12323358,
"5": 49438051
},
"numDownloads": "1,000,000,000+ downloads",
"price": 0,
"permissions": [
"android.permission.ACCESS_COARSE_LOCATION",
"android.permission.ACCESS_FINE_LOCATION",
"android.permission.ACCESS_NETWORK_STATE",
"android.permission.ACCESS_WIFI_STATE",
"android.permission.AUTHENTICATE_ACCOUNTS",
"android.permission.BATTERY_STATS",
"android.permission.BLUETOOTH",
"android.permission.READ_PHONE_STATE",
"android.permission.READ_PROFILE",
"android.permission.READ_SMS",
"android.permission.READ_SYNC_SETTINGS",
"com.nokia.pushnotifications.permission.RECEIVE",
"com.sec.android.provider.badge.permission.READ",
"com.sec.android.provider.badge.permission.WRITE",
"com.sonyericsson.home.permission.BROADCAST_BADGE"
],
"appId": "com.facebook.katana",
"userRatingCount": 82076811,
"currency": "USD",
"iconUrl": "https://lh3.googleusercontent.com/ZZPdzvlpK9r_Df9C3M7j1rNRi7hhHRvPhlklJ3lfi5jk86Jd1s0Y5wcQ1QgbVaAP5Q=w100",
"releaseDate": "Nov 14, 2018",
"appName": "Facebook",
"studioUrl": "https://www.facebook.com/facebook",
"hasInAppPurchases": 1,
"bundleId": "com.facebook.katana",
"version": "198.0.0.53.101",
"commentCount": 22211936,
"fileSizeBytes": 58044343,
"formattedPrice": "",
"categoryIds": [
"APPLICATION",
"SOCIAL"
],
"tagline": "Find friends, watch live videos, play games & save photos in your social network",
"averageUserRating": 4.0770621299744,
"primaryCategoryId": "SOCIAL",
"videoScreenUrl": "https://lh4.ggpht.com/3RG_Y8JPK0Hcyui9OcapiONP_aDWKTRZ50wqZW_wbyOF0FamAYEYZfMTW9Cs1OT1kA"
}
},
"response_msec": 11,
"status": 200
}
Document:2
{
"data": {
"com.whatsapp": {
"studioId": "WhatsApp Inc.",
"screenshotUrls": [
"https://lh3.googleusercontent.com/MMue08byixTw74ST_VkNQDUUJBgVEbjNHDYLhIuHmYhMIMJIp3KjVlnhhqZQOZUtNt8",
"https://lh3.googleusercontent.com/foFmwvVGIwWWXJIukN7png18lFjFgbw3K7BqIm8G-jsFgSTVtkCa-dDkFApUzbvzIvbe"
],
"primaryCategoryName": "Communication",
"studioName": "WhatsApp Inc.",
"description": "WhatsApp Messenger is a FREE messaging app available for Android and other smartphones.
"starRatings": {
"1": 4713598,
"2": 1917919,
"3": 4962745,
"4": 11307648,
"5": 55392894
},
"numDownloads": "1,000,000,000+ downloads",
"price": 0,
"permissions": [
"android.permission.ACCESS_COARSE_LOCATION",
"android.permission.ACCESS_FINE_LOCATION",
"android.permission.ACCESS_NETWORK_STATE",
"android.permission.ACCESS_WIFI_STATE",
"android.permission.AUTHENTICATE_ACCOUNTS",
"android.permission.BLUETOOTH",
"android.permission.BROADCAST_STICKY",
"android.permission.CAMERA",
"android.permission.CHANGE_WIFI_STATE",
"android.permission.GET_ACCOUNTS",
"android.permission.GET_TASKS",
"android.permission.INSTALL_SHORTCUT",
"android.permission.INTERNET",
"android.permission.MANAGE_ACCOUNTS",
"com.whatsapp.permission.REGISTRATION",
"com.whatsapp.permission.VOIP_CALL",
"com.whatsapp.sticker.READ"
],
"appId": "com.whatsapp",
"userRatingCount": 78294804,
"currency": "USD",
"iconUrl": "https://lh6.ggpht.com/mp86vbELnqLi2FzvhiKdPX31_oiTRLNyeK8x4IIrbF5eD1D5RdnVwjQP0hwMNR_JdA=w100",
"releaseDate": "Nov 5, 2018",
"appName": "WhatsApp Messenger",
"studioUrl": "http://www.whatsapp.com/",
"bundleId": "com.whatsapp",
"version": "2.18.341",
"commentCount": 19763316,
"fileSizeBytes": 23857699,
"formattedPrice": "",
"categoryIds": [
"APPLICATION",
"COMMUNICATION"
],
"tagline": "Simple. Personal. Secure.",
"averageUserRating": 4.4145045280457,
"primaryCategoryId": "COMMUNICATION",
"videoScreenUrl": "https://lh3.ggpht.com/aZrXAunkovhf0630Ykz1A7h2rzFX_dErd6fRiB7fNKU_DkNtetTquEra1bjc3sR2kLs"
}
},
"response_msec": 15,
"status": 200
}
As I say in the comment, this is a tricky one. I'm going to try to simplify your docs first, and then give an answer that I came up with.
You have two docs, which contain a nested object with a permissions array. Each nested object has a (potentially) different name. So, let's assume we have two simple docs like this:
id: doc1
{
"foo": {
"permissions": [
"android.permission.ACCESS_COARSE_LOCATION",
"android.permission.BATTERY_STATS"
]
}
}
id: doc2
{
"bar": {
"permissions": [
"android.permission.ACCESS_FINE_LOCATION"
]
}
}
The first one has a "foo" nested object, the second has a "bar" nested object, but both nested objects have a "permissions" array. You want to find all the documents that have a permission of "android.permission.BATTERY_STATS".
I checked out the N1QL docs for anything that might be helpful, and I especially checked out the Object Functions section. There's a function called OBJECT_UNWRAP that might do the trick. From the docs: "This function enables you to unwrap an object without knowing the name in the name-value pair."
So, if I simply unwrap the above documents, then I can basically discard the "foo" and the "bar" parts.
SELECT META(b).id, OBJECT_UNWRAP(b).permissions
FROM sstbucket b
You can put unwrap a deeper nested object if necessary, but I'm trying to keep this simple.
The results of that query would be:
[
{
"id": "doc1",
"permissions": [
"android.permission.ACCESS_COARSE_LOCATION",
"android.permission.BATTERY_STATS"
]
},
{
"id": "doc2",
"permissions": [
"android.permission.ACCESS_FINE_LOCATION"
]
}
]
And now, it's a simple ANY/SATISFIES statement to find the document:
SELECT META(b).id
FROM sstbucket b
WHERE ANY p IN OBJECT_UNWRAP(b).permissions SATISFIES p == 'android.permission.BATTERY_STATS' END;
Which would return
[
{
"id": "doc1"
}
]
So, that works. What I don't know for sure is how to create a proper index for this particular query. I created a primary index just to make it work (CREATE PRIMARY INDEX ON sstbucket), but that's not going to perform very well.
You can use OBJECT functions (https://docs.couchbase.com/server/6.0/n1ql/n1ql-language-reference/objectfun.html) and Array indexing.
If you need document ID or whole document.
CREATE INDEX ix1 ON default ( DISTINCT ARRAY (DISTINCT ARRAY permision
FOR permision IN app.permissions END)
FOR app IN OBJECT_VALUES(data) END);
SELECT META(d).id FROM default AS d
WHERE ANY app IN OBJECT_VALUES(d.data)
SATISFIES (ANY permision IN app.permissions
SATISFIES permision = "android.permission.BATTERY_STATS"
END)
END;
If you need only appId and see if it uses covering index.
CREATE INDEX ix2 ON default ( ALL ARRAY (ALL ARRAY [permision, app.appId]
FOR permision IN app.permissions END)
FOR app IN OBJECT_VALUES(data) END);
SELECT [permision, app.appId][1] AS appId FROM default AS d
UNNEST OBJECT_VALUES(d.data) AS app
UNNEST app.permissions AS permision
WWHERE [permision, app.appId] >= ["android.permission.BATTERY_STATS"] AND
[permision, app.appId] < [SUCCESSOR("android.permission.BATTERY_STATS")] ;

Using angular.js to set selected from a dynamic value in JSON files

Forgive me if this is a newbie question. I'm very new to Angular.
I've looked everywhere for this, and while there are a lot of questions answered about setting a default selected option, I haven't found one where the selected option is set dynamically from values in a JSON file.
My controller looks like this:
peopleControllers.controller('PeopleListCtrl', ['$scope','PeopleList',
function($scope, PeopleList) {
$scope.id = 'id';
$scope.people = PeopleList.query();
$scope.orderProp = 'name';
$scope.comments = 'comments';
$scope.department = 'department';
$scope.departmentList = [
{id : 1, name : "HR" },
{id : 2, name : "Accounting"},
{id : 3, name : "Marketing"}
];
}]);
PeopleList is a $resource that comes from a JSON file formatted like this:
[
{
"id": "johnasmith",
"name": "John A. Smith",
"department": "1",
"comments": "Good worker"
},
{
"id": "sarahqpublic",
"name": "Sarah Q. Public",
"department": "2",
"comments": "New hire"
},
{
"id": "janedoe",
"name": "Jane Doe",
"department": "3",
"comments": "Good resource for information"
}
]
...
And in the HTML, I have this:
ul class="people">
<li ng-repeat="person in people | filter:query | orderBy:orderProp">
<h3>{{people.name}}</h3>
<p>{{people.comments}}</p>
<select ng-model="department" ng-options="departmentList.name for department in departmentList track by department.id">
</select>
</li>
</ul>
The select statement populates with all the right info from departmentList, but its selected value ends up being blank. If I set a static value for $scope.department, like $scope.department = $scope.departmentList[1]; (for "Accounting") it works perfectly. But it doesn't seem to be able to pull the department value from the JSON file.
I know I'm missing something simple and obvious. There have got to be other people who have already asked and had answered this question, so I'm sorry if this turns out to be a duplicate. But I'm really stymied right now.
From Docs
select as and track by Do not use select as and track by in the same
expression. They are not designed to work together.
Additionally as #Chrillewoodz suggested you need to convert the string to number for department id
Markup
<select ng-model="person.department"
ng-options="department.id as department.name for department in departmentList">
</select>
Demo Plunkr
You JSON file contains only strings which is causing you to search for the value "3" in the array instead of selecting [3]. So change your JSON to department: 3 instead of department: "3" and so on.
Like this:
{
"id": "johnasmith",
"name": "John A. Smith",
"department": 1,
"comments": "Good worker"
},
{
"id": "sarahqpublic",
"name": "Sarah Q. Public",
"department": 2,
"comments": "New hire"
},
{
"id": "janedoe",
"name": "Jane Doe",
"department": 3,
"comments": "Good resource for information"
}
Also you should do select ng-model="person.department".

JSON Slurper Offsets

I have a large JSON file that I'm trying to parse with JSON Slurper. The JSON file consists of information about bugs so it has things like issue keys, descriptions, and comments. Not every issue has a comment though. For example, here is a sample of what the JSON input looks like:
{
"projects": [
{
"name": "Test Project",
"key": "TEST",
"issues": [
{
"key": "BUG-1",
"priority": "Major",
"comments": [
{
"author": "a1",
"created": "d1",
"body": "comment 1"
},
{
"author": "a2",
"created": "d2",
"body": "comment 2"
}
]
},
{
"key": "BUG-2",
"priority": "Major"
},
{
"key": "BUG-3",
"priority": "Major",
"comments": [
{
"author": "a3",
"created": "d3",
"body": "comment 3"
}
]
}
]
}
]
}
I have a method that creates Issue objects based on the JSON parse. Everything works well when every issue has at least one comment, but, once an issue comes up that has no comments, the rest of the issues get the wrong comments. I am currently looping through the JSON file based on the total number of issues and then looking for comments using how far along in the number of issues I've gotten. So, for example,
parsedData.issues.comments.body[0][0][0]
returns "comment 1". However,
parsedData.issues.comments.body[0][1][0]
returns "comment 3", which is incorrect. Is there a way I can see if a particular issue has any comments? I'd rather not have to edit the JSON file to add empty comment fields, but would that even help?
You can do this:
parsedData.issues.comments.collect { it?.body ?: [] }
So it checks for a body and if none exists, returns an empty list
UPDATE
Based on the update to the question, you can do:
parsedData.projects.collectMany { it.issues.comments.collect { it?.body ?: [] } }

Customizing JSON output CakePHP

$user = $this->User->find( 'all' );
$this->set( 'users', $user );
I have this code in my controller.
In my view I have this.
echo json_encode( compact( 'users' ) );
It outputs json like this
{
"users": [{
"User": {
"user_id": "2",
"email": "email#test.com",
"name": "Blah"
}]
}
}
Is there anyway to format this to remove the entire array wrapped in "users", and also remove every object being a member of "User".
This makes it harder to use on the front end. I'd like it to look like this.
[{
"user_id": "2",
"email": "email#test.com",
"name": "Blah"
}]
Thanks for any help.
I don't fully understand what you mean by "remove the entire array wrapped in "users"" and "remove every object being a member of "User"", but according to your desired output format example, you'll need to extract and pass the exact data that you want to be encoded to json_encode, instead of passing everything using compact.
Extracting could be done with the Set or the Hash class (depending on your Cake version)
Assuming your model returns the data in the default CakePHP format, this for example:
json_encode(Set::extract('/User/.', $users));
should give you a structure like this:
[{
"user_id": "2",
"email": "email#test.com",
"name": "Blah"
}]
and with multiple users it should look like this
[{
"user_id": "1",
"email": "foo#test.com",
"name": "Bar"
},
{
"user_id": "2",
"email": "email#test.com",
"name": "Blah"
}]
Use like this:
$users= (Set::extract('/User/.', $users));
pr($users);
It will remove Model from result Array and then json_encode or whatever further usage.
More library functions Set class Here and Hash class Here