I am having some data coming from csv which has \n character in it and I expect neo4j to add a new line when assigning that string to some attribute in node. Apparently its not working. I can see \n character as it is added in the string.
How to make it work? Thanks in Advance.
Following is one such string example from CSV:
Combo 4 4 4 5 \n\nSpare Fiber Inventory. \nMultimode Individual fibers from 9927/9928 to FDB.\nNo available spares from either BTS to FDB - New conduits would be required\n\nFrom FDB to tower top. 9 of 9 Spares available on 2.5 riser cables.
My load command:
USING PERIODIC COMMIT 500
LOAD CSV WITH HEADERS
FROM 'file:///abc.csv' AS line
WITH line WHERE line.parent <> "" AND line.type = 'LSD' AND line.parent_type = 'XYZ'
This is a hack that I made to replace the occurrences of \n with a newline. The character \ is an escape character so it will replace \n with a new line in line 4. Do not remove line 5 and combine with line 4.
LOAD CSV WITH HEADERS
FROM 'file:///abc.csv' AS line
WITH line WHERE line.parent <> ""
WITH replace(line.parent,'\\n',"
") as parent
MERGE (p:Parent {parent: parent})
RESULT:
{
"identity": 16,
"labels": [
"Parent"
],
"properties": {
"parent": "Combo 4 4 4 5
Spare Fiber Inventory.
Multimode Individual fibers from 9927/9928 to FDB.
No available spares from either BTS to FDB - New conduits would be required
From FDB to tower top. 9 of 9 Spares available on 2.5 riser cables."
}
}
Introduction
Hi! I'm trying to extract JSon from a 300K line text file that has a combination of Text output and JSon format from HTTP Result. The big size in lines makes it unable to retain the JSon manually.
Problematic
Don't have much choice, i probably need to fix it manually using a command-line. Here's how it's looks like inside the file:
[2K 100.00% - C: 164148 / 164149 - S: 263 - F: 3686 - dhcp-140-247-148-215.fas.harvard.edu:443 - id3.sshws.me
[2K 100.00% - C: 164149 / 164149 - S: 263 - F: 3686 - public-1300503051.cos.ap-shanghai.myqcloud.com:443 - id3.sshws.me
[2K
[
{
"Request": {
"ProxyHost": "pro.ant.design",
"ProxyPort": 443,
"Bug": "pro.ant.design",
"Method": "HEAD",
"Target": "id3.sshws.me",
"Payload": "GET wss://pro.ant.design/ HTTP/1.1[crlf]Host: [host][crlf]Upgrade: websocket[crlf][crlf]"
},
"ResponseLine": [
"HTTP/1.1 101 Switching Protocol",
"Server: cloudflare"
]
},
{
"Request": {
"ProxyHost": "industrialtech.ft.com",
"ProxyPort": 443,
"Bug": "industrialtech.ft.com",
"Method": "HEAD",
"Target": "id3.sshws.me",
"Payload": "GET wss://industrialtech.ft.com/ HTTP/1.1[crlf]Host: [host][crlf]Upgrade: websocket[crlf][crlf]"
},
"ResponseLine": [
"HTTP/1.1 101 Switching Protocol",
"Server: cloudflare"
]
}
]
Several problem to this if using RegEx is:
It has multiple JSon object
The Text string that doesn't part of JSon has [ and :
I realize the problem when trying to use sed regex.
sed '/^[/,/^]/!d'
You can remove all lines that start with [ and any non-whitespace char:
sed '/^\[[^[:space:]]/d' file > newfile
Details:
^ - start of a line
\[ - [ char
[^[:space:]] - any non-whitespace chars.
Alternate way to this is; to get advantage for special char. If someone wanted to remove the progress bar from the output and extract only appropiate output:
Use nano <output_file>
You will see that there's new line unicode got readed as ^M^[ in the first text. I assume it's the same as [crlf]
Use sed -e "/\^M^[/d" remove lines that contains specific unicode.
Use \ to escape ^ as RegEx.
Make sure to always find the pattern readed from the terminal and not inside a text editor app, as some of them couldn't read Unicodes.
"{
"LEINumber": "335800QRNLKAHGA1BL68",
"LegalName": "METROPOLITAN CLEARING CORPORATION OF INDIA LIMITED",
"NextRenewalDate": "04-03-2022 00:00:00 +05:30",
"LegalForm": "Public Limited Companies",
"RegistrationStatus": "ISSUED"
}"
In above Json Packet i want to remove the "" in JSON. So after removing the "" i want the data in below format.
{
"LEINumber": "335800QRNLKAHGA1BL68",
"LegalName": "METROPOLITAN CLEARING CORPORATION OF INDIA LIMITED",
"NextRenewalDate": "04-03-2022 00:00:00 +05:30",
"LegalForm": "Public Limited Companies",
"RegistrationStatus": "ISSUED"
}
Well for this simple solution is to print the data not in " " in apigee. If you use double inverted commas it will print the data with commas but if you print the data without double inverted commas it will not print in commas as shown below.
{
"LEINumber": "335800QRNLKAHGA1BL68",
"LegalName": "METROPOLITAN CLEARING CORPORATION OF INDIA LIMITED",
"NextRenewalDate": "04-03-2022 00:00:00 +05:30",
"LegalForm": "Public Limited Companies",
"RegistrationStatus": "ISSUED"
}
Hello there i am working on a dataset but its not formatted correctly. It's missing its Square brackets and comma after every object
Example:
{"is_sarcastic": 1, "headline": "thirtysomething scientists unveil doomsday clock of hair loss", "article_link": "something"}
{"is_sarcastic": 0, "headline": "dem rep. totally nails why congress is falling short on gender, racial equality", "article_link": "somethingelse"}
I want to format it such that it turns to this:
[{"is_sarcastic": 1, "headline": "thirtysomething scientists unveil doomsday clock of hair loss", "article_link": "something"},
{"is_sarcastic": 0, "headline": "dem rep. totally nails why congress is falling short on gender, racial equality", "article_link": "somethingelse"}]
I am using Python 3.x to achieve this task.
you can run the following python script, it'll output a file containing the output you desire.
import json
dataJson = []
with open('data.json') as f:
for jsonObj in f:
dataDict = json.loads(jsonObj)
dataJson.append(dataDict)
#print (dataJson)
with open('data2.json', 'w') as jsonfile:
json.dump(dataJson, jsonfile)
where data.json is the name of the file containing the dataset
I am getting unexpected token error when trying to parse this JSON string
$scope.feeds = JSON.parse('[{"id":"212216417436_10152811286407437","from":{ "category":"Movie","name":"The Lord of the Rings Trilogy","id":"212216417436"},"story":"The Lord of the Rings Trilogy shared The Hobbit\'s photo.","picture":"https://fbcdn-sphotos-c-a.akamaihd.net/hphotos-ak-xaf1/v/t1.0-9/s130x130/1912397_736719376363670_7288336626443028501_n.jpg?oh=a7e83093f61fafa7909ff84263f989e4&oe=54B4481D&__gda__=1420472696_dbd6462629a00e218c849c5ed7b49c44","link":"https://www.facebook.com/TheHobbitMovie/photos/a.291674244201521.63650.160617097307237/736719376363670/?type=1","name":"Timeline Photos","caption":"\"I have the only right.\" \n\nWhat is your favorite Thorin quote from #TheHobbit?","properties":[ { "name":"By","text":"The Hobbit","href":"https://www.facebook.com/TheHobbitMovie?ref=stream"}],"icon":"https://fbstatic-a.akamaihd.net/rsrc.php/v2/yD/r/aS8ecmYRys0.gif","actions":[ { "name":"Comment","link":"https://www.facebook.com/212216417436/posts/10152811286407437"},{ "name":"Like","link":"https://www.facebook.com/212216417436/posts/10152811286407437"}],"privacy":{ "value":""},"type":"photo","status_type":"shared_story","object_id":"736719376363670","application":{ "name":"Photos","id":"2305272732"},"created_time":"2014-10-12T21:35:41+0000","updated_time":"2014-10-12T21:37:17+0000"},{ "id":"21785951839_10152794856921840","from":{ "category":"App page","name":"9GAG","id":"21785951839"},"message":"Must NOT let my girlfriend see this!\nhttp://9gag.tv/p/a9OrlY?ref=fbl9\n\nWatch more awesome videos on your phone:\nAndroid ► http://goo.gl/Rbi9FP\niPhone/iPad ► http://goo.gl/NFMm9p","picture":"https://fbexternal-a.akamaihd.net/safe_image.php?d=AQCee84KqIBIdnIX&w=158&h=158&url=http%3A%2F%2Fd1d4324mp0stc8.cloudfront.net%2Fmedia%2Fphoto%2FpPgXmO5QJ_600w_v1.jpg","link":"http://9gag.tv/p/a9OrlY?ref=fbl9","name":"This \"Marry Me\" Lip-Sync Proposal Is So Touching And Amazing | 9GAG.tv","caption":"9gag.tv","description":"This sets the bar pretty high!","icon":"https://fbstatic-a.akamaihd.net/rsrc.php/v2/yD/r/aS8ecmYRys0.gif","actions":[ { "name":"Comment","link":"https://www.facebook.com/21785951839/posts/10152794856921840"},{ "name":"Like","link":"https://www.facebook.com/21785951839/posts/10152794856921840"}],"privacy":{ "value":""},"type":"link","status_type":"shared_story","created_time":"2014-10-12T21:15:00+0000","updated_time":"2014-10-12T21:15:46+0000","shares":{ "count":160}}]');
Expanded string object:
{
"id":"212216417436_10152811286407437",
"from":{
"category":"Movie",
"name":"The Lord of the Rings Trilogy",
"id":"212216417436"
},
"story":"The Lord of the Rings Trilogy shared The Hobbit\'s photo.",
"picture":"https://fbcdn-sphotos-c-a.akamaihd.net/hphotos-ak-xaf1/v/t1.0-9/s130x130/1912397_736719376363670_7288336626443028501_n.jpg?oh=a7e83093f61fafa7909ff84263f989e4&oe=54B4481D&__gda__=1420472696_dbd6462629a00e218c849c5ed7b49c44",
"link":"https://www.facebook.com/TheHobbitMovie/photos/a.291674244201521.63650.160617097307237/736719376363670/?type=1",
"name":"Timeline Photos",
"caption":"\"I have the only right.\" \n\nWhat is your favorite Thorin quote from #TheHobbit?",
"properties":[
{
"name":"By",
"text":"The Hobbit",
"href":"https://www.facebook.com/TheHobbitMovie?ref=stream"
}
],
"icon":"https://fbstatic-a.akamaihd.net/rsrc.php/v2/yD/r/aS8ecmYRys0.gif",
"actions":[
{
"name":"Comment",
"link":"https://www.facebook.com/212216417436/posts/10152811286407437"
},
{
"name":"Like",
"link":"https://www.facebook.com/212216417436/posts/10152811286407437"
}
],
"privacy":{
"value":""
},
"type":"photo",
"status_type":"shared_story",
"object_id":"736719376363670",
"application":{
"name":"Photos",
"id":"2305272732"
},
"created_time":"2014-10-12T21:35:41+0000",
"updated_time":"2014-10-12T21:37:17+0000"
},
{
"id":"21785951839_10152794856921840",
"from":{
"category":"App page",
"name":"9GAG",
"id":"21785951839"
},
"message":"Must NOT let my girlfriend see this!\nhttp://9gag.tv/p/a9OrlY?ref=fbl9\n\nWatch more awesome videos on your phone:\nAndroid ► http://goo.gl/Rbi9FP\niPhone/iPad ► http://goo.gl/NFMm9p",
"picture":"https://fbexternal-a.akamaihd.net/safe_image.php?d=AQCee84KqIBIdnIX&w=158&h=158&url=http%3A%2F%2Fd1d4324mp0stc8.cloudfront.net%2Fmedia%2Fphoto%2FpPgXmO5QJ_600w_v1.jpg",
"link":"http://9gag.tv/p/a9OrlY?ref=fbl9",
"name":"This \"Marry Me\" Lip-Sync Proposal Is So Touching And Amazing | 9GAG.tv",
"caption":"9gag.tv",
"description":"This sets the bar pretty high!",
"icon":"https://fbstatic-a.akamaihd.net/rsrc.php/v2/yD/r/aS8ecmYRys0.gif",
"actions":[
{
"name":"Comment",
"link":"https://www.facebook.com/21785951839/posts/10152794856921840"
},
{
"name":"Like",
"link":"https://www.facebook.com/21785951839/posts/10152794856921840"
}
],
"privacy":{
"value":""
},
"type":"link",
"status_type":"shared_story",
"created_time":"2014-10-12T21:15:00+0000",
"updated_time":"2014-10-12T21:15:46+0000",
"shares":{
"count":160
}
}
verify your json string. If it is ok try this:
var jsonStr="your json string";
var json=JSON.stringify(jsonStr);
json=JSON.parse(json)
When you're typing a string in javascript, the \ is treated as escaping character, while you need it in your json to actually be a \ and not escaping character.
You need to change the \" to \\" and \n to \\n, while keeping the \' because it is already an escaping character and it should be parsed okay