I'm using Talend ETL Tool and extracting data from json files and storing them in Mysql database.
But I get the error while reading in very first json. For reading json I'm using tExtractJSONFileds component.
I'm sure about the configuation set up in talend etl tool its right. I believe there is some problem in json file.
While extracting the component shows error like this
Exception in component tExtractJSONFields_1
javax.xml.stream.XMLStreamException: java.io.IOException: Unexpected symbol: COMMA
at de.odysseus.staxon.base.AbstractXMLStreamReader.initialize(AbstractXMLStreamReader.java:218)
at de.odysseus.staxon.json.JsonXMLStreamReader.<init>(JsonXMLStreamReader.java:65)
at de.odysseus.staxon.json.JsonXMLInputFactory.createXMLStreamReader(JsonXMLInputFactory.java:148)
at de.odysseus.staxon.json.JsonXMLInputFactory.createXMLStreamReader(JsonXMLInputFactory.java:44)
at de.odysseus.staxon.base.AbstractXMLInputFactory.createXMLEventReader(AbstractXMLInputFactory.java:118)
I dont know how to deal with JSONs, So Acc to this error can anyone help me where could be the error in JSON file ?
Is there any value passed as NULL or something else ?
Sample JSON
[
[, {
"tstamp": "123456",
"event": "tgegfght",
"is_duplicate": false,
"farm": "dyhetygdht",
"uid": "tutyvbrtyvtrvy",
"clientip": "52351365136",
"device_os_label": "MICROSOFT_WINDOWS_7",
"device_browser_label": "MOZILLA_FIREFOX",
"geo_country_code": "MA",
"geo_region_code": "55",
"geo_city_name_normalized": "agadir",
"referer": "www.abc.com",
"txn": "etvevv5r",
"txn_isnew": true,
"publisher_id": 126,
"adspot_id": 11179502,
"ad_spot": 5188,
"format_id": 1611,
"misc": {
"PUBLISHER_FOLDER": "retvrect",
"NO_PROMO": "rctrctrc",
"SECTION": "evtrevr",
"U_COMMON_ALLOW": "0",
"U_Auth": "0"
},
"handler": "uint"
}, , ]
Thanks in advance !!
You have extra empty commas in your sample json.
Your Sample Json should look like
[{
"tstamp": "123456",
"event": "tgegfght",
"is_duplicate": false,
"farm": "dyhetygdht",
"uid": "tutyvbrtyvtrvy",
"clientip": "52351365136",
"device_os_label": "MICROSOFT_WINDOWS_7",
"device_browser_label": "MOZILLA_FIREFOX",
"geo_country_code": "MA",
"geo_region_code": "55",
"geo_city_name_normalized": "agadir",
"referer": "www.abc.com",
"txn": "etvevv5r",
"txn_isnew": true,
"publisher_id": 126,
"adspot_id": 11179502,
"ad_spot": 5188,
"format_id": 1611,
"misc": {
"PUBLISHER_FOLDER": "retvrect",
"NO_PROMO": "rctrctrc",
"SECTION": "evtrevr",
"U_COMMON_ALLOW": "0",
"U_Auth": "0"
},
"handler": "uint"
}]
OR
[
{
"somethinghere": "its value"
},
"somethingelse": "its value"
]
Your sample json is not valid json, due to the spurious extra commas on the second and last lines. Json only allows commas BETWEEN elements of a vector or object, and empty elements are not allowed.
Related
So I have a JSON file I got from Postman which is returning as an empty object. This is how I'm reading it.
import regscooter from './json_files/reginald_griffin_scooter.json'
const scoot = regscooter;
const CustomerPage = () => {...}
reginald_griffin_scooter.json
{
"success": true,
"result": {
"id": "hhhhhhhhhh",
"model": "V1 Scooter",
"name": "hhhhhhhhhh",
"status": "active",
"availabilityStatus": "not-available",
"availabilityTrackingOn": true,
"serial": "hhhhhhhhhhhh",
"createdByUser": "hhhhhhhhK",
"createdByUsername": "hhhhhhhh",
"subAssets": [
"F0lOjWBAnG"
],
"parts": [
"hhhhhhhh"
],
"assignedCustomers": [
"hhhhhhhhh"
],
"createdAt": "2019-12-03T21:47:26.218Z",
"updatedAt": "2020-06-26T22:05:54.526Z",
"customFieldsAsset": [
{
"id": "hhhhhhh",
"name": "MAC",
"value": "hhhhhhhh",
"asset": "hhhhhhhhhh",
"user": "hhhhhhhhh",
"createdAt": "2019-12-03T21:47:26.342Z",
"updatedAt": "2019-12-11T16:29:24.732Z"
},
{
"id": "hhhhhhhh",
"name": "IMEI",
"value": "hhhhhhh",
"asset": "hhhhhhh",
"user": "hhhhhhhhhh",
"createdAt": "2019-12-03T21:47:26.342Z",
"updatedAt": "2019-12-11T16:29:24.834Z"
},
{
"id": "hhhhhhhhh",
"name": "Key Number",
"value": "NA",
"asset": "hhhhhhhhh",
"user": "hhhhhhhhhhh",
"createdAt": "2019-12-03T21:47:26.342Z",
"updatedAt": "2019-12-11T16:29:24.911Z"
}
]
}
}
The error is that "const scoot" is being shown as an empty object {}. I made sure to save a ton of times everywhere. I am able to read through the imported JSON file in other variables in similar ways, so I don't know why I can't parse this one. I just want to access the JSON object inside this. Also I omitted some information with hhhhh because of confidentiality.
EDIT: The code works, but it still has a red line beneath result when I do:
const scoot = regscooter.result.id;
It would be much more effective if you will provide an example in codesandbox or so.
However at first look it might be a parser issue ( maybe you are using Webpack with missing configuration for parsing files with json extension ), meaning we need more information to provide you with a full answer ( maybe solution ? ).
Have you tried to do the next:
const scoot = require('./json_files/reginald_griffin_scooter.json');
I have below json from which I wants to extract '07199fca-b43f-4e58-b0fc-c1e254f34ac0' values. I got with multiple regex available in regex101 but every time I get all UUID value any way by which I get Unique value that has no json key assign.
JSON Syntax:
[
[
{
"clientId": 178,
"uniqueId": "5f7f919f-7e0f-4a1e-9a91-89b673896da6",
"displayName": "Automation Client Test",
"productVersion": "7.9.0.0"
},
{
"clientId": 1206,
"uniqueId": "096b3549-6899-4621-854c-e682aeb543bd",
"displayName": "TestClient1",
"productVersion": "7.9.0.0"
},
{
"clientId": 1356,
"uniqueId": "faad0e20-dd29-4146-8a8f-37648749aa4e",
"displayName": "Client Automation",
"productVersion": "7.9.0.0"
}
],
"07199fca-b43f-4e58-b0fc-c1e254f34ac0"
]
From your question it is not 100% clear what you are looking for, but if you are looking for the last element of the list that also satisfies [a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12} regex (with quotes around it), then:
(?:['"])([a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12})(?:['"]\s*]\s*$)
See Regex Demo
(?:['"]) Matches ' or ".
([a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12}) Your regex captured in capture group 1.
(?:['"]\s*]\s*$) Matches ' or " followed by 0 or more white space characters followed by a ] followed by 0 or more white space characters followed by the end of string.
The code:
import re
s = """ [
[
{
"clientId": 178,
"uniqueId": "5f7f919f-7e0f-4a1e-9a91-89b673896da6",
"displayName": "Automation Client Test",
"productVersion": "7.9.0.0"
},
{
"clientId": 1206,
"uniqueId": "096b3549-6899-4621-854c-e682aeb543bd",
"displayName": "TestClient1",
"productVersion": "7.9.0.0"
},
{
"clientId": 1356,
"uniqueId": "faad0e20-dd29-4146-8a8f-37648749aa4e",
"displayName": "Client Automation",
"productVersion": "7.9.0.0"
}
],
"07199fca-b43f-4e58-b0fc-c1e254f34ac0"
]"""
m = re.search(r"""(?:['"])([a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12})(?:['"]\s*]\s*$)""", s, flags=re.DOTALL|re.IGNORECASE)
if m:
print(m[1])
Prints:
07199fca-b43f-4e58-b0fc-c1e254f34ac0
JSON is a well-structured data, using regular expressions for parsing JSON is not the best idea, I would rather recommend going for JSON Extractor instead
The relevant JsonPath query would be as simple as $[1]
Demo:
More information: API Testing With JMeter and the JSON Extractor
I wanted to use osmtogeojson in order to upload osm data to mapbox. However Mapbox tells me "Unknown filetype" and when I analyse any result from osmtogeojson with geojsonhint, I always get :
1:1 error Parse error on line 1:
" t y p e "
^
Expecting 'STRING', 'NUMBER', 'NULL', 'TRUE', 'FALSE', '{', '[', got 'INVALID'
Here is an example of data I obtained this way :
"type": "FeatureCollection",
"features": [
{
"type": "Feature",
"id": "node/1853272897",
"properties": {
"timestamp": "2017-03-11T12:50:59Z",
"version": "4",
"changeset": "46761722",
"user": "sbiribizio",
"uid": "354284",
"name": "Capo Linaro",
"natural": "cape",
"wikidata": "Q3657144",
"wikipedia": "it:Capo Linaro",
"id": "node/1853272897"
},
"geometry": {
"type": "Point",
"coordinates": [
11.8357546,
42.028944
]
}
}
]
}
So I don't know where the problem comes from (I thought of different encodings maybe).
So, yes I just had to save it in UFT-8. However it still doesn't work in mapbox, but the question was for geojsonhint verification, so this is the correct answer!
I'm trying to use some results exported in JSON of a script called "Mixed Content Scan" (it's a script in order to search on a website if there is some mixed HTTP/HTTPS content and if all your pages are ok in HTTPS).
I'm a beginner with JSON, I read and watched a lot of tutorials in order to understand how to structure JSON data but I'm stumbling on something.
Here is a sample of my data (first 3 lines) :
{"message":"Scanning https://mywebsite.com/","context":[],"level":250,"level_name":"NOTICE","channel":"MCS","datetime":{"date":"2018-10-05 23:48:50.268196","timezone_type":3,"timezone":"America/New_York"},"extra":[]}
{"message":"00000 - https://mywebsite.com/","context":[],"level":400,"level_name":"ERROR","channel":"MCS","datetime":{"date":"2018-10-05 23:48:50.760948","timezone_type":3,"timezone":"America/New_York"},"extra":[]}
{"message":"http://mywebsite.com/wp-content/uploads/2015/03/image.jpg","context":[],"level":300,"level_name":"WARNING","channel":"MCS","datetime":{"date":"2018-10-05 23:48:50.761082","timezone_type":3,"timezone":"America/New_York"},"extra":[]}
I know I need to wrap my data around some {} or [] (tried both), but I think I'm missing something, for example, every JSON data validator websites are telling me that I have an error between 2 lines when I add a "," when I try to have multiple results into it.
How can I upgrade this raw data in order for a JSON validator to validate it?
Thanks!
How's this
[{
"message": "Scanning https://mywebsite.com/",
"context": [],
"level": 250,
"level_name": "NOTICE",
"channel": "MCS",
"datetime": {
"date": "2018-10-05 23:48:50.268196",
"timezone_type": 3,
"timezone": "America/New_York"
},
"extra": []
}, {
"message": "00000 - https://mywebsite.com/",
"context": [],
"level": 400,
"level_name": "ERROR",
"channel": "MCS",
"datetime": {
"date": "2018-10-05 23:48:50.760948",
"timezone_type": 3,
"timezone": "America/New_York"
},
"extra": []
}, {
"message": "http://mywebsite.com/wp-content/uploads/2015/03/image.jpg",
"context": [],
"level": 300,
"level_name": "WARNING",
"channel": "MCS",
"datetime": {
"date": "2018-10-05 23:48:50.761082",
"timezone_type": 3,
"timezone": "America/New_York"
},
"extra": []
}]
Entries in an array need to be separated by commas.
My JSON looks something like this:
{
"Master" : {
"Major" : "S",
"Minor" : "E",
"IPAddress" : "0.0.0.0",
"Detail":"<root>
<key keyname=\"state\">3</key>
<key keyname=\"oldState\">1</key>
<key keyname=\"currency\"></key>
<key keyname=\"denomination\"></key></root>",
"SourceCreateDate" : "2014-04-03T14:02:57.182+0200"
},
"Messages" : [{
"MessageCode" : "0",
"MessageType" : "8"
}]
}
I'm getting an 'Invalid Characters found' error when validating this. Where are the invalid characters and how can I make this JSON valid?
{
"Master": {
"Major": "S",
"Minor": "E",
"IPAddress": "0.0.0.0",
"Detail": "<root><key keyname=\"state\">3</key><key keyname=\"oldState">1</key><key keyname=\"currency\"></key><key keyname=\"denomination\"></key></root>",
"SourceCreateDate": "2014-04-03T14:02:57.182+0200"
},
"Messages": [
{
"MessageCode": "0",
"MessageType": "8"
}
]
}
JSON validator: http://jsonlint.com/
Edit: Explication: when you open a " you need to close it on the same line. So you have to put your xml on a single line or to escape it.
JSON only accepts single line Strings.
A work-around would be:
"Detail": [
"<root>",
",<key keyname=\"state\">3</key>",
"<key keyname=\"oldState\">1</key>",
"<key keyname=\"currency\"></key>",
"<key keyname=\"denomination\"></key></root>"
],
You also have the option to replace line breaks into \n.
The invalid characters are the line breaks in the "Detail" element. You'll need to escape them. Something like the solution presented here should work.
try this,
{
"Master": {
"Major": "S",
"Minor": "E",
"IPAddress": "0.0.0.0",
"Detail": "<root><key keyname=\"state\">3</key><key keyname=\"oldState\">1</key><key keyname=\"currency\"></key><key keyname=\"denomination\"></key></root>",
"SourceCreateDate": "2014-04-03T14:02:57.182+0200"
},
"Messages": [
{
"MessageCode": "0",
"MessageType": "8"
}
]
}
i think there were some hidden junk characters were there, like next line(line breaks) or tab spaces etc.. that why it was giving error. so make sure "Details" key will be there in single line