Related
I am trying to write a json schema to import a csv file. A simplified version of the file is:
000,OUTCOME,111,20220320,FT
001,A.Name1,,,A,
002,A.Name2,,20220101,C,
999,2
000 is a header block
001 & 002 are the body and can have multiple instances. Structurally they are the same
999 is the trailer
the schema I have is
{
"conf_name": "FileTest",
"has_blocks": true,
"has_headers": false,
"enabled_extensions": [
{
"extension": "csv",
"delimiter": ","
}
],
"file_type": "FileTest",
"columns": [
{
"blockName": "000",
"name": "",
"startPosition": "",
"maxLength": "",
"minLength": "",
"data": {
"mandatory": "M",
"type": "text",
"format": "",
"values": [
"OUTCOME"
],
"conditional": ""
}
},
{
"blockName": "000",
"name": "",
"startPosition": "",
"maxLength": "",
"minLength": "",
"data": {
"mandatory": "M",
"type": "int",
"format": "",
"values": "",
"conditional": ""
}
},
{
"blockName": "000",
"name": "",
"startPosition": "",
"maxLength": "",
"minLength": "",
"data": {
"mandatory": "M",
"type": "date",
"format": "yyyyMMdd",
"values": "",
"conditional": ""
}
},
{
"blockName": "000",
"name": "",
"startPosition": "",
"maxLength": "",
"minLength": "",
"data": {
"mandatory": "M",
"type": "text",
"format": "",
"values": "",
"conditional": ""
}
},
{
"blockName": "001",
"name": "",
"startPosition": "",
"maxLength": "",
"minLength": "",
"data": {
"mandatory": "M",
"type": "text",
"format": "",
"values": "",
"conditional": ""
}
},
{
"blockName": "001",
"name": "",
"startPosition": "",
"maxLength": "",
"minLength": "",
"data": {
"mandatory": "O",
"type": "text",
"format": "",
"values": "",
"conditional": ""
}
},
{
"blockName": "001",
"name": "",
"startPosition": "",
"maxLength": "",
"minLength": "",
"data": {
"mandatory": "C",
"type": "date",
"format": "yyyyMMdd",
"values": "",
"conditional": {
"columnName": "Column5",
"operation": "==C THEN NOTEMPTY",
"value": ""
}
}
},
{
"blockName": "001",
"name": "Column5",
"startPosition": "",
"maxLength": "",
"minLength": "",
"data": {
"mandatory": "M",
"type": "text",
"format": "",
"values": [
"A",
"B",
"C"
],
"conditional": ""
}
},
{
"blockName": "001",
"name": "",
"startPosition": "",
"maxLength": "",
"minLength": "",
"data": {
"mandatory": "O",
"type": "text",
"format": "",
"values": [
"Y",
"N"
],
"conditional": ""
}
},
{
"blockName": "002",
"name": "",
"startPosition": "",
"maxLength": "",
"minLength": "",
"data": {
"mandatory": "M",
"type": "text",
"format": "",
"values": "",
"conditional": ""
}
},
{
"blockName": "002",
"name": "",
"startPosition": "",
"maxLength": "",
"minLength": "",
"data": {
"mandatory": "O",
"type": "text",
"format": "",
"values": "",
"conditional": ""
}
},
{
"blockName": "002",
"name": "",
"startPosition": "",
"maxLength": "",
"minLength": "",
"data": {
"mandatory": "C",
"type": "date",
"format": "yyyyMMdd",
"values": "",
"conditional": {
"columnName": "Column5",
"operation": "==C THEN NOTEMPTY",
"value": ""
}
}
},
{
"blockName": "002",
"name": "",
"startPosition": "",
"maxLength": "",
"minLength": "",
"data": {
"mandatory": "M",
"type": "text",
"format": "",
"values": [
"A",
"B",
"C"
],
"conditional": ""
}
},
{
"blockName": "002",
"name": "",
"startPosition": "",
"maxLength": "",
"minLength": "",
"data": {
"mandatory": "O",
"type": "text",
"format": "",
"values": [
"Y",
"N"
],
"conditional": ""
}
},
{
"blockName": "999",
"name": "",
"startPosition": "",
"maxLength": "",
"minLength": "",
"data": {
"mandatory": "M",
"type": "int",
"format": "",
"values": "",
"conditional": ""
}
}
]
}
I apologise now if that is an affront to JSON users everywhere!
This works if there are both 001 and 002 lines in the file, but I've now been told that there may times when the file is all 001 or 002 but not both. Is there a way to make them optional? "oneOf" seemed likely, but I could not fathom how to use it in this situation.
My other issue was with the conditional entry. The data does not have a header row so I'm not sure how to check it. The intention is that if the 5th column contains a "C" then the 4th should have a date in it.
Any help would be gratefully received. Thanks
I have a JSON, from which I need to strip down the name and position values of every subFolder to picture the folder structure from inside the JSON in a ngx-treeview in Angular.
I only want to have the subFolders, including the templates in the subFolders. Since the structure of the JSON object doesn't exactly match at all places and is not homogenous.
How do I dynamically push the subFolder values into the treeview (ngx-treeview) in the exact folder structure from the JSON?
The ngx-treeview JSON should look something like this:
templates = new TreeviewItem({
text: "Abrechnung",
value: 1,
children: [
{
text: "Fahrtkosten",
value: 11,
},
{
text: "Lohn",
value: 12,
},
],
});
My JSON:
{
"soap:Envelope": {
"soap:Body": {
"ns2:getTemplateTreeResponse": {
"return": {
"subFolders": {
"name": "Vorlagen",
"description": "",
"subFolders": {
"name": "VorlagenB",
"description": "",
"subFolders": [
{
"name": "Abrechnung",
"description": "",
"subFolders": [
{
"name": "Fahrtkosten",
"description": "",
"documentTypes": [
{
"fullName": "\\library\\MVP\\Vorlagen\\Fahrtkostenabrechnung.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Fahrtkostenzusatz.template",
"description": "",
"metadata": ""
}
]
},
{
"name": "Lohn",
"description": "",
"documentTypes": [
{
"fullName": "\\library\\MVP\\Vorlagen\\Gehaltsabrechnung.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Lohnbescheid.template",
"description": "",
"metadata": ""
}
]
}
],
"documentTypes": [
{
"fullName": "\\library\\MVP\\Vorlagen\\Fahrtkostenabrechnung.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Fahrtkostenzusatz.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Gehaltsabrechnung.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Lohnbescheid.template",
"description": "",
"metadata": ""
}
]
},
{
"name": "Datenschutz",
"description": "",
"subFolders": [
{
"name": "Datenschutzbeauftragter",
"description": "",
"documentTypes": [
{
"fullName": "\\library\\MVP\\Vorlagen\\DatenschutzerklärungHD.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\DatenschutzerklärungXD.template",
"description": "",
"metadata": ""
}
]
},
{
"name": "Datenschutzerklärung",
"description": "",
"documentTypes": [
{
"fullName": "\\library\\MVP\\Vorlagen\\DatenschutzerklärungKD.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\DatenschutzerklärungMA.template",
"description": "",
"metadata": ""
}
]
}
],
"documentTypes": [
{
"fullName": "\\library\\MVP\\Vorlagen\\DatenschutzerklärungHD.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\DatenschutzerklärungKD.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\DatenschutzerklärungMA.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\DatenschutzerklärungXD.template",
"description": "",
"metadata": ""
}
]
},
{
"name": "Mitarbeiterinfos",
"description": "",
"subFolders": {
"name": "Urlaub",
"description": "",
"documentTypes": [
{
"fullName": "\\library\\MVP\\Template.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Urlaubsantragbescheid.template",
"description": "",
"metadata": ""
}
]
},
"documentTypes": [
{
"fullName": "\\library\\MVP\\Template.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Urlaubsantragbescheid.template",
"description": "",
"metadata": ""
}
]
},
{
"name": "Sonstiges",
"description": "",
"subFolders": [
{
"name": "Freitext Briefe",
"description": "",
"documentTypes": [
{
"fullName": "\\library\\MVP\\Vorlagen\\Anschreiben1.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Anschreiben2.template",
"description": "",
"metadata": ""
}
]
},
{
"name": "Rückfragen",
"description": "",
"documentTypes": [
{
"fullName": "\\library\\MVP\\Vorlagen\\Testvorlage.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\TestvorlageRgfrage.template",
"description": "",
"metadata": ""
}
]
}
],
"documentTypes": [
{
"fullName": "\\library\\MVP\\Vorlagen\\Anschreiben1.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Anschreiben2.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Testvorlage.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\TestvorlageRgfrage.template",
"description": "",
"metadata": ""
}
]
},
{
"name": "Urlaub",
"description": "",
"subFolders": {
"name": "Urlaubsantrag",
"description": "",
"documentTypes": [
{
"fullName": "\\library\\MVP\\Vorlagen\\Urlaubsanfrage.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Urlaubsantrag.template",
"description": "",
"metadata": ""
}
]
},
"documentTypes": [
{
"fullName": "\\library\\MVP\\Vorlagen\\Urlaubsanfrage.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Urlaubsantrag.template",
"description": "",
"metadata": ""
}
]
}
],
"documentTypes": [
{
"fullName": "\\library\\MVP\\Vorlagen\\Anschreiben1.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Anschreiben2.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\DatenschutzerklärungHD.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\DatenschutzerklärungKD.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\DatenschutzerklärungMA.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\DatenschutzerklärungXD.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Fahrtkostenabrechnung.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Fahrtkostenzusatz.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Gehaltsabrechnung.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Lohnbescheid.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Template.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Testvorlage.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\TestvorlageRgfrage.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Urlaubsanfrage.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Urlaubsantrag.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Urlaubsantragbescheid.template",
"description": "",
"metadata": ""
}
]
},
"documentTypes": [
{
"fullName": "\\library\\MVP\\Vorlagen\\Anschreiben1.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Anschreiben2.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\DatenschutzerklärungHD.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\DatenschutzerklärungKD.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\DatenschutzerklärungMA.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\DatenschutzerklärungXD.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Fahrtkostenabrechnung.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Fahrtkostenzusatz.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Gehaltsabrechnung.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Lohnbescheid.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Template.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Testvorlage.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\TestvorlageRgfrage.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Urlaubsanfrage.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Urlaubsantrag.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Urlaubsantragbescheid.template",
"description": "",
"metadata": ""
}
]
}
}
}
}
}
}
#evilstiefel was faster than me but I'm with him. Sounds more like a data-transformation issue instead of an Angular one.
See a small StackBlitz sample to see my solution to the problem: https://stackblitz.com/edit/angular-basic-starter-fvewdw?file=src%2Fapp%2Fapp.component.ts
This sounds more like a data analysis problem than an Angular problem. Consider the following helper method that takes your data structure and returns an array which recursively populates a structure that has a folder-name, associated document types and, if present, an array of sub-folders:
const listName = (node: any, aggregate = []): any[] => {
let documentTypes = [];
if (node) {
if (node.documentTypes && node.documentTypes instanceof Array) {
documentTypes = node.documentTypes.map(docType => docType["fullName"]);
}
}
if (node.subFolders && node.subFolders instanceof Array) {
return aggregate.concat({
name: node.name,
documentTypes,
subFolders: node.subFolders.map(folder => listName(folder))
});
} else if (node.subFolders) {
return aggregate.concat({
name: node.name,
documentTypes,
subFolders: [listName(node.subFolders, aggregate)]
});
} else {
return aggregate.concat({ name: node.name, documentTypes, subFolders: [] });
}
};
const tree =
data["soap:Envelope"]["soap:Body"]["ns2:getTemplateTreeResponse"]["return"][
"subFolders"
];
const structure = listName(tree);
There might very well be a shorter version of this recursive function, but it should give you an idea of how to "destructure" your source JSON. Note that in my example, your json-data was loaded into the variable data.
I have nested JSON of type
[{
"emails": [{
"label": "",
"primary": "",
"relationdef_id": "",
"type": "",
"value": ""
}],
"licenses": [{
"allocated": "",
"parent_type": "",
"parentid": "",
"product_type": "",
"purchased_license_id": "",
"service_type": ""
}, {
"allocated": "",
"parent_type": "",
"parentid": "",
"product_type": "",
"purchased_license_id": "",
"service_type": ""
}]
}, {
"emails": [{
"label": "",
"primary": "",
"relationdef_id": "",
"type": "",
"value": ""
}],
"licenses": [{
"allocated": "2016-04-26 01:46:26",
"parent_type": "",
"parentid": "",
"product_type": "",
"purchased_license_id": "",
"service_type": ""
}]
}]
which is not able to be converted to athena table.
I have tried to update it to list of objects also
{
"emails": [{
"label": "",
"primary": "",
"relationdef_id": "",
"type": "",
"value": ""
}
],
"licenses": [{
"allocated": "",
"parent_type": "",
"parentid": "",
"product_type": "",
"purchased_license_id": "",
"service_type": ""
},{
"allocated": "",
"parent_type": "",
"parentid": "",
"product_type": "",
"purchased_license_id": "",
"service_type": ""
}
]
}
{
"emails": [{
"label": "",
"primary": "",
"relationdef_id": "",
"type": "",
"value": ""
}
],
"licenses": [{
"allocated": "",
"parent_type": "",
"parentid": "",
"product_type": "",
"purchased_license_id": "",
"service_type": ""
}
]
}
{
"emails": [{
"label": "",
"primary": "",
"relationdef_id": "",
"type": "",
"value": ""
}
],
"licenses": [{
"allocated": "",
"parent_type": "",
"parentid": "",
"product_type": "",
"purchased_license_id": "",
"service_type": ""
}
]
}
with Query:
CREATE EXTERNAL TABLE `test_orders1`(
`emails` array<struct<`label`: string, `primary`: string,`relationdef_id`: string,`type`: string, `value`: string>>,
`licenses` array<struct<`allocated`: string, `parent_type`: string, `parentid`: string, `product_type`: string,`purchased_license_id`: string, `service_type`: string>>)
ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe'
WITH SERDEPROPERTIES ( 'ignore.malformed.json' = 'true')
STORED AS INPUTFORMAT
'org.apache.hadoop.mapred.TextInputFormat'
OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
but only 1 row is formed.
Is there a way where i can use Nested json of type JSONArray into Athena table?
Or how can I change Nested Json that will work for me?
When querying JSON data Athena requires the files to be formatted with one JSON document per line. It's unclear from your question if this is the case or not, the examples you give are multiline, but perhaps that's only to make the question more clear.
The table DDL you include looks like it should work on the second example data, provided that it is formatted as one document per line, e.g.
{"emails": [{"label": "", "primary": "", "relationdef_id": "", "type": "", "value": ""}], "licenses": [{"allocated": "", "parent_type": "", "parentid": "", "product_type": "", "purchased_license_id": "", "service_type": ""}, { "allocated": "", "parent_type": "", "parentid": "", "product_type": "", "purchased_license_id": "", "service_type": ""}]}
{"emails": [{"label": "", "primary": "", "relationdef_id": "", "type": "", "value": ""}], "licenses": [{"allocated": "", "parent_type": "", "parentid": "", "product_type": "", "purchased_license_id": "", "service_type": ""}]}
{"emails": [{"label": "", "primary": "", "relationdef_id": "", "type": "", "value": ""}], "licenses": [{"allocated": "", "parent_type": "", "parentid": "", "product_type": "", "purchased_license_id": "", "service_type": ""}]}
I am trying to run search from elastic based on output from another application which dumps data in JSON format. Here is the format :
{
"currentRow": 100,
"fields": [
{ "name": "dDocName" }, { "name": "dDocTitle" }, { "name": "dDocType" }, { "name": "dSecurityGroup" },
{ "name": "dInDate" }, { "name": "xColor" }, { "name": "xPersonType" }, { "name": "xRegionDefinition" },
{ "name": "xLibraryGUID" }, { "name": "dDocLastModifiedDate" }, { "name": "xIdentityNum" },
{ "name": "xLonTriggeyu" }, { "name": "xIntField" }, { "name": "dRevClassID" }, { "name": "xFFTest" },
{ "name": "xWCWorkflowAssignment" }, { "name": "dDocClass" }, { "name": "xWebsiteObjectType" },
{ "name": "xCustomerCode" }, { "name": "xInvoiceNum" }, { "name": "AlternateFormat" }, { "name": "dDocAuthor" },
{ "name": "xfruit" }, { "name": "xSupplierNum" }, { "name": "xEBSParam" }, { "name": "xTestTree" },
{ "name": "xVideoRenditions" }, { "name": "xStorageRule" }, { "name": "xstatecitymemo" },
{ "name": "xPOHeaderId" }, { "name": "xTREELOCATION" }, { "name": "xDamConversionType" },
{ "name": "xInvoiceAmount" }, { "name": "xDiscussionType" }, { "name": "dDocFunction" },
{ "name": "xModifiedBy" }, { "name": "xCustomerTaxPayerId" }, { "name": "dOutDate" },
{ "name": "xIPMSYS_BATCH_SEQ" }, { "name": "dDocLastModifier" }, { "name": "dFormat" },
{ "name": "dRendition2" }, { "name": "dRendition1" }, { "name": "xCustomerName" }, { "name": "xHideThread" },
{ "name": "xGender" }, { "name": "xWCTags" }, { "name": "xExtURL" }, { "name": "xTestFolder1" },
{ "name": "xPackagedConversions" }, { "name": "xClbraRoleList" }, { "name": "xFFTest1" },
{ "name": "xInvoiceCurrency" }, { "name": "dDocCreatedDate" }, { "name": "xWebsites" },
{ "name": "xTestFiddler" }, { "name": "xDontShowInListsForWebsites" }, { "name": "dDocAccount" },
{ "name": "URL" }, { "name": "xClbraUserList" }, { "name": "xAvaya_Region" }, { "name": "dCreateDate" },
{ "name": "dID" }, { "name": "xSri2" }, { "name": "dExtension" }, { "name": "xSri1" },
{ "name": "xfwm_cat_Mercados" }, { "name": "dWebExtension" }, { "name": "xcateg1" }, { "name": "xChecksum" },
{ "name": "xPONum" }, { "name": "dDocCreator" }, { "name": "VaultFileSize" }, { "name": "dRevLabel" },
{ "name": "xFirstName" }, { "name": "xCMUTest" }, { "name": "xDiscussionCount" }, { "name": "xClbraAliasList" },
{ "name": "xPartitionId" }, { "name": "dGif" }, { "name": "xIPMSYS_APP_ID" }, { "name": "dFullTextFormat" },
{ "name": "xTest1" }, { "name": "xFamilyName" }, { "name": "xInvoice" }, { "name": "xInvoiceDate" },
{ "name": "dRevisionID" }, { "name": "xWebsiteSection" }, { "name": "xWCWorkflowApproverUserList" },
{ "name": "WebFileSize" }, { "name": "xComments" }, { "name": "xWebFlag" }, { "name": "xNewtest" },
{ "name": "xOptionListIssue" }, { "name": "xtest" }, { "name": "xIPMSYS_BATCH_ID1" }, { "name": "xIdcProfile" },
{ "name": "dOriginalName" }, { "name": "dDocOwner" }, { "name": "dPublishType" }, { "name": "otsFormat" },
{ "name": "otsCharset" }, { "name": "otsLanguage" }, { "name": "SCORE" }, { "name": "srfDocSnippet" }
],
"rows": [
["WCCPS7_024401", "test1", "EBSAttachment", "AOK-Public", "5/3/167:18AM", "", "", "IDCNULL", "", "5/3/167:19AM", "", "", "0", "24401", "", "", "", "", "", "", "", "wccuser", "", "", "", "0", "", "DispByContentId", "", "", "", "", "", "N/A", "", "", "", "", "", "wccuser", "Application/unknown", "", "", "", "", "", "", "", "", "", "", "", "", "5/3/167:19AM", "", "", "", "", "/cs/groups/aok-public/documents/ebsattachment/czdf/mdi0/~edisp/wccps7_024401", "", "", "5/3/167:19AM", "24801", "", "", "", "", "", "", "2cd4124073fed81c624af0101ba28bda16db650fee35cbc8fd629904dead1b09/SHA-256", "", "wccuser", "377", "1", "", "", "0", "", "", "archiv.gif", "", "", "", "", "", "", "1", "", "", "377", "", "", "", "", "", "0", "EBSProfile", "UntitledDocument", "wccuser", "", "", "", "", "3", ""],
["WCCPS7_024202", "DLEASE_RAW_response", "Document", "AOK-Public", "4/19/1611:11AM", "", "", "IDCNULL", "", "4/19/1611:11AM", "", "", "0", "24202", "", "", "", "", "", "", "", "weblogic", "", "", "", "0", "", "DispByContentId", "", "", "", "", "", "N/A", "", "", "", "", "", "weblogic", "text/plain", "", "", "", "", "", "", "", "", "", "", "", "", "4/19/1611:11AM", "", "", "", "", "/cs/groups/aok-public/documents/document/czdf/mdi0/~edisp/wccps7_024202.txt", "", "", "4/19/1611:11AM", "24402", "", "txt", "", "", "txt", "", "2e2a98a3af833032d4f2b5ec3a8c62b80edeb13ac417d472744c713e4cae27e5/SHA-256", "", "weblogic", "594", "1", "", "", "0", "", "", "ucm_document.png", "", "txt", "", "", "", "", "1", "", "", "594", "", "", "", "", "", "0", "", "DLEASE_RAW_response.txt", "weblogic", "", "", "", "", "3", ""]
// ...
Have dumped this data to a json file and uploaded it to elastic.
I am unable to create a query which would list items / data based on specific values for each of the fields.
For example, how should I set up a query which will return all items where dDocAuthor is weblogic?
First it should be proper json format to index to elasticsearch.For Indexing use BULK API which is provided in elasticsearch documentation.
Then you can query based on your requirement.
{
"ws_result":
[
{
"token": "",
"norm_token": "",
"len": "",
"type": "",
"pos": "",
"prop": "",
"stag": "",
"child":
[
{
"token": "",
"norm_token":"",
"len": "",
"type": "",
"pos": "",
"prop": "",
"stag": "",
"child":
[
{
"token": "",
"norm_token":"",
"len": "",
"type": "",
"pos": "",
"prop": "",
"stag": "",
"child": [ ]
},
{
"token": "",
"norm_token":"",
"len": "",
"type": "",
"pos": "",
"prop": "",
"stag": "",
"child": [ ]
}
]
},
{
"token": "",
"norm_token":"",
"len": "",
"type": "",
"pos": "",
"prop": "",
"stag": "",
"child":
[
{
"token": "",
"norm_token":"",
"len": "",
"type": "",
"pos": "",
"prop": "",
"stag": "",
"child": [ ]
}
]
}
]
},
{
"token": "",
"norm_token":"",
"len": "",
"type": "",
"pos": "",
"prop": "",
"stag": "2",
"child": [ ]
},
{
"token": "",
"norm_token": "",
"len": "",
"type": "",
"pos": "",
"prop": "",
"stag": "",
"child": [ ]
}
]
}
Such that some children are empty some is not, and some children contain more children. How do I actually parse this thing and get what I want. I am totally new with Json, and I am trying to use Gson. What I want is to get a value of a token with specific type in the nested Json. Thanks a lot for any help and directions.
I tried use com.google.gson.stream.JsonReader, but ist not working
JsonReader jsonReader = new JsonReader(new StringReader(result));
jsonReader.beginObject();
while(jsonReader.hasNext()){
String field = jsonReader.nextName();
if (field.equals("type")){
System.out.println(jsonReader.nextString());
} else if (field.equals("token")){
System.out.println(jsonReader.nextString());
} else {
jsonReader.skipValue();
}
}
jsonReader.endObject();
Parse your json recursively like this:
http://snipplr.com/view/71742/java-reflection-and-recursive-json-deserializer-using-gson/
private void parse(JsonObject o, PackagingResponse r){
Iterator<Entry<String, JsonElement>> i = o.entrySet().iterator();
while(i.hasNext()){
Entry<String, JsonElement> e = i.next();
JsonElement el = e.getValue();
if(el.isJsonObject())
parse(el.getAsJsonObject(), r);
//......
}
}