Elasticsearch query format - json

I am trying to run search from elastic based on output from another application which dumps data in JSON format. Here is the format :
{
"currentRow": 100,
"fields": [
{ "name": "dDocName" }, { "name": "dDocTitle" }, { "name": "dDocType" }, { "name": "dSecurityGroup" },
{ "name": "dInDate" }, { "name": "xColor" }, { "name": "xPersonType" }, { "name": "xRegionDefinition" },
{ "name": "xLibraryGUID" }, { "name": "dDocLastModifiedDate" }, { "name": "xIdentityNum" },
{ "name": "xLonTriggeyu" }, { "name": "xIntField" }, { "name": "dRevClassID" }, { "name": "xFFTest" },
{ "name": "xWCWorkflowAssignment" }, { "name": "dDocClass" }, { "name": "xWebsiteObjectType" },
{ "name": "xCustomerCode" }, { "name": "xInvoiceNum" }, { "name": "AlternateFormat" }, { "name": "dDocAuthor" },
{ "name": "xfruit" }, { "name": "xSupplierNum" }, { "name": "xEBSParam" }, { "name": "xTestTree" },
{ "name": "xVideoRenditions" }, { "name": "xStorageRule" }, { "name": "xstatecitymemo" },
{ "name": "xPOHeaderId" }, { "name": "xTREELOCATION" }, { "name": "xDamConversionType" },
{ "name": "xInvoiceAmount" }, { "name": "xDiscussionType" }, { "name": "dDocFunction" },
{ "name": "xModifiedBy" }, { "name": "xCustomerTaxPayerId" }, { "name": "dOutDate" },
{ "name": "xIPMSYS_BATCH_SEQ" }, { "name": "dDocLastModifier" }, { "name": "dFormat" },
{ "name": "dRendition2" }, { "name": "dRendition1" }, { "name": "xCustomerName" }, { "name": "xHideThread" },
{ "name": "xGender" }, { "name": "xWCTags" }, { "name": "xExtURL" }, { "name": "xTestFolder1" },
{ "name": "xPackagedConversions" }, { "name": "xClbraRoleList" }, { "name": "xFFTest1" },
{ "name": "xInvoiceCurrency" }, { "name": "dDocCreatedDate" }, { "name": "xWebsites" },
{ "name": "xTestFiddler" }, { "name": "xDontShowInListsForWebsites" }, { "name": "dDocAccount" },
{ "name": "URL" }, { "name": "xClbraUserList" }, { "name": "xAvaya_Region" }, { "name": "dCreateDate" },
{ "name": "dID" }, { "name": "xSri2" }, { "name": "dExtension" }, { "name": "xSri1" },
{ "name": "xfwm_cat_Mercados" }, { "name": "dWebExtension" }, { "name": "xcateg1" }, { "name": "xChecksum" },
{ "name": "xPONum" }, { "name": "dDocCreator" }, { "name": "VaultFileSize" }, { "name": "dRevLabel" },
{ "name": "xFirstName" }, { "name": "xCMUTest" }, { "name": "xDiscussionCount" }, { "name": "xClbraAliasList" },
{ "name": "xPartitionId" }, { "name": "dGif" }, { "name": "xIPMSYS_APP_ID" }, { "name": "dFullTextFormat" },
{ "name": "xTest1" }, { "name": "xFamilyName" }, { "name": "xInvoice" }, { "name": "xInvoiceDate" },
{ "name": "dRevisionID" }, { "name": "xWebsiteSection" }, { "name": "xWCWorkflowApproverUserList" },
{ "name": "WebFileSize" }, { "name": "xComments" }, { "name": "xWebFlag" }, { "name": "xNewtest" },
{ "name": "xOptionListIssue" }, { "name": "xtest" }, { "name": "xIPMSYS_BATCH_ID1" }, { "name": "xIdcProfile" },
{ "name": "dOriginalName" }, { "name": "dDocOwner" }, { "name": "dPublishType" }, { "name": "otsFormat" },
{ "name": "otsCharset" }, { "name": "otsLanguage" }, { "name": "SCORE" }, { "name": "srfDocSnippet" }
],
"rows": [
["WCCPS7_024401", "test1", "EBSAttachment", "AOK-Public", "5/3/167:18AM", "", "", "IDCNULL", "", "5/3/167:19AM", "", "", "0", "24401", "", "", "", "", "", "", "", "wccuser", "", "", "", "0", "", "DispByContentId", "", "", "", "", "", "N/A", "", "", "", "", "", "wccuser", "Application/unknown", "", "", "", "", "", "", "", "", "", "", "", "", "5/3/167:19AM", "", "", "", "", "/cs/groups/aok-public/documents/ebsattachment/czdf/mdi0/~edisp/wccps7_024401", "", "", "5/3/167:19AM", "24801", "", "", "", "", "", "", "2cd4124073fed81c624af0101ba28bda16db650fee35cbc8fd629904dead1b09/SHA-256", "", "wccuser", "377", "1", "", "", "0", "", "", "archiv.gif", "", "", "", "", "", "", "1", "", "", "377", "", "", "", "", "", "0", "EBSProfile", "UntitledDocument", "wccuser", "", "", "", "", "3", ""],
["WCCPS7_024202", "DLEASE_RAW_response", "Document", "AOK-Public", "4/19/1611:11AM", "", "", "IDCNULL", "", "4/19/1611:11AM", "", "", "0", "24202", "", "", "", "", "", "", "", "weblogic", "", "", "", "0", "", "DispByContentId", "", "", "", "", "", "N/A", "", "", "", "", "", "weblogic", "text/plain", "", "", "", "", "", "", "", "", "", "", "", "", "4/19/1611:11AM", "", "", "", "", "/cs/groups/aok-public/documents/document/czdf/mdi0/~edisp/wccps7_024202.txt", "", "", "4/19/1611:11AM", "24402", "", "txt", "", "", "txt", "", "2e2a98a3af833032d4f2b5ec3a8c62b80edeb13ac417d472744c713e4cae27e5/SHA-256", "", "weblogic", "594", "1", "", "", "0", "", "", "ucm_document.png", "", "txt", "", "", "", "", "1", "", "", "594", "", "", "", "", "", "0", "", "DLEASE_RAW_response.txt", "weblogic", "", "", "", "", "3", ""]
// ...
Have dumped this data to a json file and uploaded it to elastic.
I am unable to create a query which would list items / data based on specific values for each of the fields.
For example, how should I set up a query which will return all items where dDocAuthor is weblogic?

First it should be proper json format to index to elasticsearch.For Indexing use BULK API which is provided in elasticsearch documentation.
Then you can query based on your requirement.

Related

Making blocks optional and conditional entry where there is no header name

I am trying to write a json schema to import a csv file. A simplified version of the file is:
000,OUTCOME,111,20220320,FT
001,A.Name1,,,A,
002,A.Name2,,20220101,C,
999,2
000 is a header block
001 & 002 are the body and can have multiple instances. Structurally they are the same
999 is the trailer
the schema I have is
{
"conf_name": "FileTest",
"has_blocks": true,
"has_headers": false,
"enabled_extensions": [
{
"extension": "csv",
"delimiter": ","
}
],
"file_type": "FileTest",
"columns": [
{
"blockName": "000",
"name": "",
"startPosition": "",
"maxLength": "",
"minLength": "",
"data": {
"mandatory": "M",
"type": "text",
"format": "",
"values": [
"OUTCOME"
],
"conditional": ""
}
},
{
"blockName": "000",
"name": "",
"startPosition": "",
"maxLength": "",
"minLength": "",
"data": {
"mandatory": "M",
"type": "int",
"format": "",
"values": "",
"conditional": ""
}
},
{
"blockName": "000",
"name": "",
"startPosition": "",
"maxLength": "",
"minLength": "",
"data": {
"mandatory": "M",
"type": "date",
"format": "yyyyMMdd",
"values": "",
"conditional": ""
}
},
{
"blockName": "000",
"name": "",
"startPosition": "",
"maxLength": "",
"minLength": "",
"data": {
"mandatory": "M",
"type": "text",
"format": "",
"values": "",
"conditional": ""
}
},
{
"blockName": "001",
"name": "",
"startPosition": "",
"maxLength": "",
"minLength": "",
"data": {
"mandatory": "M",
"type": "text",
"format": "",
"values": "",
"conditional": ""
}
},
{
"blockName": "001",
"name": "",
"startPosition": "",
"maxLength": "",
"minLength": "",
"data": {
"mandatory": "O",
"type": "text",
"format": "",
"values": "",
"conditional": ""
}
},
{
"blockName": "001",
"name": "",
"startPosition": "",
"maxLength": "",
"minLength": "",
"data": {
"mandatory": "C",
"type": "date",
"format": "yyyyMMdd",
"values": "",
"conditional": {
"columnName": "Column5",
"operation": "==C THEN NOTEMPTY",
"value": ""
}
}
},
{
"blockName": "001",
"name": "Column5",
"startPosition": "",
"maxLength": "",
"minLength": "",
"data": {
"mandatory": "M",
"type": "text",
"format": "",
"values": [
"A",
"B",
"C"
],
"conditional": ""
}
},
{
"blockName": "001",
"name": "",
"startPosition": "",
"maxLength": "",
"minLength": "",
"data": {
"mandatory": "O",
"type": "text",
"format": "",
"values": [
"Y",
"N"
],
"conditional": ""
}
},
{
"blockName": "002",
"name": "",
"startPosition": "",
"maxLength": "",
"minLength": "",
"data": {
"mandatory": "M",
"type": "text",
"format": "",
"values": "",
"conditional": ""
}
},
{
"blockName": "002",
"name": "",
"startPosition": "",
"maxLength": "",
"minLength": "",
"data": {
"mandatory": "O",
"type": "text",
"format": "",
"values": "",
"conditional": ""
}
},
{
"blockName": "002",
"name": "",
"startPosition": "",
"maxLength": "",
"minLength": "",
"data": {
"mandatory": "C",
"type": "date",
"format": "yyyyMMdd",
"values": "",
"conditional": {
"columnName": "Column5",
"operation": "==C THEN NOTEMPTY",
"value": ""
}
}
},
{
"blockName": "002",
"name": "",
"startPosition": "",
"maxLength": "",
"minLength": "",
"data": {
"mandatory": "M",
"type": "text",
"format": "",
"values": [
"A",
"B",
"C"
],
"conditional": ""
}
},
{
"blockName": "002",
"name": "",
"startPosition": "",
"maxLength": "",
"minLength": "",
"data": {
"mandatory": "O",
"type": "text",
"format": "",
"values": [
"Y",
"N"
],
"conditional": ""
}
},
{
"blockName": "999",
"name": "",
"startPosition": "",
"maxLength": "",
"minLength": "",
"data": {
"mandatory": "M",
"type": "int",
"format": "",
"values": "",
"conditional": ""
}
}
]
}
I apologise now if that is an affront to JSON users everywhere!
This works if there are both 001 and 002 lines in the file, but I've now been told that there may times when the file is all 001 or 002 but not both. Is there a way to make them optional? "oneOf" seemed likely, but I could not fathom how to use it in this situation.
My other issue was with the conditional entry. The data does not have a header row so I'm not sure how to check it. The intention is that if the 5th column contains a "C" then the 4th should have a date in it.
Any help would be gratefully received. Thanks

How do I strip this JSON in Angular?

I have a JSON, from which I need to strip down the name and position values of every subFolder to picture the folder structure from inside the JSON in a ngx-treeview in Angular.
I only want to have the subFolders, including the templates in the subFolders. Since the structure of the JSON object doesn't exactly match at all places and is not homogenous.
How do I dynamically push the subFolder values into the treeview (ngx-treeview) in the exact folder structure from the JSON?
The ngx-treeview JSON should look something like this:
templates = new TreeviewItem({
text: "Abrechnung",
value: 1,
children: [
{
text: "Fahrtkosten",
value: 11,
},
{
text: "Lohn",
value: 12,
},
],
});
My JSON:
{
"soap:Envelope": {
"soap:Body": {
"ns2:getTemplateTreeResponse": {
"return": {
"subFolders": {
"name": "Vorlagen",
"description": "",
"subFolders": {
"name": "VorlagenB",
"description": "",
"subFolders": [
{
"name": "Abrechnung",
"description": "",
"subFolders": [
{
"name": "Fahrtkosten",
"description": "",
"documentTypes": [
{
"fullName": "\\library\\MVP\\Vorlagen\\Fahrtkostenabrechnung.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Fahrtkostenzusatz.template",
"description": "",
"metadata": ""
}
]
},
{
"name": "Lohn",
"description": "",
"documentTypes": [
{
"fullName": "\\library\\MVP\\Vorlagen\\Gehaltsabrechnung.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Lohnbescheid.template",
"description": "",
"metadata": ""
}
]
}
],
"documentTypes": [
{
"fullName": "\\library\\MVP\\Vorlagen\\Fahrtkostenabrechnung.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Fahrtkostenzusatz.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Gehaltsabrechnung.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Lohnbescheid.template",
"description": "",
"metadata": ""
}
]
},
{
"name": "Datenschutz",
"description": "",
"subFolders": [
{
"name": "Datenschutzbeauftragter",
"description": "",
"documentTypes": [
{
"fullName": "\\library\\MVP\\Vorlagen\\DatenschutzerklärungHD.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\DatenschutzerklärungXD.template",
"description": "",
"metadata": ""
}
]
},
{
"name": "Datenschutzerklärung",
"description": "",
"documentTypes": [
{
"fullName": "\\library\\MVP\\Vorlagen\\DatenschutzerklärungKD.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\DatenschutzerklärungMA.template",
"description": "",
"metadata": ""
}
]
}
],
"documentTypes": [
{
"fullName": "\\library\\MVP\\Vorlagen\\DatenschutzerklärungHD.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\DatenschutzerklärungKD.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\DatenschutzerklärungMA.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\DatenschutzerklärungXD.template",
"description": "",
"metadata": ""
}
]
},
{
"name": "Mitarbeiterinfos",
"description": "",
"subFolders": {
"name": "Urlaub",
"description": "",
"documentTypes": [
{
"fullName": "\\library\\MVP\\Template.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Urlaubsantragbescheid.template",
"description": "",
"metadata": ""
}
]
},
"documentTypes": [
{
"fullName": "\\library\\MVP\\Template.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Urlaubsantragbescheid.template",
"description": "",
"metadata": ""
}
]
},
{
"name": "Sonstiges",
"description": "",
"subFolders": [
{
"name": "Freitext Briefe",
"description": "",
"documentTypes": [
{
"fullName": "\\library\\MVP\\Vorlagen\\Anschreiben1.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Anschreiben2.template",
"description": "",
"metadata": ""
}
]
},
{
"name": "Rückfragen",
"description": "",
"documentTypes": [
{
"fullName": "\\library\\MVP\\Vorlagen\\Testvorlage.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\TestvorlageRgfrage.template",
"description": "",
"metadata": ""
}
]
}
],
"documentTypes": [
{
"fullName": "\\library\\MVP\\Vorlagen\\Anschreiben1.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Anschreiben2.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Testvorlage.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\TestvorlageRgfrage.template",
"description": "",
"metadata": ""
}
]
},
{
"name": "Urlaub",
"description": "",
"subFolders": {
"name": "Urlaubsantrag",
"description": "",
"documentTypes": [
{
"fullName": "\\library\\MVP\\Vorlagen\\Urlaubsanfrage.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Urlaubsantrag.template",
"description": "",
"metadata": ""
}
]
},
"documentTypes": [
{
"fullName": "\\library\\MVP\\Vorlagen\\Urlaubsanfrage.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Urlaubsantrag.template",
"description": "",
"metadata": ""
}
]
}
],
"documentTypes": [
{
"fullName": "\\library\\MVP\\Vorlagen\\Anschreiben1.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Anschreiben2.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\DatenschutzerklärungHD.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\DatenschutzerklärungKD.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\DatenschutzerklärungMA.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\DatenschutzerklärungXD.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Fahrtkostenabrechnung.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Fahrtkostenzusatz.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Gehaltsabrechnung.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Lohnbescheid.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Template.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Testvorlage.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\TestvorlageRgfrage.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Urlaubsanfrage.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Urlaubsantrag.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Urlaubsantragbescheid.template",
"description": "",
"metadata": ""
}
]
},
"documentTypes": [
{
"fullName": "\\library\\MVP\\Vorlagen\\Anschreiben1.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Anschreiben2.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\DatenschutzerklärungHD.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\DatenschutzerklärungKD.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\DatenschutzerklärungMA.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\DatenschutzerklärungXD.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Fahrtkostenabrechnung.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Fahrtkostenzusatz.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Gehaltsabrechnung.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Lohnbescheid.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Template.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Testvorlage.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\TestvorlageRgfrage.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Urlaubsanfrage.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Urlaubsantrag.template",
"description": "",
"metadata": ""
},
{
"fullName": "\\library\\MVP\\Vorlagen\\Urlaubsantragbescheid.template",
"description": "",
"metadata": ""
}
]
}
}
}
}
}
}
#evilstiefel was faster than me but I'm with him. Sounds more like a data-transformation issue instead of an Angular one.
See a small StackBlitz sample to see my solution to the problem: https://stackblitz.com/edit/angular-basic-starter-fvewdw?file=src%2Fapp%2Fapp.component.ts
This sounds more like a data analysis problem than an Angular problem. Consider the following helper method that takes your data structure and returns an array which recursively populates a structure that has a folder-name, associated document types and, if present, an array of sub-folders:
const listName = (node: any, aggregate = []): any[] => {
let documentTypes = [];
if (node) {
if (node.documentTypes && node.documentTypes instanceof Array) {
documentTypes = node.documentTypes.map(docType => docType["fullName"]);
}
}
if (node.subFolders && node.subFolders instanceof Array) {
return aggregate.concat({
name: node.name,
documentTypes,
subFolders: node.subFolders.map(folder => listName(folder))
});
} else if (node.subFolders) {
return aggregate.concat({
name: node.name,
documentTypes,
subFolders: [listName(node.subFolders, aggregate)]
});
} else {
return aggregate.concat({ name: node.name, documentTypes, subFolders: [] });
}
};
const tree =
data["soap:Envelope"]["soap:Body"]["ns2:getTemplateTreeResponse"]["return"][
"subFolders"
];
const structure = listName(tree);
There might very well be a shorter version of this recursive function, but it should give you an idea of how to "destructure" your source JSON. Note that in my example, your json-data was loaded into the variable data.

Develop bargraphs from json object list directly

I have multiple json scripts with a similar format as:
{
"name": "xyz",
"slug": "xyz",
"supplier": "xyz.limited",
"attributes": [
{
"name": "mass",
"productConfiguration": "base",
"description": "",
"value": "0.24",
"minimumValue": "",
"maximumValue": "",
"measurementUnit": "kg",
"uuid": "",
"attributeClassUuid": "",
"productUuid": ""
},
{
"name": "length",
"productConfiguration": "base",
"description": "assumed to be length",
"value": "58.0",
"minimumValue": "",
"maximumValue": "",
"measurementUnit": "mm",
"uuid": "",
"attributeClassUuid": "",
"productUuid": ""
},
{
"name": "height",
"productConfiguration": "base",
"description": "assumed to be height",
"value": "25.0",
"minimumValue": "",
"maximumValue": "",
"measurementUnit": "mm",
"uuid": "",
"attributeClassUuid": "",
"productUuid": ""
},
{
"name": "width",
"productConfiguration": "base",
"description": "assumed to be width",
"value": "58.0",
"minimumValue": "",
"maximumValue": "",
"measurementUnit": "mm",
"uuid": "",
"attributeClassUuid": "",
"productUuid": ""
},
{
"name": "momentum-storage",
"productConfiguration": "base",
"description": "",
"value": "0.050",
"minimumValue": "",
"maximumValue": "",
"measurementUnit": "N m s",
"uuid": "",
"attributeClassUuid": "",
"productUuid": ""
},
{
"name": "maximum-torque",
"productConfiguration": "base",
"description": "",
"value": "0.007",
"minimumValue": "",
"maximumValue": "",
"measurementUnit": "N m",
"uuid": "",
"attributeClassUuid": "",
"productUuid": ""
},
{
"name": "voltage",
"productConfiguration": "base",
"description": "voltage is given in DC current",
"value": "12.0",
"minimumValue": "",
"maximumValue": "",
"measurementUnit": "V",
"uuid": "",
"attributeClassUuid": "",
"productUuid": ""
},
{
"name": "maximum-power",
"productConfiguration": "base",
"description": "",
"value": "9.0",
"minimumValue": "",
"maximumValue": "",
"measurementUnit": "W",
"uuid": "",
"attributeClassUuid": "",
"productUuid": ""
},
{
"name": "lifetime",
"productConfiguration": "base",
"description": "",
"value": "",
"minimumValue": "5.0",
"maximumValue": "",
"measurementUnit": "yr",
"uuid": "",
"attributeClassUuid": "",
"productUuid": ""
}
]
}
Each of these files have different set of attributes defined. I would like to plot the number of times each attribute is repeated across all files. I try to do this using python3.5 functionalities 'os' and 'panda.DataFrame'. However, got lost somewhere! Can use some help with this. Thanks in advance!

Can I split compressed files into json components in Camel via Spring DSL

In a nutshell, I need to take a gzipped file containing json very similar to this example, unzip it (I know how to do that), get each json object as a string and push it to AMQ from where it will be popped to a webservice. I'm fine with all of this with one object, but I will be receiving a file that represents an array. If this were an array of strings or xml, I see how Camel processes it, but I don't see a way to split json. Also, this will require streaming as these files can be very large. Edited to try to make request clearer, and provide a sample json.
[
{
"rickenbackerRepair": {
"estimateId": 22788411
},
"repairShop": {
"inspectionSite": {
"inspectionDate": ""
},
"repairFacility": {
"companyIdCode": "",
"companyName": "",
"city": "",
"stateProvince": "",
"zipPostalCode": "",
"country": ""
},
"repairInformation": {
"guitarDateInShop": "",
"guitarTimeInShop": "",
"authorizationMemo": "",
"guitarTargetCompletionDate": "",
"guitarTargetCompletionTime": "",
"guitarCompletionDate": "",
"guitarCompletionTime": ""
},
"locationOfguitar": {
"city": "",
"stateProvince": "",
"zipPostalCode": "",
"country": ""
}
},
"instrumentIdentifier": {
"guitar": {
"claimRelated": {
"primaryPointOfImpact": "",
"secondaryPointOfImpact": ""
},
"identification": {
"databaseguitarCode": "",
"manufacturingStateProvince": "",
"serialNumber": "",
"guitarCondition": "",
"productionDate": "",
"year": "",
"model": "",
"guitarType": "",
"bodyStyle": "",
"trimCode": "",
"trimColor": "",
"optionsList": ""
}
}
},
"lin": [{
"internalControl": {
"lineIndicator": ""
},
"part": {
"description": {
"partType": "",
"descriptionJudgmentFlag": "",
"oemPartNumber": "",
"priceIncludedIndicator": "",
"alternatePartIndicator": "",
"taxableFlag": "",
"databasePartPrice": "",
"actualPartPrice": "",
"priceJudgmentFlag": "",
"certifiedFlag": "",
"quantity": ""
},
"nonOemSupplier": {
"companyIdCode": "",
"nonOemPartNumber": "",
"nonOemSupplierUserOverride": "",
"nonOemSupplierMemo": ""
},
"adjustment": {
"percent": "",
"amount": ""
}
},
"labor": {
"description": {
"type": "",
"actualHours": "",
"hoursJudgmentFlag": "",
"typeJudgmentFlag": ""
},
"miscSublet": {
"amount": "",
"subletFlag": ""
}
}
}],
"stl": {
"subtotal": [{
"totalType": "",
"totalTypeCode": "",
"subtotalDetail": {
"taxableAmount": ""
}
}]
}
},
{
"rickenbackerRepair": {
"estimateId": 22788412
},
"repairShop": {
"inspectionSite": {
"inspectionDate": ""
},
"repairFacility": {
"companyIdCode": "",
"companyName": "",
"city": "",
"stateProvince": "",
"zipPostalCode": "",
"country": ""
},
"repairInformation": {
"guitarDateInShop": "",
"guitarTimeInShop": "",
"authorizationMemo": "",
"guitarTargetCompletionDate": "",
"guitarTargetCompletionTime": "",
"guitarCompletionDate": "",
"guitarCompletionTime": ""
},
"locationOfguitar": {
"city": "",
"stateProvince": "",
"zipPostalCode": "",
"country": ""
}
},
"instrumentIdentifier": {
"guitar": {
"claimRelated": {
"primaryPointOfImpact": "",
"secondaryPointOfImpact": ""
},
"identification": {
"databaseguitarCode": "",
"manufacturingStateProvince": "",
"serialNumber": "",
"guitarCondition": "",
"productionDate": "",
"year": "",
"model": "",
"guitarType": "",
"bodyStyle": "",
"trimCode": "",
"trimColor": "",
"optionsList": ""
}
}
},
"lin": [{
"internalControl": {
"lineIndicator": ""
},
"part": {
"description": {
"partType": "",
"descriptionJudgmentFlag": "",
"oemPartNumber": "",
"priceIncludedIndicator": "",
"alternatePartIndicator": "",
"taxableFlag": "",
"databasePartPrice": "",
"actualPartPrice": "",
"priceJudgmentFlag": "",
"certifiedFlag": "",
"quantity": ""
},
"nonOemSupplier": {
"companyIdCode": "",
"nonOemPartNumber": "",
"nonOemSupplierUserOverride": "",
"nonOemSupplierMemo": ""
},
"adjustment": {
"percent": "",
"amount": ""
}
},
"labor": {
"description": {
"type": "",
"actualHours": "",
"hoursJudgmentFlag": "",
"typeJudgmentFlag": ""
},
"miscSublet": {
"amount": "",
"subletFlag": ""
}
}
}],
"stl": {
"subtotal": [{
"totalType": "",
"totalTypeCode": "",
"subtotalDetail": {
"taxableAmount": ""
}
}]
}
}
]
You should be able to use a jsonpath expression to split the incoming message (file) and process each element individually.
<route>
<from uri="file://path" />
<split>
<jsonpath>$.</jsonpath>
<to uri="direct:doSomething">
</split>
</route>

How to Parse this Json using Gson and get the field I want?

{
"ws_result":
[
{
"token": "",
"norm_token": "",
"len": "",
"type": "",
"pos": "",
"prop": "",
"stag": "",
"child":
[
{
"token": "",
"norm_token":"",
"len": "",
"type": "",
"pos": "",
"prop": "",
"stag": "",
"child":
[
{
"token": "",
"norm_token":"",
"len": "",
"type": "",
"pos": "",
"prop": "",
"stag": "",
"child": [ ]
},
{
"token": "",
"norm_token":"",
"len": "",
"type": "",
"pos": "",
"prop": "",
"stag": "",
"child": [ ]
}
]
},
{
"token": "",
"norm_token":"",
"len": "",
"type": "",
"pos": "",
"prop": "",
"stag": "",
"child":
[
{
"token": "",
"norm_token":"",
"len": "",
"type": "",
"pos": "",
"prop": "",
"stag": "",
"child": [ ]
}
]
}
]
},
{
"token": "",
"norm_token":"",
"len": "",
"type": "",
"pos": "",
"prop": "",
"stag": "2",
"child": [ ]
},
{
"token": "",
"norm_token": "",
"len": "",
"type": "",
"pos": "",
"prop": "",
"stag": "",
"child": [ ]
}
]
}
Such that some children are empty some is not, and some children contain more children. How do I actually parse this thing and get what I want. I am totally new with Json, and I am trying to use Gson. What I want is to get a value of a token with specific type in the nested Json. Thanks a lot for any help and directions.
I tried use com.google.gson.stream.JsonReader, but ist not working
JsonReader jsonReader = new JsonReader(new StringReader(result));
jsonReader.beginObject();
while(jsonReader.hasNext()){
String field = jsonReader.nextName();
if (field.equals("type")){
System.out.println(jsonReader.nextString());
} else if (field.equals("token")){
System.out.println(jsonReader.nextString());
} else {
jsonReader.skipValue();
}
}
jsonReader.endObject();
Parse your json recursively like this:
http://snipplr.com/view/71742/java-reflection-and-recursive-json-deserializer-using-gson/
private void parse(JsonObject o, PackagingResponse r){
Iterator<Entry<String, JsonElement>> i = o.entrySet().iterator();
while(i.hasNext()){
Entry<String, JsonElement> e = i.next();
JsonElement el = e.getValue();
if(el.isJsonObject())
parse(el.getAsJsonObject(), r);
//......
}
}