I need to index a patent catalog that has the following data structure:
"cpc": [
{
"class": "61",
"section": "A",
"sequence": "1",
"subclass": "K",
"subgroup": "06",
"main-group": "45",
"classification-value": "I"
},
{
"class": "61",
"section": "A",
"sequence": "2",
"subclass": "K",
"subgroup": "506",
"main-group": "31",
"classification-value": "I"
}
]
I was wondering what is the right approach here. I might be able to use cpc.class and combine it with multiValued="true".
I would like to find documents that match a certain CPC code. The CPC code can be partial. Right now my solution is simply use a nested reference with multiValued=true. Is there a better way of doing this?
<field name="cpc.class" type="int" indexed="true" stored="true" multiValued="true" />
<field name="cpc.section" type="string" indexed="true" stored="true" multiValued="true" />
<field name="cpc.sequence" type="int" indexed="true" stored="true" multiValued="true" />
<field name="cpc.subclass" type="string" indexed="true" stored="true" multiValued="true" />
<field name="cpc.subgroup" type="int" indexed="true" stored="true" multiValued="true" />
<field name="cpc.main-group" type="int" indexed="true" stored="true" multiValued="true" />
<field name="cpc.classification-value" type="string" indexed="true" stored="true" multiValued="true" />
The problem with this implementation is that it returns documents not actually matching the search criteria. Example:
"cpc.section:A",
"cpc.class:61",
"cpc.subclass:Q",
"cpc.main-group:8"
I get documents not having this combination. I think the current way implements the search so that every field is a list and matching values in any combination are returned. I need to narrow it down so only the right combinations are returned.
The best way to index this with Solr is to split out the nested data structures (cpcs) to flat documents and have the patent_id included there. That way an arbitrary combination of partial cpcs can be searched.
Related
We are trying to build a JasperReport for an estimate via JSON. The header and lines (multiple) are fine. However, there is one more node in the JSON file which requires to be looped in the detail table. We are not able to achieve it using subdataset. Currently, we are using JSONQL to iterate the lines.
We are unable to loop the taxes node via subdataset.
Below is the sample JSON input:
{
"taxes": [
{
"tax_slab_name": "VAT Exempt",
"tax_amount": "AED 20.00"
},
{
"tax_slab_name": "VAT 0%",
"tax_amount": "AED 30.00"
},
{
"tax_slab_name": "VAT 5 %",
"tax_amount": "AED 50.00"
}
],
"header": [
{
"estimate_no": "EST-000054",
"bill_to_address": "111 Stamm Cliffs Suite 285, Heaney Ville, Jebel Ali, Dubai, United Arab Emirates (UAE)",
"ship_to_address": "66015 شارع بلال السهلي, ممر ميسر مدني, Jebel Ali, Dubai, United Arab Emirates (UAE)",
"estimate_date": "2020-05-08",
"expiry_date": "2020-05-23",
"sub_total": 14000,
"discount": 0,
"tax_amt": 450,
"total": 14450,
"conditions": "All charges are in United States Dollar (USD) unless otherwise stated. E. and O.E.",
"status": "SAVED",
"contact_name": "Kerluke, Bartell and Dickinson (مجموعة الداوود وأولاده)"
}
],
"lines": [
{
"item_name": "Enterprise Product Development - Java/JSF/PF",
"tax_slab": 1,
"uom": "box",
"quantity": 1,
"rate": 1000,
"amount": 1000
},
{
"item_name": "Server Setup",
"tax_slab": 2,
"uom": "cm",
"quantity": 2,
"rate": 2000,
"amount": 4000
},
{
"item_name": "Backup and DR Service",
"tax_slab": 2,
"uom": "cm",
"quantity": 2,
"rate": 2000,
"amount": 4000
}
]
}
Datasource Type: JSONQL
JasperReport Version: 6.12.2 (latest)
Never mind. Got it working with sub-data-set.
<subDataset name="TaxSDS" uuid="1701ce05-88ab-4435-85c5-3ad38e0b941b">
<queryString language="jsonql">
<![CDATA[..taxes]]>
</queryString>
<field name="tax_slab_name" class="java.lang.String">
<property name="net.sf.jasperreports.jsonql.field.expression" value="tax_slab_name"/>
<fieldDescription><![CDATA[Tax Slabs]]></fieldDescription>
</field>
<field name="tax_slab_amt" class="java.lang.String">
<property name="net.sf.jasperreports.jsonql.field.expression" value="tax_amount"/>
<fieldDescription><![CDATA[Tax Amount]]></fieldDescription>
</field>
</subDataset>
<subDataset name="LinesSDS" uuid="1701ce05-88ab-4435-85c5-3ad38e0b941b">
<queryString language="jsonql">
<![CDATA[..lines]]>
</queryString>
<field name="item_name" class="java.lang.String">
<property name="net.sf.jasperreports.jsonql.field.expression" value="item_name"/>
<fieldDescription><![CDATA[Item]]></fieldDescription>
</field>
<field name="uom" class="java.lang.String">
<property name="net.sf.jasperreports.jsonql.field.expression" value="uom"/>
<fieldDescription><![CDATA[UOM]]></fieldDescription>
</field>
</subDataset>
<queryString language="jsonql">
<![CDATA[]]>
</queryString>
<field name="invoiceNo" class="java.lang.String">
<property name="net.sf.jasperreports.jsonql.field.expression" value="header.estimate_no"/>
<fieldDescription><![CDATA[Estimate No]]></fieldDescription>
</field>
my json is as follows
"values": [
{
"purchase": {
"name":"bags"
},
"weekSpend": [
{
"weekStartDate": 20181105,
"spend":100
},
{
"weekStartDate": 20181112,
"spend":200
}
]
},
{
"purchase": {
"name":"shoes"
},
"weekSpend": [
{
"weekStartDate": 20181105,
"spend":100
},
{
"weekStartDate": 20181112,
"spend":200
}
]
},
]
I want to displayed the grid where rows are
productname and (columns by weekstartdate)
I followed the technique provided in How to show column at Crosstab even the data is absent
However the productname are listed in row but the column are showing only first column.
my data set is as follows
<queryString language="jsonql">
<![CDATA[]]>
</queryString>
<field name="productName" class="java.lang.String">
<property name="net.sf.jasperreports.jsonql.field.expression" value="product.name"/>
<fieldDescription><![CDATA[product.name]]></fieldDescription>
</field>
<field name="weekStartDate" class="java.lang.String[]">
<property name="net.sf.jasperreports.jsonql.field.expression" value="weekSpend.weekStartDate"/>
</field>
<group name="activityDateGroup">
<groupExpression><![CDATA[$F{weekStartDate}]]></groupExpression>
</group>
my crossdata datasource is as follows
<crosstabDataset isDataPreSorted="true">
<dataset>
<datasetRun subDataset="crossTabDataSet" uuid="e7b27508-8a48-4785-a48e-c646249df9a9">
<dataSourceExpression><![CDATA[((net.sf.jasperreports.engine.data.JsonQLDataSource)$P{REPORT_DATA_SOURCE}).subDataSource("..values(#size > 0).*")]]></dataSourceExpression>
</datasetRun>
</dataset>
</crosstabDataset>
and bucketexpression is on weekstartdate.
With these using JsonQL still only the first column is displayed
How do i get list of column based on weekstardate from iterating from rootnode.
i am using jasper studio 6.5.
Thanks
Anjana.
There is an invalid field mapping: weekStartDate to java.lang.String[], that is not supported in JSONQL.
Instead you could have this subDataset expression:
<datasetRun subDataset="crossTabDataSet" uuid="e7b27508-8a48-4785-a48e-c646249df9a9">
<dataSourceExpression><![CDATA[((net.sf.jasperreports.engine.data.JsonQLDataSource)$P{REPORT_DATA_SOURCE}).subDataSource("..values(#size > 0)..weekSpend.*")]]></dataSourceExpression>
</datasetRun>
with these field mappings:
<field name="productName" class="java.lang.String">
<property name="net.sf.jasperreports.jsonql.field.expression" value="^{2}.purchase.name"/>
</field>
<field name="weekStartDate" class="java.lang.Integer">
<property name="net.sf.jasperreports.jsonql.field.expression" value="weekStartDate"/>
</field>
Hi Am trying to index mysql db to solr. But its only indexing id. I tried a lot. But no success. But I tried to index geo location in the same db for testing and its indexing all the cities successfully. I just what to know what am missing. This query contains some descriptions of the products like "Solr not returning results indexed from MySQL".
My db config
<entity name="geo"
pk="id"
query="select id, title from `inlive`"
deltaImportQuery="select id, title from `inlive`"
deltaQuery="select id, title from `inlive`"
>
<field column="id" name="id"/>
<field column="title" name="title"/>
</entity>
Schema
<?xml version="1.0" encoding="UTF-8" ?>
<uniqueKey>id</uniqueKey>
<schema name="geo" version="1.5">
<field name="_version_" type="long" indexed="true" stored="true"/>
<field name="id" type="string" indexed="true" stored="true" required="true" multiValued="false" />
<field name="title" type="string" indexed="true" stored="true" multiValued="false" required="true" />
</schema>
Result
{
"responseHeader":{
"status":0,
"QTime":3,
"params":{
"q":"*:*",
"indent":"on",
"wt":"json",
"_":"1506369405758"}},
"response":{"numFound":2,"start":0,"docs":[
{
"id":"1",
"_version_":1579543508130201600},
{
"id":"2",
"_version_":1579543508218281984}]
}}
Debug
{
"responseHeader": {
"status": 0,
"QTime": 275
},
"initArgs": [
"defaults",
[
"config",
"data-config.xml"
]
],
"command": "full-import",
"mode": "debug",
"documents": [
{
"id": [
1
],
"_version_": [
1579544391043776500
]
},
{
"id": [
2
],
"_version_": [
1579544391046922200
]
}
],
"verbose-output": [
"entity:geo",
[
"document#1",
[
"query",
"select id, title from `inlive`",
"time-taken",
"0:0:0.10",
null,
"----------- row #1-------------",
"id",
1,
"title",
"test title",
null,
"---------------------------------------------"
],
"document#2",
[
null,
"----------- row #1-------------",
"id",
2,
"title",
"test title 2",
null,
"---------------------------------------------"
],
"document#3",
[]
]
],
"status": "idle",
"importResponse": "",
"statusMessages": {
"Total Requests made to DataSource": "1",
"Total Rows Fetched": "2",
"Total Documents Processed": "2",
"Total Documents Skipped": "0",
"Full Dump Started": "2017-09-25 20:21:57",
"": "Indexing completed. Added/Updated: 2 documents. Deleted 0 documents.",
"Committed": "2017-09-25 20:21:57",
"Optimized": "2017-09-25 20:21:57",
"Time taken": "0:0:0.255"
}
}
Finally I found the issue for this solr.
Am using solr 6.6
So I've to rename managed-schema to schema as per this tutorial in this link http://makble.com/how-to-index-mysql-data-in-solr. Also I made a test run with geo cities. By default it took managed schema as main schema it took my old test schema fields. Its not getting schema.xml.
So the solution is need to define schema.xml
<schemaFactory class="ClassicIndexSchemaFactory"/>
Reference link : Data-config.xml and mysql - I can load only "id" column
or rename managed-schema to schema.xml or delete managed-schema
I am using a nested JSON file as mockup data in a ODATA SAPUI5 application, but I cannot access the nested data.
JSON content
[{
"testcase": {
"specification": "SRS PR 28717 – Deposit in Brazilian Reais",
"execution": {
"description": "DESC",
"bca_cn_acct_01": {
"header": {
"section": "Field / Section Name",
"data": "Data to Enter / Value to Select",
"action": "Activity / Check / Comment"
},
"frame": {
"ID": 1,
"title": "Create Account: Initial Screen",
"data": [{
"key": "Contract Start",
"value": "02/16/2000",
"action": ""
}, {
"key": "Contract Manager",
"value": "GH_RAMOSCL",
"action": ""
}, {
"key": "Product",
"value": "BR_IOFTC3",
"action": ""
}, {
"key": "Account Holder",
"value": "GH_IOF_COR",
"action": "Press Enter"
}]
}
}
},
"result": "Teste"
}}]
In my view file, I want to access the data inside the tag, as a list:
<core:View controllerName="sap.ui.demo.MockServer.controller.App" xmlns:core="sap.ui.core" xmlns:html="http://www.w3.org/1999/xhtml"
xmlns="sap.m">
<!-- <List headerText="{i18n>headerText}" id="list" items="{/Meetups}" noDataText="{i18n>noDataText}">
<items>
<ObjectListItem number="{ path: 'EventDate', type: 'sap.ui.model.type.DateTime', formatOptions: { style: 'medium' } }" title="{Title}">
<attributes>
<ObjectAttribute text="{Description}"/>
</attributes>
</ObjectListItem>
</items>
</List>
<Button press="onPressAction" text="{i18n>loadFirstItems}"></Button> -->
<List headerText="{i18n>headerText}" id="list" items="{/Meetups}" noDataText="{i18n>noDataText}">
<items>
<ObjectListItem number="{ path: 'EventDate', type: 'sap.ui.model.type.DateTime', formatOptions: { style: 'medium' } }"
title="{testcase/execution/description}">
<attributes>
<ObjectAttribute text="{specification}"/>
</attributes>
</ObjectListItem>
</items>
</List>
<Table id="table" items="{/Meetups}">
<columns>
<Column width="12em">
<Text text="Field / Section Name"/>
</Column>
<Column minScreenWidth="Tablet" demandPopin="true">
<Text text="Data to Enter / Value to Select"/>
</Column>
<Column minScreenWidth="Tablet" demandPopin="true">
<Text text="Activity / Check / Comment"/>
</Column>
</columns>
<items>
<ColumnListItem>
<cells>
<ObjectIdentifier title="{testcase/execution/bca_cn_acct_01/frame/data/key}" class="sapMTableContentMargin"/>
<!-- <Text text="{testcase/execution/bca_cn_acct_01/bca_dte_event_begin_d}" /> -->
<Text text="{testcase/execution/bca_cn_acct_01/frame/data/value}"/>
</cells>
</ColumnListItem>
</items>
</Table>
<Button press="onPressAction" text="{i18n>loadFirstItems}"></Button>
However, data from fields key and value are not retrieved as list, although they are an array inside the JSON data tag.
I use as base the sample tutorial from SAPUI5 library: https://sapui5.hana.ondemand.com/#docs/guide/7a78f1b707c248fd9ec53dcb5f10814c.html
And below you can find the metadata file I have been using:
<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<edmx:Edmx Version="1.0"
xmlns:edmx="http://schemas.microsoft.com/ado/2007/06/edmx">
<edmx:DataServices
xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata" m:DataServiceVersion="1.0">
<Schema Namespace="NerdMeetup.Models"
xmlns:d="http://schemas.microsoft.com/ado/2007/08/dataservices"
xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata"
xmlns="http://schemas.microsoft.com/ado/2006/04/edm">
<EntityType Name="Meetup">
<ComplexType Name="testcase">
<ComplexType Name="execution">
<!-- Create Payment Item -->
<ComplexType Name="bca_cn_acct_01">
<ComplexType Name="header">
<Property Name="section" Type="Edm.String" Nullable="false" />
<Property Name="data" Type="Edm.String" Nullable="false" />
<Property Name="action" Type="Edm.String" Nullable="false" />
</ComplexType>
<ComplexType Name="frame">
<Property Name="ID" Type="Edm.Int32" Nullable="false" />
<Property Name="title" Type="Edm.String" Nullable="false" />
<ComplexType Name="data">
<Property Name="key" Type="Edm.Int32" Nullable="false" />
<Property Name="value" Type="Edm.Int32" Nullable="false" />
<Property Name="action" Type="Edm.Int32" Nullable="true" />
</ComplexType>
</ComplexType>
<Property Name="bca_dte_event_begin_d" Type="Edm.String" Nullable="false" />
<Property Name="bca_dte_orgunit_cnmgt" Type="Edm.String" Nullable="false" />
<Property Name="fspr_prodext_y" Type="Edm.String" Nullable="false" />
<Property Name="bca_dte_bupa_acchold" Type="Edm.String" Nullable="false" />
</ComplexType>
</ComplexType>
<Property Name="specification" Type="Edm.String" Nullable="true" />
</ComplexType>
</EntityType>
<EntityContainer Name="NerdMeetups" m:IsDefaultEntityContainer="true">
<EntitySet Name="Meetups" EntityType="NerdMeetup.Models.Meetup" />
<FunctionImport Name="FindUpcomingMeetups" EntitySet="Meetups" ReturnType="Collection(NerdMeetup.Models.Meetup)" m:HttpMethod="GET" />
</EntityContainer>
</Schema>
</edmx:DataServices>
</edmx:Edmx>
Any ideas?
Hugs
I guess you can either bind the entire array:
{testcase/execution/bca_cn_acct_01/frame/data/}
Or bind an specific element of the array:
{testcase/execution/bca_cn_acct_01/frame/data/0/key}
You could also use a formatter to receive an array and concatenate all the keys of the array check the
Formatter SAPUI5 documentation
I am learning solr (5.5.0) and running in a standalone mode.
Here is the solr-data-config.xml:
<dataConfig>
<dataSource type="JdbcDataSource" driver="com.mysql.jdbc.Driver" url="jdbc:mysql://127.0.0.1/DBN" user="root" password="root"/>
<document>
<entity name="planexample" query="select * from plan where userid=60 limit 3">
<field column="planid" name="id" />
<field column="userid" name="userid" />
<field column="planname" name="planname" />
<field column="del" name="del" />
</entity>
</document>
</dataConfig>
And I add fields accordingly into managed-schema:
<field name="userid" type="int" indexed="true" stored="true" required="true" multiValued="true"/>
<field name="planname" type="string" indexed="true" stored="true" required="true" multiValued="true"/>
<field name="del" type="int" indexed="true" stored="true" required="true" multiValued="true"/>
<field name="id" type="string" indexed="true" stored="true" required="true" multiValued="false" />
But when I do query on the webpage of solr, for example, ip:8983/solr/corename/select?q=del%3A1&wt=json&indent=true or ip:8983/solr/corename/select?q=userid%3A60&wt=json&indent=true, results only show the id except userid,planname,del, how to make those fields visible in results? And what does the score field mean in results?
One example of results:
{
"responseHeader": {
"status": 0,
"QTime": 1,
"params": {
"indent": "true",
"q": "del:1",
"_": "1461050814466",
"wt": "json"
}
},
"response": {
"numFound": 10,
"start": 0,
"maxScore": 0.90468985,
"docs": [
{
"id": "985963",
"score": 0.90468985
},
{
"id": "1033572",
"score": 0.90468985
},
{
"id": "1763216",
"score": 0.90468985
}
]
}
}
You need to provide a fl parameter, to get all stored fields,
do &fl=*, or a comma seperated list of fields you want i.e.
&fl=id,userid,planname,del for example.