I have a json structure like this:
{
"foo": "123",
"bar" : {
"baz" : ["1","2","3"],
"faz" : "hello"
}
}
Which I am trying to represent in Solr 6.2 and this schema fails to give me the expected result:
<field name="_root_" type="string" docValues="false" indexed="true" stored="false" />
<field name="foo" type="string" indexed="true" stored="true"/>
<field name="bar" type="string" indexed="true" stored="true"/>
<field name="bar.baz" type="strings" indexed="true" stored="true"/>
<field name="bar.faz" type="string" indexed="true" stored="true"/>
The resulting schema is this:
{
"foo": "123",
"bar" : "",
"bar.baz" : ["1","2","3"],
"bar.faz" : "hello"
}
use multivalued=true for baz
dont use bar.baz. just give baz and faz
change these fields
<field name="bar.baz" type="strings" indexed="true" stored="true"/>
<field name="bar.faz" type="string" indexed="true" stored="true"/>
to
<field name="baz" type="string" multiValued="true" indexed="true" stored="true"/>
<field name="faz" type="string" indexed="true" stored="true"/>
Related
I want to index two tables from MySQL using Apache Solr. Please see my data-config and schema files below.
<dataConfig>
<dataSource type="JdbcDataSource" driver="com.mysql.jdbc.Driver" url="jdbc:mysql://localhost:3306/test" user="root" password="root" batchSize="1" />
<document name="tb_location">
<entity name="tb_location" query="SELECT * FROM tb_location">
<field column="loc_code" name="id"/>
<field column="loc_code" name="loc_code"/>
<field column="loc_name" name="loc_name"/>
<field column="loc_name" name="loc_name_ci"/>
<field column="ADM1_FULL_NAME" name="state"/>
</entity>
</document>
<document name="person">
<entity name="person" query="SELECT * FROM person">
<field column="id" name="personid"/>
<field column="fname" name="fname"/>
<field column="lname" name="lname"/>
<field column="town" name="town"/>
</entity>
</document>
</dataConfig>
Schema.xml
<field name="id" type="string" indexed="true" stored="true" required="true" multiValued="false" /> -
<field name="loc_code" type="string" indexed="true" stored="true" required="true" multiValued="false" />
<field name="loc_name" type="string" indexed="true" stored="true" required="true" multiValued="false" />
<field name="loc_name_ci" type="string_ci" indexed="true" stored="true" required="true" multiValued="false" />
<field name="state" type="string" indexed="true" stored="true" required="true" multiValued="false" />
<field name="personid" type="string" indexed="true" stored="true" required="true" multiValued="false" />
<field name="fname" type="string" indexed="true" stored="true" required="true" multiValued="false" />
<field name="lname" type="string" indexed="true" stored="true" required="true" multiValued="false" />
<field name="town" type="string" indexed="true" stored="true" required="true" multiValued="false" />
Also i created unique id for each tables (id and personid). But when i execute the dataimport module, nothing is fetched or indexed. Can someone help me to figure out where exactly the problem ?
Please check the below link for Multiple indexes...
Multiple indexes
Fixed it !!! data-config.xml should be as follows.
<dataConfig>
<dataSource type="JdbcDataSource" driver="com.mysql.jdbc.Driver" url="jdbc:mysql://localhost:3306/test" user="root" password="root" batchSize="1" />
<document name="tb_location">
<entity name="tb_location" query="SELECT * FROM tb_location">
<field column="loc_code" name="id"/>
<field column="loc_code" name="loc_code"/>
<field column="loc_name" name="loc_name"/>
<field column="loc_name" name="loc_name_ci"/>
<field column="ADM1_FULL_NAME" name="state"/>
</entity>
<entity name="person" query="SELECT * FROM person">
<field column="id" name="personid"/>
<field column="fname" name="fname"/>
<field column="lname" name="lname"/>
<field column="town" name="town"/>
</entity>
</document>
</dataConfig>
I did a basic solr setup, Configured dataImportHandler and create very simple data config file with two fields and indexed it. It all worked fine.. But now I am adding new fields there and doing full import after that but for some reason new fields are just not showing in search result ( using solr interface for search). I have tried restarting solr, running config-reload to no effect.
this is my data config file. Not sure what's wrong here.
<dataConfig>
<dataSource type="JdbcDataSource" driver="com.mysql.jdbc.Driver" url="jdbc:mysql://localhost/msl4" user="root" password=""/>
<document>
<entity name="hub_contents" query="select * from hub_contents" deltaQuery="select * from hub_contents where last_modified > '${dataimporter.last_index_time}'">
<field column="id_original" name="id" />
<field column="title" name="title" />
<field column="parent_id" name="parent_id" />
<field column="item_type" name="item_type" />
<field column="status" name="status" />
<field column="updated_at" name="updated_at" />
</entity>
</document>
</dataConfig>
You can add the below fields in your schema.xml
<field name="id" type="long" indexed="true" stored="true"/>
<field name="title" type="text_general" indexed="true" stored="true"/>
<field name="parent_id" type="long" indexed="true" stored="true"/>
<field name="item_type" type="text_general" indexed="true" stored="true"/>
<field name="status" type="text_general" indexed="true" stored="true" />
<field name="updated_at" type="date" indexed="true" stored="true"/>
It is left to you what type(fieldType) you want to add depending on your requirement.
indexed: true if this field should be indexed (searchable or
sortable)
stored: true if this field should be retrievable
Add the below tag:
<uniqueKey>id</uniqueKey>
This is to use to determine and enforce document uniqueness.
I need to store special characters in my Solr index.
I'll try to index a mysql database that contains special characters with Solr, but when I search in the generated index the special characters are stored like this: �.
The configuration files are these:
data-config.xml
<dataConfig>
<dataSource type="JdbcDataSource"
encoding="UTF-8"
URIEncoding="UTF-8"
driver="com.mysql.jdbc.Driver"
url="jdbc:mysql://localhost:3306/db_name"
user="***"
password="***" />
<document name="content">
<entity name="id" query="SELECT * FROM organization"
deltaImportQuery="SELECT * FROM organization" deltaQuery="SELECT * FROM organization">
<field column="organization_id" name="organization_id" />
<field column="name" name="name" />
<field column="cb_url" name="cb_url" />
<field column="cb_image_url" name="cb_image_url" />
<field column="fb_url" name="fb_url" />
<field column="tw_url" name="tw_url" />
<field column="ln_url" name="ln_url" />
<field column="city" name="city" />
<field column="region" name="region" />
<field column="state" name="state" />
<field column="cb_description" name="cb_description" />
<field column="last_change_date" name="last_change_date" />
<field column="al_id" name="al_id" />
<field column="al_url" name="al_url" />
<field column="al_followers" name="al_followers" />
<field column="al_pre_money_valuation" name="al_pre_money_valuation" />
<field column="al_raising_amount" name="al_raising_amount" />
<field column="al_blog_url" name="al_blog_url" />
<field column="fb_name" name="fb_name" />
<field column="fb_id" name="fb_id" />
<field column="fb_about" name="fb_about" />
<field column="fb_companyoverview" name="fb_companyoverview" />
<field column="fb_checkins" name="fb_checkins" />
<field column="fb_founded" name="fb_founded" />
<field column="fb_likes" name="fb_likes" />
<field column="fb_latitude" name="fb_latitude" />
<field column="fb_longitude" name="fb_longitude" />
<field column="fb_street" name="fb_street" />
<field column="fb_talkingabout" name="fb_talkingabout" />
<field column="ln_name" name="ln_name" />
<field column="ln_id" name="ln_id" />
<field column="ln_companytype" name="ln_companytype" />
<field column="ln_industries" name="ln_industries" />
<field column="ln_status" name="ln_status" />
<field column="ln_blogrssurl" name="ln_blogrssurl" />
<field column="ln_employeecountrange" name="ln_employeecountrange" />
<field column="ln_specialties" name="ln_specialties" />
<field column="ln_description" name="ln_description" />
<field column="ln_endyear" name="ln_endyear" />
<field column="ln_numfollowers" name="ln_numfollowers" />
<field column="tw_name" name="tw_name" />
<field column="tw_id" name="tw_id" />
<field column="tw_followers" name="tw_followers" />
<field column="tw_friendscount" name="tw_friendscount" />
<field column="tw_favouritescount" name="tw_favouritescount" />
<field column="tw_statusescount" name="tw_statusescount" />
</entity>
</document>
</dataConfig>
schema.xml
<schema>
<fieldType name="text_split" class="solr.TextField">
<analyzer>
<tokenizer class="solr.WhitespaceTokenizerFactory" />
<filter class="solr.LowerCaseFilterFactory" />
</analyzer>
</fieldType>
<field name="organization_id" type="string" indexed="true"
stored="true" required="true" />
<field name="name" type="text_general" indexed="true" stored="true" />
<field name="cb_description" type="text_split" indexed="true"
stored="true" />
<field name="fb_about" type="text_split" indexed="true" stored="true" />
<field name="fb_companyoverview" type="text_split" indexed="true"
stored="true" />
<field name="ln_description" type="text_split" indexed="true"
stored="true" />
<field name="cb_url" type="text_split" indexed="true" stored="true" />
<field name="cb_image_url" type="text_split" indexed="true"
stored="true" />
<field name="fb_url" type="text_split" indexed="true" stored="true" />
<field name="tw_url" type="text_split" indexed="true" stored="true" />
<field name="ln_url" type="text_split" indexed="true" stored="true" />
<field name="city" type="text_split" indexed="true" stored="true" />
<field name="region" type="text_split" indexed="true" stored="true" />
<field name="state" type="text_split" indexed="true" stored="true" />
<field name="last_change_date" type="text_split" indexed="true"
stored="true" />
<field name="al_id" type="text_split" indexed="true" stored="true" />
<field name="al_url" type="text_split" indexed="true" stored="true" />
<field name="al_followers" type="text_split" indexed="true"
stored="true" />
<field name="al_pre_money_valuation" type="text_split" indexed="true"
stored="true" />
<field name="al_raising_amount" type="text_split" indexed="true"
stored="true" />
<field name="al_blog_url" type="text_split" indexed="true"
stored="true" />
<field name="fb_name" type="text_split" indexed="true" stored="true" />
<field name="fb_id" type="text_split" indexed="true" stored="true" />
<field name="fb_checkins" type="text_split" indexed="true"
stored="true" />
<field name="fb_founded" type="text_split" indexed="true" stored="true" />
<field name="fb_likes" type="text_split" indexed="true" stored="true" />
<field name="fb_latitude" type="text_split" indexed="true"
stored="true" />
<field name="fb_longitude" type="text_split" indexed="true"
stored="true" />
<field name="fb_street" type="text_split" indexed="true" stored="true" />
<field name="fb_talkingabout" type="text_split" indexed="true"
stored="true" />
<field name="ln_name" type="text_split" indexed="true" stored="true" />
<field name="ln_id" type="text_split" indexed="true" stored="true" />
<field name="ln_companytype" type="text_split" indexed="true"
stored="true" />
<field name="ln_industries" type="text_split" indexed="true"
stored="true" />
<field name="ln_status" type="text_split" indexed="true" stored="true" />
<field name="ln_blogrssurl" type="text_split" indexed="true"
stored="true" />
<field name="ln_employeecountrange" type="text_split" indexed="true"
stored="true" />
<field name="ln_specialties" type="text_split" indexed="true"
stored="true" />
<field name="ln_endyear" type="text_split" indexed="true" stored="true" />
<field name="ln_numfollowers" type="text_split" indexed="true"
stored="true" />
<field name="tw_name" type="text_split" indexed="true" stored="true" />
<field name="tw_id" type="text_split" indexed="true" stored="true" />
<field name="tw_followers" type="text_split" indexed="true"
stored="true" />
<field name="tw_friendscount" type="text_split" indexed="true"
stored="true" />
<field name="tw_favouritescount" type="text_split" indexed="true"
stored="true" />
<field name="tw_statusescount" type="text_split" indexed="true"
stored="true" />
</schema>
Solr:4.8.1,I have a field called age which stores a single character like A or C and is stored is the field
<field name="age" type="text_general" indexed="true" stored="true"/>
When I get results back from other searches I can see the field age and its value but when I search for example age:* it returns 0 results. This just happened recently as I have been working with this field for a month and it worked fine but now nothing returns. I altered the schema a few times but nothing regarding this field. The only thing I can think of is that I accidentally put an invalid value into the age field of the mysql database that I import from, but fixed that and re-imported it.
I have searched this problem and found that <defaultSearchField> needs to be set but those results were older and that field is now depreciated.
EDIT:
My data config is:
<dataConfig>
<dataSource type="JdbcDataSource"
driver="com.mysql.jdbc.Driver"
url="jdbc:mysql://localhost/gtw"
user="root"
password=""/>
<document>
<entity name="id" query="select id,price,title,description,main_image,retailer,link,age,gender,type,category,creation_date from solr_listings">
<field column="category" name="category" splitBy=","></field>
</entity>
</document>
</dataConfig>
The only thing that is different from the default example schema are the fields I added below:
<field name="id" type="string" indexed="true" stored="true" required="true" multiValued="false" />
<field name="title" type="text_general" indexed="true" stored="true" />
<field name="description" type="text_general" indexed="true" stored="true"/>
<field name="retailer" type="text_general" indexed="true" stored="true"/>
<field name="category" type="text_general" indexed="true" stored="true" />
<field name="main_image" type="text_general" indexed="true" stored="true"/>
<field name="last_modified" type="date" indexed="true" stored="true"/>
<field name="link" type="string" indexed="true" stored="true" />
<field name="gender" type="text_general" indexed="true" stored="true"/>
<field name="age" type="text_general" indexed="true" stored="true"/>
<field name="type" type="text_general" indexed="true" stored="true" />
<field name="creation_date" type="date" indexed="true" stored="true" />
<field name="price" type="float" indexed="true" stored="true"/>
I am trying to use solr for indexing data from my data base.
After I index data, when I query *.*
I get just the id field in result. not all the fields which I had in my query.
My data-config.xml
<document name="content">
<entity name="documen" query="SELECT indexId ,brand_id, category_id, product_name from Production">
<field column="indexId" name="id" />
<field column="category_id" name="categoryid" />
<field column="brand_id" name="brandid" />
<field column="product_name" name="id" />
</entity>
</document>
My schema.xml looks like this :
<field name="id" type="int" indexed="true" stored="true" required="true"/>
<field name="categoryid" type="int" indexed="true" stored="true"/>
<field name="brandid" type="int" indexed="true" stored="true" />
<field name="productname" type="string" indexed="true" stored="true"/>
When I query using *.* I get
<doc>
<str name="id">1</str>
<long name="_version_">1426653005792411648</long></doc>
<doc>
<str name="id">2</str>
<long name="_version_">1426653005793460224</long></doc>
<doc>
I get only "id" field as result.
Actually, whatever field is in "uniquekey" tag is returned as query result