I have 4 CSV files which I am indexing in the same core.
PRESCRIBER.csv
Fields: PRESCRIBER_ID, PRESCRIBER_TYPE, FIRST_NAME, LAST_NAME, MIDDLE_INITIAL, DEA_NUMBER, DEA_SUFFIX, UPIN_NUMBER, SPIN_NUMBER, ACTIVE_FLAG, RESPONSIBLE_PRESCRIBER_ID, FACILITY_ID, LOCK_COL
PRESCRIBER_ADDRESS.csv
Fields: PRESCRIBER_ID, ADDRESS_ID, LOCATION_TYPE, PRIMARY_FLAG, LOCK_COL, PRESCRIBER_LOCATION_CODE
PRESCRIBER_TELEPHONE.csv
Fields: PRESCRIBER_ID, TELEPHONE_ID, PRIMARY_FLAG, LOCK_COL
TELEPHONE.csv
Fields: TELEPHONE_ID, TELEPHONE_TYPE, AREA_CODE, TELEPHONE_NUMBER, EXTENSION, LOCK_COL
I want to imitate the below SQL query in SOLR:
SELECT * FROM PRESCRIBER INNER JOIN PRESCRIBER_TELEPHONE ON
PRESCRIBER.PRESCRIBER_ID = PRESCRIBER_TELEPHONE.PRESCRIBER_ID
INNER JOIN TELEPHONE ON
PRESCRIBER_TELEPHONE.TELEPHONE_ID = TELEPHONE.TELEPHONE_ID
WHERE LAST_NAME LIKE '%A%' AND FIRST_NAME LIKE '%B%'
AND ISNULL(PRESCRIBER_TELEPHONE.PRIMARY_FLAG,'') = 'Y'
I have written the below SOLR query:
http://160.110.9.39:8983/solr/prscrbrDtls/select?q=*:*&fq=FIRST_NAME:*B*&fq=LAST_NAME:*A*&fq:{!join+from=PRESCRIBER_ID+to=PRESCRIBER_ID}TELEPHONE_ID{!join+from=TELEPHONE_ID+to=TELEPHONE_ID}PRIMARY_FLAG:Y&rows=5000
Issue:
I am only seeing the fields from PRESCRIBER.csv.
{"PRESCRIBER_ID":[XX],
"PRESCRIBER_TYPE":["MD"],
"FIRST_NAME":["ELIZABETH"],
"LAST_NAME":["CHAVEZ"],
"MIDDLE_INITIAL":["O"],
"DEA_NUMBER":["XX"],
"ACTIVE_FLAG":["X"],
"LOCK_COL":[3],
"id":"d35e4c68-9cb2-477a-ad03-7ab25bc5490d",
"_version_":1608393436695625743}
I want to see all the fields together from PRESCRIBER, PRESCRIBER_TELEPHONE and TELEPHONE in the response JSON.
Any pointers on how to do that?
Am I writing the query correct?
Additional Question:
How can I remove _str fields from SOLR response?
You're writing your query correct, but joins in Solr does not allow you to fetch anything else than what's in the primary collection you're querying. It's one of the fundamental limitations of joins in Solr.
This is different from the concept of a join in a relational database because no information is being truly joined. An appropriate SQL analogy would be an "inner query".
If you need to fetch data on multiple sides of a query, you can use a Streaming Expression, with either innerJoin, hashJoin, leftOuterJoin or one of the other join decorators. Streaming Expressions requires a fairly recent version of Solr.
Related
My table looks like as follows; (I populated it from excel)
I want to extract some data from another table so I use sql joins. Since my column contains comma separated values, i try to use join with Or but no success. Is this right way to do joins?
I look for quick solution for this comma separated columns.
eg:
SELECT * FROM test.types as a
inner join test.`matric as ma on (a.category= SUBSTRING_INDEX(ma.`Function Code AA`,',',1)
or a.category= SUBSTRING_INDEX(ma.`Function Code AA`,',',2)
or a.category= SUBSTRING_INDEX(ma.`Function Code AA`,',',3)
or a.category= SUBSTRING_INDEX(ma.`Function Code AA`,',',4)
or a.category= SUBSTRING_INDEX(ma.`Function Code AA`,',',5)
or a.category= SUBSTRING_INDEX(ma.`Function Code AA`,',',6))
and a.type = ma.`function Code NN` and ma.`Priority` = "T1"
You have a horrible data model. You probably cannot speed up the query very much. But you can at least simplify the code.
SELECT *
FROM test.types t JOIN
test.matric ma
ON FIND_IN_SET(t.category, REPLACE(ma.`Function Code AA`, ', ', ',')) > 0 OR
t.type = ma.`function Code NN`
WHERE ma.`Priority` = 'T1';
However, you should fix your data model!!!. Here are some issues:
Databases have very poor string processing capabilities.
Values should be stored using the correct type.
Foreign keys should be declared properly.
Such a structure prevents the database from using indexes, partitions, and the best optimization methods.
SQL has a great way to store lists. It is not called a string. It is called a table.
I have a list of categories that are saved in a string. One is on an article table the other on an ad table. I need the script to return all rows that have a combination of any of these categories between the two tables.
Category string on both tables:
Civic,Community,Sports,Business
And my MySQL script
SELECT `Ad_ID`, `Ad_URL`, `Ad_Image`, `Ad_Type`, `Ad_Cities`, `Ad_Categories`
FROM `Ad_TABLE`
INNER JOIN `Article_TABLE` ON `Article_TABLE`.`Article_Cat` = `Ad_TABLE`.`Ad_Cat`
WHERE Article_TABLE.Article_Cat LIKE '%Civic%'
OR Article_TABLE.Article_Cat LIKE '%Community%'
OR Article_TABLE.Article_Cat LIKE '%Sports%'
OR Article_TABLE.Article_Cat LIKE '%Business%'
AND Ad_TABLE.Ad_Cat LIKE '%Civic%'
OR Ad_TABLE.Ad_Cat LIKE '%Community%'
OR Ad_TABLE.Ad_Cat LIKE '%Sports%'
OR Ad_TABLE.Ad_Cat LIKE '%Business%'
The script only returns records that are only in one of these categories, but there are records that are in multiple categories and I need it to return those as well.
How can I get it to where it finds all matching categories between the two tables?
I suspect you need to add parenthesis to the WHERE clause:
SELECT `Ad_ID`, `Ad_URL`, `Ad_Image`, `Ad_Type`, `Ad_Cities`, `Ad_Categories`
FROM `Ad_DB`
INNER JOIN `Article_DB` ON `Article_DB`.`Article_Cat` = `Ad_DB`.`Ad_Cat`
WHERE (Article_DB.Article_Cat LIKE '%Civic%'
OR Article_DB.Article_Cat LIKE '%Community%'
OR Article_DB.Article_Cat LIKE '%Sports%'
OR Article_DB.Article_Cat LIKE '%Business%')
AND (Ad.DB_Cat LIKE '%Civic%'
OR Ad.DB_Cat LIKE '%Community%'
OR Ad.DB_Cat LIKE '%Sports%'
OR Ad.DB_Cat LIKE '%Business%')
I'm not sure exactly how your tables are structured, but this query will return rows where Article_DB.Article_Cat AND Ad.DB_Cat contain one of your categories. You likely want to rethink the way you store data in these tables so that you're not duplicating data.
Here's a guess. This is just a guess, based on a possible interpretation of the nebulous specification.
To "match" rows that have one or more of these four categories in common
Civic,Community,Sports,Business
We could use a query with join predicates like this:
SELECT ...
FROM `Ad_DB` d
JOIN `Article_DB` r
ON ( FIND_IN_SET('Civic' ,d.ad_categories)
AND FIND_IN_SET('Civic' ,r.article_cat )
)
OR ( FIND_IN_SET('Community' ,d.ad_categories)
AND FIND_IN_SET('Community' ,r.article_cat )
)
OR ( FIND_IN_SET('Sports' ,d.ad_categories)
AND FIND_IN_SET('Sports' ,r.article_cat )
)
OR ( FIND_IN_SET('Business' ,d.ad_categories)
AND FIND_IN_SET('Business' ,r.article_cat )
)
ORDER BY ...
NOTE: I understand that we get what we get, and sometimes we get handed a database that stores comma separated lists.
Storing comma separated lists in a column is a SQL Antipattern. For anyone that has an interest as to why it's an antipattern, I recommend Chapter 2 of Bill Karwin's excellent book
https://www.amazon.com/SQL-Antipatterns-Programming-Pragmatic-Programmers/dp/1934356557
I get a MySQL Error saying, I cannot use more than 61 tables in a join. I need to avoid this error. How do I do it? Please Help.
select
view_pdg_institutes.user_id as User_ID,
view_pdg_institutes.institute_id as Teacher_ID,
view_pdg_institutes.institute_name as Institute_Name,
view_pdg_institutes.user_email as Email,
view_pdg_institutes.contact_person_name as Contact_Person,
view_pdg_institutes.alternative_contact_no as Alternative_Mobile_No,
view_pdg_institutes.primary_contact_no as Mobile_No,
view_pdg_institutes.correspondance_address as Address,
view_pdg_institutes.other_communication_mode as Preferred_Contact_Mode,
view_pdg_institutes.size_of_faculty as Size_of_Faculty,
view_pdg_institutes.operation_hours_from as Operation_Hours_From,
view_pdg_institutes.operation_hours_to as Operation_Hours_To,
view_pdg_institutes.teaching_xp as Teaching_Experience,
view_pdg_institutes.installment_allowed as Installment_Allowed,
view_pdg_institutes.about_fees_structure as About_Fees_Structure,
view_pdg_institutes.no_of_demo_class as No_of_Demo_Classes,
view_pdg_institutes.demo_allowed as Demo_Allowed,
view_pdg_institutes.price_per_demo_class as Price_Per_Demo_Class,
view_pdg_tuition_batch.tuition_batch_id as Batch_ID,
view_pdg_batch_subject.subject_name as Subject_Name,
view_pdg_batch_subject.subject_type as Subject_Type,
view_pdg_batch_subject.academic_board as Academic_Board,
view_pdg_batch_fees.fees_type as Fees_Type,
view_pdg_batch_fees.fees_amount as Fees_Amount,
view_pdg_tuition_batch.course_days as Course_Days,
view_pdg_tuition_batch.days_per_week as Days_Per_Week,
view_pdg_tuition_batch.class_duration as Class_Duration,
view_pdg_tuition_batch.class_type as Class_Type,
view_pdg_tuition_batch.course_length as Course_Length,
view_pdg_tuition_batch.course_length_type as Course_Length_Type,
view_pdg_tuition_batch.no_of_locations as No_of_Locations,
view_pdg_tuition_batch.class_capacity_id as Class_Capacity_ID,
view_pdg_tutor_location.locality as Locality,
view_pdg_tutor_location.address as Address,
view_pdg_batch_class_timing.class_timing as Class_Timing
from view_pdg_tuition_batch
left join view_pdg_institutes on (view_pdg_tuition_batch.tutor_institute_user_id = view_pdg_institutes.user_id)
left join view_pdg_batch_subject on (view_pdg_batch_subject.tuition_batch_id = view_pdg_tuition_batch.tuition_batch_id)
left join view_pdg_batch_fees on (view_pdg_batch_fees.tuition_batch_id = view_pdg_tuition_batch.tuition_batch_id)
left join view_pdg_batch_class_timing on (view_pdg_batch_class_timing.tuition_batch_id = view_pdg_tuition_batch.tuition_batch_id)
left join view_pdg_tutor_location on (view_pdg_tutor_location.tuition_batch_id = view_pdg_tuition_batch.tuition_batch_id)
group by view_pdg_tuition_batch.tuition_batch_id;
I need a solution that would not require changing the current approach of writing the query.
I don't think it's possible to do what you're asking without some elaborate changes in the way you store and query data. You can
denormalize your DB to store JSON data;
create materialized views, emulating them via triggers, because they're absent in MySQL;
use temporary tables;
join partial selects by hand at the call site;
compile MySQL with another join limit;
use proper SQL engine like Postgres, that doesn't suffer from such stupid things.
Insert the contents of each view into its own temporary table. Then do the same query with the temporary table names substituted for the original view names.
It is showing Sum(with table name) is not valid. Kindly help:
Modws.DisplayDataGrid(dgvClosingBalance,
"Select
Invoice.Customer, Invoice.Sum(Total),
RptTempTable.Sum(INVOICETOTAL), RptTempTable.Sum(CNTOTAL),
RptTempTable.Sum(DEBITTOTAL), RptTempTable.Sum(RECEIPTTOTAL)
From Invoice
inner join RptTempTable on Invoice.Customer = RptTempTable.Customer")
RptTempTable.Sum(INVOICETOTAL) should be Sum(RptTempTable.INVOICETOTAL)
The same goes for the other calls to sum()
The table prefix belongs to the column name not the function call.
MySQL will accept this invalid SQL and will return "inderminate" (aka "random") values instead.
To understand the implications of MySQL's "loose" (aka "sloppy") group by implementation you might want to read these articles:
http://www.mysqlperformanceblog.com/2006/09/06/wrong-group-by-makes-your-queries-fragile/
http://rpbouman.blogspot.de/2007/05/debunking-group-by-myths.html
I have a table 'users_category'
'users_category' : id, prefix, name
and table 'users'
'users' : categories, etc...
where users.categories is an array of users_category.id ids
User can be part of any category, if I stored the array of categories they are part of as a serialized array, how can I run a query to check if they are part of 'category x'
Ex:
SELECT users.*, users_category.* FROM 'users', 'users_category' WHERE users.categories='category x' AND users_category.id = 'category x'
I can't run a 'LIKE' command because the users.categories is serialized. Is their any way to search within the serialized data. Also I know that the above query may have errors
Even though normalizing the table structure is a better way to move forward but if adopting that route is not optimal at this point then you may try this query:
SELECT u.*, uc.*
FROM users u, users_category uc
WHERE uc.name='X' AND FIND_IN_SET(uc.id, u.categories)
It uses mysql function FIND_IN_SET().
Working demo available at: http://sqlfiddle.com/#!2/7f392/3