MySQl update column with values in a specific format - mysql

I am working on an old application and mysql db that is sadly all in a bit of a mess.
I am interested in a column needed for export that contains names of editors for particular documents. The values in this column should be formatted in a particular way, and should be consistent which they are currently not.
A document can have many editors. I have a table containing all distinct editors which I need to seomhow match against the current (incorrectly formatted) editors value. I know this is not the best way to go about doing this (it is how the application currently works) but I need the editors column to be formatted as follows:
"Surname, Initial. and Surname, Initial. and Surname, Initial"
E.g.:
"Bloggs, J. and Doe, J."
"Smith, M.J. and Bloggs, J. and Jones, P"
"Williams, S. and Smith, M.J. Doe, J."
However the column is currently inconsistent with no real formatting in place:
"J Bloggs, J Doe"
"MJ Smith, J Bloggs and P Jones"
"Williams, S, M.J Smith. Doe, J."
etc...
Here are my current tables:
Documents:
| title | editors |
------------------------------------------------
| doc title 1 | J Bloggs, J Doe |
| doc title 2 | M Smith, J Bloggs and P Jones |
| doc title 3 | Williams, S, M Smith. Doe, J. |
Eitors:
| initial | name |
------------------------
| J. | Bloggs |
| J. | Doe |
| M.J. | Smith |
| P. | Jones |
| S. | Williams |
For testing purposes I have added a column (formatted_editors) to the Documents table:
| title | editors | formatted_editors |
---------------------------------------------------------------------------------
Which I need to look something like this:
| title | editors | formatted_editors |
---------------------------------------------------------------------------------
| doc title 1 | J Bloggs, J Doe | Bloggs, J. and Doe, J.
| doc title 2 | M Smith, J Bloggs and P Jones | Smith, M.J. and Bloggs, J. and Jones, P
| doc title 3 | Williams, S, M Smith. Doe, J. | Williams, S. and Smith, M.J. Doe, J.
I have tried the following (and various others) but cant seem to have any luck - I think it may have something to do with my groupings ?
SELECT d. title, d.editors, group_concat(e.name, ', ', e.initial SEPARATOR ' and ') as formatted_editors
FROM documents d
INNER JOIN editors e
ON d.editors LIKE concat('%', e.name, '%')
WHERE d.editors LIKE concat('%', e.name, '%')
and et.editors LIKE concat('%', e.initial, '%')
GROUP BY et.editors;
Any help / pointers would be appreciated,


No "real" reliable way to solve this then... I used some basic SQL to fix up / match what I can and did the rest manually.
Thanks,
Jon.

Related

SQL query that searches comma-delimited field

I have a student table which looks something like this:
id | name | school_descriptors
-------------------------------------------------------
1 | Rob | Comp Sci,Undergraduate,2020
2 | Tim | Business,MBA,2022
3 | Matt | Business,MBA,2022
4 | Jack | Law,Masters,2024
5 | Steph | Comp Sci,Masters,2022
The school_descriptors field is just one column, and stores information about the Course, Qualification and Graduation year as a comma-delimited string. (it's terribly designed and I wish it could be split up into its own fields, but it can't right now (I am not the database owner))
I want to provide an interface where teachers can quickly find students that match certain Course, Qualifications and Graduation years, and thus would like to create relevant queries.
Question 1: For example, I would like a teacher to be able to select from the UI: "Business", "MBA" and get returned students with ID 2 and 3. Specifically, an example question I have is: Find students who are in the Business Course and doing the MBA qualification:
SELECT * FROM student_table WHERE school_descriptors LIKE '%Business%' AND school_descriptors LIKE '%MBA%'
The query I have in mind is a basic LIKE query, but I can't help but think there is a more efficient query that can take advantage of the fact that the school_descriptor string is 1) always in a specific order (e.g. course, qualification, graduation), and 2) comma-delimited, and thus could be perhaps split. The table currently sits at ~5000 rows so relatively small but is expected to grow.
Related question 2: Find students who are in the Comp Sci Course and graduating after 2019:
Would it be possible to split the school_descriptors field and add a >2019 operand?
Many thanks!

In MySql you can use the function SUBSTRING_INDEX() to split the column school_descriptors.
This will work only if the positions of Course, Qualification and Graduation year are fixed.
select *,
substring_index(school_descriptors, ',', 1) Course,
substring_index(substring_index(school_descriptors, ',', 2), ',', -1) Qualification,
substring_index(school_descriptors, ',', -1) Graduation
from student_table
See the demo.
Results:
> id | name | school_descriptors | Course | Qualification | Graduation
> -: | :---- | :-------------------------- | :------- | :------------ | :---------
> 1 | Rob | Comp Sci,Undergraduate,2020 | Comp Sci | Undergraduate | 2020
> 2 | Tim | Business,MBA,2022 | Business | MBA | 2022
> 3 | Matt | Business,MBA,2022 | Business | MBA | 2022
> 4 | Jack | Law,Masters,2024 | Law | Masters | 2024
> 5 | Steph | Comp Sci,Masters,2022 | Comp Sci | Masters | 2022

select id, name,
substring_index(school_descriptors,',',1) as course,
substring_index(substring(school_descriptors,length(substring_index(school_descriptors,',',1))+2,200),',',1) as Qualifications,
substring_index(school_descriptors,',',-1) as year
from student;
output:
+------+-------+----------+----------------+------+
| id | name | course | Qualifications | year |
+------+-------+----------+----------------+------+
| 1 | Rob | Comp Sci | Undergraduate | 2020 |
| 2 | Tim | Business | MBA | 2022 |
| 3 | Matt | Business | MBA | 2022 |
| 4 | Jack | Law | Masters | 2024 |
| 5 | Steph | Comp Sci | Masters | 2022 |
+------+-------+----------+----------------+------+
A link to the docs, in case you want to know about SUBSTRING_INDEX()

Answer 1:
SELECT * FROM student_table WHERE school_descriptors REGEXP ['Business','MBA']
By using this query you can get all the records that are having Business OR MBA.
If you want to select only Business, MBA you can try like this
SELECT * FROM student_table WHERE school_descriptors LIKE '%Business,MBA%'
Answer 2:
SELECT *
FROM student
WHERE
SUBSTRING_INDEX(SUBSTRING_INDEX(school_descriptors , ',', 1), ',', -1)='Comp Sci'
AND
SUBSTRING_INDEX(SUBSTRING_INDEX(school_descriptors , ',', 3), ',', -1)> 2019;

How to search for multiple terms using AND in a SQL query?

I have a simple web app listing different data types in searchable tables.
I'm using MySQL for the database.
For example, there's a list of Employees with FirstName, LastName, and so on.
Example Employees table:
ID | FIRSTNAME | LASTNAME | EMAILADDRESS
---------------------------------------------
001 | John | Smith | js#gmail.com
002 | John | Jones | jj#gmail.com
003 | Michael | Smith | ms#gmail.com
004 | Betty | Taylor-Smith | bts#gmail.com
There's a simple text field for searches.
Search for John Smith results are anything has a John AND Smith in it:
001 | John | Smith
Search for John, results are anything that has John in it:
001 | John | Smith
002 | John | Jones
Search for Smith, results are anything that has Smith in it:
001 | John | Smith
003 | Michael | Smith
004 | Betty | Taylor-Smith
So ... my question is ... how should I write the SQL query for this?
select * from employees where ...
If I search for John Smith ...
((firstname like "%John%") OR (firstname like "%Smith%"))
AND
((lastname like "%John%") OR (lastname like "%Smith%"));
AND
((emailaddress like "%John%") OR (emailaddress like "%Smith%"));
Returns zero results because the emailaddress doesn't match.
If I change ANDs to ORs ...
((firstname like "%John%") OR (firstname like "%Smith%"))
OR
((lastname like "%John%") OR (lastname like "%Smith%"));
OR
((emailaddress like "%John%") OR (emailaddress like "%Smith%"));
Then I get all four results because there's a John or a Smith in every row ... which is not the result I want, I only want the first row 001.
001 | John | Smith | js#gmail.com
002 | John | Jones | jj#gmail.com
003 | Michael | Smith | ms#gmail.com
003 | Betty | Taylor-Smith | bts#gmail.com
How should I approach this SQL query?

Seems like you want:
((firstname LIKE "%John%") OR (lastname LIKE "%John%") OR (emailaddress LIKE "%John%"))
AND
((firstname LIKE "%Smith%") OR (lastname LIKE "%Smith%") OR (emailaddress LIKE "%Smith%"))

Could you potentially use CONCAT_WS to meet your requirements?
SELECT *
FROM employees
WHERE CONCAT_WS(" ", firstname, lastname) LIKE "%John Smith%"
OR firstname LIKE "%John Smith%" # in case user provides just a first name
OR lastname LIKE "%John Smith%" # in case user provides just a last name
OR emailaddress LIKE "%John Smith%" # support searching by e-mail at the same time

SQL: Outputting From Two Tables

Question Posed: List the people and the schools for people who attended schools in Missouri.
I have a table called Post and a table called People. Each person in people has a unitid that corresponds to a unitid in post, basically saying where they go to college at. Each college in Post has a unitid, and a state abbreviation field, stabbr.
I am able to use the following to get the formmated names, and UnitID from people but can't figure out how to add on the Universities Name as well. Universities names are instnm.
select concat_ws(" ",surname,givenname,middleInitial) as 'whole_name',
unitid
from people
where
unitid in (select unitid from post where stabbr like 'MO');
How Would I go about adding in the university names?
Output:
| Kittinger Rita G | 451316 |
| Cagle Marie P | 456302 |
| Martinez Shana C | 460996 |
| Price John O | 480338 |
| Kurtz Melvin L | 481580 |
| Olmstead Tabitha J | 481988 |
| Wynn Stephanie D | 481997 |
| Garrett Richard L | 482398 |
| Bell Diane R | 482398 |
+---------------------+--------+

How to condense a column like this?

I've tried finding something like this, but to no avail...
This is about a system of tables for a customer management system. In particular, I need to create a note history for each customer.
So, I have a table 'customers' with the columns customers.customer_ID, customers.lastname, customers.firstname, customers.postal_code, customers.city and customers.street;
and another table 'notes' with the columns notes.note_ID, notes.customer_ID, notes.subject, notes.description and notes.entered_on
Now I need to create a third table search which condenses much of the information above. It has the tables search.contact_ID, search.name, search.address and search.history. This is supposed to look like this:
contacts:
contact_ID | lastname | firstname | ...
------------+-----------+-----------+-----
1 | Doe | John | ...
2 | Dane | Jane | ...
note:
note_ID | contact_ID | subject | description | entered_on
--------+---------------+-----------------------+-----------------------+----------------
1 | 1 | call received | John Doe called us to | 2014-05-03
| | | ask for an offer |
2 | 1 | offer made | We called John Doe to | 2014-06-03
| | | submit our offer |
3 | 2 | advertisement call | We called Jane Dane to| 2014-06-03
| | | inform her of our |
| | | latest offer |
4 | 1 | offer accepted | John Doe called to | 2014-08-03
| | | accept our offer |
search:
contact_ID | name | address | history
------------+---------------+---------------------------------+-------------------
1 | Doe, John | 55 Main Street, 12345 Oldtown | 'On 2014-08-03 offer accepted: John Doe accepted our offer.
| | | On 2014-06-03 offer made: We called John Doe to submit our offer.
| | | On 2014-05-03 call received: John Doe called us to ask for an offer.'
2 | Dane, Jane | 111 Wall Street, 67890 Newtown | 'On 2014-06-03 advertisement call: We called Jane Dane to submit our offer.'
While I can deal with much of the rest, I have no idea how to generate the history information. My idea was as follows
WHILE
customers.customer_ID = note.customer_ID
AND
note.entered_on = GREATEST(note.entered_on)
DO
SET customers.note_history = CONCAT_WS(' | ', CONCAT_WS(': ',note.subject,note.description), customers.note_history);
But that one isn't necessarily chronological. Also how do I transform that into a statement compatible with the SELECT INTO used for the creation of the rest of the table?

Sounds like a case for a Group-By, along with GROUP_CONCAT
CREATE TABLE search (PRIMARY KEY(contact_ID))
SELECT contact_ID, CONCAT(lastname,', ',firstname) AS name, address,
GROUP_CONCAT(CONCAT('On ',entered_on,' ',subject,': ',description)
ORDER BY note_ID SEPARATOR "\n") AS history
FROM contacts LEFT JOIN note USING (contact_ID)
GROUP BY contact_ID
If dont want to use CREATE TABLE .. SELECT ... , can first just create (or truncate!) the table, and then use INSERT INTO ... SELECT ... instead.

Group by X or Y?

I'm trying to figure out how to GROUP BY on multiple columns. I want to group items when the SSN or the address matches. For example, here are three records:
account_number | name | ssn | address
---------------+--------------+-------------+----------------------
23952352340 | SMITH INC | 123-45-6789 | P.O. BOX 123
3459450340 | JOHN SMITH | 123-45-6789 | 123 EVERGREEN TERRACE
45949459494 | JANE SMITH | 395-23-1924 | 123 EVERGREEN TERRACE
And here's what I'd like to end up with:
names
----------------------
SMITH INC, JOHN SMITH, JANE SMITH
Any suggestions?

You can't do this easily in MySQL.
The problem is that the relation "is similar to" as you define it is not transitive. In your example, Smith Inc is similar to John Smith (per SSN) and John Smith is similar to Jane Smith (per name), but Smith Inc is not similar to Jane Smith. So there is no single value that all records could be compared with and GROUP BY won't help here.
In other systems which support recursion you could build a transitive closure of this relation which would allow grouping, but this is not an easy task in MySQL.

Like this:
SELECT
name,
ssn,
COUNT(*)
FROM TheTable
GROUP BY
name,
ssn