I have a table 'products' which has a column partnumber.
I want to ignore special characters from record while searching.
Suppose i have following 5 records in partnumber:
XP-12345
MV-334-3454
XP1-5555
VX-AP-XP-1000
VT1232223
Now, If i try to search "XP1", then Output should be come like following records
XP-12345
XP1-5555
VX-AP-XP-1000
How to write mysql query for this ?
You can achieve this functionality using concat() function. As I can review your and Jorden answer comment that you want to search string XP1 with ignore special charecter like -,_,# .
So you can use this query
SELECT partnumber FROM products
WHERE partnumber LIKE concat('%XP','_','1%')
OR partnumber LIKE '%XP1%';;
Note: Require output you can check on SQLFIDDLE and You can adjust query based on your additional requirement.
Define a MySQL function which strips the symbols from a provided string.
DELIMITER //
CREATE FUNCTION STRIP_SYMBOLS(input VARCHAR(255))
RETURNS VARCHAR(255) DETERMINISTIC NO SQL
BEGIN
DECLARE output VARCHAR(255) DEFAULT '';
DECLARE c CHAR(1);
DECLARE i INT DEFAULT 1;
WHILE i < LENGTH(input) DO
SET c = SUBSTRING(input, i, 1);
IF c REGEXP '[a-zA-Z0-9]' THEN
SET output = CONCAT(output, c);
END IF;
SET i = i + 1;
END WHILE;
RETURN output;
END//
DELIMITER ;
Then select the records from your table where the partnumber with the symbols stripped contains XP1:
SELECT * FROM products WHERE STRIP_SYMBOLS(partnumber) LIKE '%XP1%';
-- Returns: XP-12345, XP1-5555, VX-AP-XP-1000
This might be painfully slow if your table is large. In this case, look into generated columns (if you have MySQL 5.7.6 or higher) or creating a trigger (if an earlier version) to keep a column in your table updated with the partnumber with symbols stripped.
You need to use REGEXP to allow for containing searches.
EX:
SELECT partnumber
FROM partnumber_tbl
WHERE name REGEXP '[XP1]\-';
This will let it search the database to find anything containing X,P,1.
Here is a live example.
And regexp info for you to look more into. Official docs, I hate reading them, especially oracles.
.......
You could do something like:
SELECT partnumber
FROM products
WHERE REPLACE(partnumber, "_", "")) LIKE '%XP1%');
Related
I currently have an MySQL (Aurora MySQL based on MySQL 5.7.x) PROCEDURE that takes a stringified list of ids as TEXT and uses a CONCAT function to create a prepared statement for execution.
While the procedure works most of the time, I am getting truncation errors on a few calls to it (likely that have a lot of IDs). The error message is essentially:
com.mysql.jdbc.MysqlDataTruncation Data truncation: Data too long for column 'stuffIds' at row 1
My initial thought was that we had selected too small of a type for the PROCEDURE param stuffIds, but when I checked, I discovered it's type is TEXT. It seems REALLY unlikely that our service is maxing out that type, which to my knowledge is limited at 2^31 chars (or 2GB of data). We just don't have enough IDs in any list to hit that limit.
My next thought was that maybe it's related to the CONCAT function that we're using to create the prepared statement? But while I've seen lots of issues & answers around GROUP_CONCAT and its max length (and the related configs), I have not been able to locate any info on the max length of string supported by MySQL's CONCAT function.
The only thing left I could think of was that maybe it's related specifically to AWS Aurora's implementation of MySQL? I think Aurora is supposed to be pretty close to vanilla MySQL, but maybe this is an edge case?
The PROCEDURE itself (the names have been changed to protect the innocent) raising this error looks like:
DELIMITER //
CREATE PROCEDURE GetThingsByStuffIds(stuffIds TEXT)
BEGIN
SET #stmt = CONCAT('SELECT ThingId FROM ThingTable WHERE StuffId IN (', stuffIds, ')');
PREPARE insert_stm FROM #stmt;
EXECUTE insert_stm;
DEALLOCATE PREPARE insert_stm;
END //
DELIMITER ;
Any insights are appreciated. Thanks!
UPDATE/NOTE/WARNING!
As #Schwern pointed out, I had my MySQL TEXT type info wrong!
which to my knowledge is limited at 2^31 chars (or 2GB of data)
That part of my question was very incorrect! Unlike PostgreSQL, as #Schwern states below, MySQL has three sized variants of TEXT - TEXT (65,535 chars), MEDIUMTEXT (16,777,215 chars), LONGTEXT (4,294,967,295 chars). I misunderstood the length limits of TEXT, hence why my call was exceeding the limits. The full info on these types in MySQL is here. Adding this note so my misinformation in the question doesn't mislead anyone!
text cannot be larger than 2^16 bytes or 65,535. Depending on the character set, characters can take up multiple bytes, but if it's simple UTF-8 it's probably one byte per character.
You could use mediumtext (2^24 bytes) or longtext (2^32 bytes), but really your table needs to be redesigned. Storing lists as comma separated strings is extremely inefficient, complex, and error prone. Instead, use a many-to-many join table.
create thing_stuff (
thing_id int not null references thing(id),
stuff_id int not null references stuff(id)
);
-- Thing 1 has Stuff 2 and 5.
insert into thing_stuff (thing_id, stuff_id) values
(1, 2), (1, 5);
-- Get all the Stuff associated with Thing 1.
select stuff.*
from stuff s
join thing_stuff ts on ts.stuff_id = s.id
where thing_id = 1;
-- Thing 1 no longer has Stuff 5.
delete from thing_stuff
where thing_id = 1 and stuff_id = 5;
You will encounter a Problem as the IN Clause has a limit to the number of values it can hold and it is directly linked the max_allowed_packet size
A better Solution is to split the id string and add it to a temporary table and join both, with big numbers of elements this is also faster
The split function i lend form https://stackoverflow.com/a/11835246/5193536
CREATE TABLE ThingTable(StuffId int,ThingId int)
INSERT INTO ThingTable VALUES (1,1),(2,2),(3,3),(5,5)
CREATE FUNCTION SPLIT_STR(
x VARCHAR(255),
delim VARCHAR(12),
pos INT
)
RETURNS VARCHAR(255)
RETURN REPLACE(SUBSTRING(SUBSTRING_INDEX(x, delim, pos),
LENGTH(SUBSTRING_INDEX(x, delim, pos -1)) + 1),
delim, '');
CREATE PROCEDURE GetThingsByStuffIds(stuffIds TEXT)
BEGIN
DECLARE a INT Default 0 ;
DECLARE str TEXT;
DROP TEMPORARY TABLE IF EXISTS my_temp_table;
CREATE TEMPORARY TABLE my_temp_table(aint int);
simple_loop: LOOP
SET a=a+1;
SET str=SPLIT_STR(stuffIds,",",a);
IF str='' THEN
LEAVE simple_loop;
END IF;
#Do Inserts into temp table here with str going into the row
insert into my_temp_table values (str);
END LOOP simple_loop;
SELECT ThingId FROM ThingTable t1 INNER JOIN my_temp_table t2 ON t1.StuffId = t2.aint;
END
CALL GetThingsByStuffIds('1,2,3,4,5,6')
| ThingId |
| ------: |
| 1 |
| 2 |
| 3 |
| 5 |
✓
db<>fiddle here
I am 90% sure that this is not possible but I need expert advice.
Is this possible to filter out columns on the basis of datatypes using any method in select list in oracle or in any other DBMS.
eg.
select datatype('DATE') from my_table;
To see all the columns in result having datatype DATE. I know above query is not possible according to me(specially the datatype function :) ) but acknowledge if it is.
In Oracle, You may not directly run a query containing specific datatypes. However, you can create a procedure to construct a query with table name and datatype as arguments to contain specific columns and return it as a REF CURSOR.
CREATE OR REPLACE PROCEDURE proc_getsql(
p_table_name IN VARCHAR2,
p_datatype IN VARCHAR2 ,
p_query OUT SYS_REFCURSOR )
IS
v_query VARCHAR2(1000) ;
BEGIN
OPEN p_query FOR
SELECT 'SELECT '
|| LISTAGG(COLUMN_NAME,',') WITHIN GROUP (
ORDER BY column_name )
||' FROM '||p_table_name query_to_run
FROM USER_TAB_COLUMNS
WHERE TABLE_NAME = p_table_name
AND DATA_TYPE LIKE '%'
||p_datatype
||'%';
END;
You may then call or fetch from this CURSOR depending on your requirement.
VAR x REFCURSOR;
EXEC proc_getsql('EMPLOYEES','DATE', :x);
PRINT :x;
Output
QUERY_TO_RUN
------------
SELECT HIRE_DATE,DATE_OF_BIRTH FROM EMPLOYEES
Let's say a have a stored procedure SetCustomerName which has an input parameter Name, and I have a table customers with column Name.
So inside my stored procedure I want to set customer's name. If I write
UPDATE customers SET Name = Name;
this is incorrect and I see 2 other ways:
UPDATE customers SET Name = `Name`;
UPDATE customers SET customers.Name = Name;
First one works, but I didn't find in documentation that I can wrap parameters inside ` characters. Or did I miss it in the documentation (link is appreciated in this case).
What other ways are there and what is the standard way for such a case? Renaming input parameter is not good for me (because I have automatic object-relational mapping if you know what I mean).
UPDATE:
So, there is a link about backticks (http://dev.mysql.com/doc/refman/5.0/en/identifiers.html) but it's not explained deep enough how to use them (how to use them with parameters and column names).
And there is a very strange thing (at least for me): You can use backticks either way:
UPDATE customers SET Name = `Name`;
//or
UPDATE customers SET `Name` = Name;
//or even
UPDATE customers SET `Name` = `Name`;
and they all work absolutely the same way.
Don't you think this is strange? Is this strange behavior explained somewhere?
Simplest way to distinguished between your parameter and column (if both name is same) is to add table name in your column name.
UPDATE customers SET customers.Name = Name;
Even you can also add database prefix like
UPDATE yourdb.customers SET yourdb.customers.Name = Name;
By adding database name you can perform action on more than 1 database from single store procedure.
I think that your first example is actually backwards. If you're trying to set the "Name" column to the "Name" input parameter, I believe it should be:
UPDATE customers SET `Name` = Name;
And for the second example, you can set table aliases the same way that you do in all other statements:
UPDATE customers AS c SET c.Name = Name;
Not necessarily correct, but a fair way to better argument/parameter management, as well readability with easier understanding, especially while working with the SQL;
DROP PROCEDURE IF EXISTS spTerminalDataDailyStatistics; DELIMITER $$
CREATE PROCEDURE spTerminalDataDailyStatistics(
IN TimeFrom DATETIME,
IN DayCount INT(10),
IN CustomerID BIGINT(20)
) COMMENT 'Daily Terminal data statistics in a date range' BEGIN
# Validate argument
SET #TimeFrom := IF(TimeFrom IS NULL, DATE_FORMAT(NOW(), '%Y-%m-01 00:00:00'), TimeFrom);
SET #DayCount := IF(DayCount IS NULL, 5, DayCount);
SET #CustomerID := CustomerID;
# Determine parameter
SET #TimeTo = DATE_ADD(DATE_ADD(#TimeFrom, INTERVAL #DayCount DAY), INTERVAL -1 SECOND);
# Do the job
SELECT DATE_FORMAT(TD.TerminalDataTime, '%Y-%m-%d') AS DataPeriod,
COUNT(0) AS DataCount,
MIN(TD.TerminalDataTime) AS Earliest,
MAX(TD.TerminalDataTime) AS Latest
FROM pnl_terminaldata AS TD
WHERE TD.TerminalDataTime BETWEEN #TimeFrom AND #TimeTo
AND (#CustomerID IS NULL OR TD.CustomerID = #CustomerID)
GROUP BY DataPeriod
ORDER BY DataPeriod ASC;
END $$
DELIMITER ;
CALL spTerminalDataDailyStatistics('2021-12-01', 2, 1801);
Using backticks in MySQL query syntax is documented here:
http://dev.mysql.com/doc/refman/5.0/en/identifiers.html
So yes, your first example (using backticks) is correct.
Here is the link you are asking for:
http://dev.mysql.com/doc/refman/5.0/en/identifiers.html
The backticks are called "identifier quote" in MySql
It must be simple, but I'm making my first steps into Postgres functions and I can't find anything that works...
I'd like to create a function that will modify a table and / or column and I can't find the right way of specifying my tables and columns as arguments in my function.
Something like:
CREATE OR REPLACE FUNCTION foo(t table)
RETURNS void AS $$
BEGIN
alter table t add column c1 varchar(20);
alter table t add column c2 varchar(20);
alter table t add column c3 varchar(20);
alter table t add column c4 varchar(20);
END;
$$ LANGUAGE PLPGSQL;
select foo(some_table)
In another case, I'd like to have a function that alters a certain column from a certain table:
CREATE OR REPLACE FUNCTION foo(t table, c column)
RETURNS void AS $$
BEGIN
UPDATE t SET c = "This is a test";
END;
$$ LANGUAGE PLPGSQL;
Is it possible to do that?
You must defend against SQL injection whenever you turn user input into code. That includes table and column names coming from system catalogs or from direct user input alike. This way you also prevent trivial exceptions with non-standard identifiers. There are basically three built-in methods:
1. format()
1st query, sanitized:
CREATE OR REPLACE FUNCTION foo(_t text)
RETURNS void
LANGUAGE plpgsql AS
$func$
BEGIN
EXECUTE format('
ALTER TABLE %I ADD COLUMN c1 varchar(20)
, ADD COLUMN c2 varchar(20)', _t);
END
$func$;
format() requires Postgres 9.1 or later. Use it with the %I format specifier.
The table name alone may be ambiguous. You may have to provide the schema name to avoid changing the wrong table by accident. Related:
INSERT with dynamic table name in trigger function
How does the search_path influence identifier resolution and the "current schema"
Aside: adding multiple columns with a single ALTER TABLE command is cheaper.
2. regclass
You can also use a cast to a registered class (regclass) for the special case of existing table names. Optionally schema-qualified. This fails immediately and gracefully for table names that are not be valid and visible to the calling user. The 1st query sanitized with a cast to regclass:
CREATE OR REPLACE FUNCTION foo(_t regclass)
RETURNS void
LANGUAGE plpgsql AS
$func$
BEGIN
EXECUTE 'ALTER TABLE ' || _t || ' ADD COLUMN c1 varchar(20)
, ADD COLUMN c2 varchar(20)';
END
$func$;
Call:
SELECT foo('table_name');
Or:
SELECT foo('my_schema.table_name'::regclass);
Aside: consider using just text instead of varchar(20).
3. quote_ident()
The 2nd query sanitized:
CREATE OR REPLACE FUNCTION foo(_t regclass, _c text)
RETURNS void
LANGUAGE plpgsql AS
$func$
BEGIN
EXECUTE 'UPDATE ' || _t -- sanitized with regclass
|| ' SET ' || quote_ident(_c) || ' = ''This is a test''';
END
$func$;
For multiple concatenations / interpolations, format() is cleaner ...
Related answers:
Table name as a PostgreSQL function parameter
Postgres functions vs prepared queries
Case sensitive!
Be aware that unquoted identifiers are not cast to lower case here. When used as identifier in SQL [Postgres casts to lower case automatically][7]. But here we pass strings for dynamic SQL. When escaped as demonstrated, CaMel-case identifiers (like UserS) will be preserved by doublequoting ("UserS"), just like other non-standard names like "name with space" "SELECT"etc. Hence, names are case sensitive in this context.
My standing advice is to use legal lower case identifiers exclusively and never worry about that.
Aside: single quotes are for values, double quotes are for identifiers. See:
Are PostgreSQL column names case-sensitive?
Let's say a have a stored procedure SetCustomerName which has an input parameter Name, and I have a table customers with column Name.
So inside my stored procedure I want to set customer's name. If I write
UPDATE customers SET Name = Name;
this is incorrect and I see 2 other ways:
UPDATE customers SET Name = `Name`;
UPDATE customers SET customers.Name = Name;
First one works, but I didn't find in documentation that I can wrap parameters inside ` characters. Or did I miss it in the documentation (link is appreciated in this case).
What other ways are there and what is the standard way for such a case? Renaming input parameter is not good for me (because I have automatic object-relational mapping if you know what I mean).
UPDATE:
So, there is a link about backticks (http://dev.mysql.com/doc/refman/5.0/en/identifiers.html) but it's not explained deep enough how to use them (how to use them with parameters and column names).
And there is a very strange thing (at least for me): You can use backticks either way:
UPDATE customers SET Name = `Name`;
//or
UPDATE customers SET `Name` = Name;
//or even
UPDATE customers SET `Name` = `Name`;
and they all work absolutely the same way.
Don't you think this is strange? Is this strange behavior explained somewhere?
Simplest way to distinguished between your parameter and column (if both name is same) is to add table name in your column name.
UPDATE customers SET customers.Name = Name;
Even you can also add database prefix like
UPDATE yourdb.customers SET yourdb.customers.Name = Name;
By adding database name you can perform action on more than 1 database from single store procedure.
I think that your first example is actually backwards. If you're trying to set the "Name" column to the "Name" input parameter, I believe it should be:
UPDATE customers SET `Name` = Name;
And for the second example, you can set table aliases the same way that you do in all other statements:
UPDATE customers AS c SET c.Name = Name;
Not necessarily correct, but a fair way to better argument/parameter management, as well readability with easier understanding, especially while working with the SQL;
DROP PROCEDURE IF EXISTS spTerminalDataDailyStatistics; DELIMITER $$
CREATE PROCEDURE spTerminalDataDailyStatistics(
IN TimeFrom DATETIME,
IN DayCount INT(10),
IN CustomerID BIGINT(20)
) COMMENT 'Daily Terminal data statistics in a date range' BEGIN
# Validate argument
SET #TimeFrom := IF(TimeFrom IS NULL, DATE_FORMAT(NOW(), '%Y-%m-01 00:00:00'), TimeFrom);
SET #DayCount := IF(DayCount IS NULL, 5, DayCount);
SET #CustomerID := CustomerID;
# Determine parameter
SET #TimeTo = DATE_ADD(DATE_ADD(#TimeFrom, INTERVAL #DayCount DAY), INTERVAL -1 SECOND);
# Do the job
SELECT DATE_FORMAT(TD.TerminalDataTime, '%Y-%m-%d') AS DataPeriod,
COUNT(0) AS DataCount,
MIN(TD.TerminalDataTime) AS Earliest,
MAX(TD.TerminalDataTime) AS Latest
FROM pnl_terminaldata AS TD
WHERE TD.TerminalDataTime BETWEEN #TimeFrom AND #TimeTo
AND (#CustomerID IS NULL OR TD.CustomerID = #CustomerID)
GROUP BY DataPeriod
ORDER BY DataPeriod ASC;
END $$
DELIMITER ;
CALL spTerminalDataDailyStatistics('2021-12-01', 2, 1801);
Using backticks in MySQL query syntax is documented here:
http://dev.mysql.com/doc/refman/5.0/en/identifiers.html
So yes, your first example (using backticks) is correct.
Here is the link you are asking for:
http://dev.mysql.com/doc/refman/5.0/en/identifiers.html
The backticks are called "identifier quote" in MySql