Hi I have the following data column where the typical data will look like:
Row 1: RCS CARD: THANK YOU FOR YOUR PURCHASE AT PICK N PAY ON CARD ...1820 FOR R371.71 ON 14-03-2013 AT 09:46. AVAIL CREDIT R67. FOR QUERIES CALL 0861028889
Row 2: RCS CARD: THANK YOU FOR YOUR PURCHASE AT PICK N PAY ON CARD ...6825 FOR R3061.93 ON 14-03-2013 AT 09:45. AVAIL CREDIT R39. FOR QUERIES CALL 0861028889
I need to be able to extract the R371.71 and R3061.93 from row 1 and 2. What is the most accurate way to do this? Keeping in mind that R amount will change from row to row so a simple substring will not work?
Any advice would be extremely helpful.
Thanks,
Jonathan
Well the proper way to do it is to use regexp in an external script/app since MySQL doesn't support regular expression sub strings.
If you do insist on using SQL the only way I could think of is by assuming that the string starts with:
RCS CARD: THANK YOU FOR YOUR PURCHASE AT PICK N PAY ON CARD
and just ignore that part. so the SQL should be:
SELECT SUBSTR(t, LOCATE('FOR', t, 61)+5 ,LOCATE('ON', t, 61)-1-LOCATE('FOR', t, 61)-5)
FROM DATA
Again I would use regexp but you can see it's working in this SQLFiddle: http://sqlfiddle.com/#!2/966ad/7
If the column in concern has consistent text format as you mentioned in the question, then you can make use of substring_index, locate and substring functions to find the amount value.
select
-- column_name,
substring_index( substring( column_name,
locate( 'FOR R', column_name, 1 )
+ length( 'FOR R' )
- 1
), ' ', 1
) as amount
from table_name
where
column_name like '%RCS CARD: THANK YOU FOR YOUR PURCHASE AT PICK N PAY ON CARD%';
Demo # MySQL 5.5.32 Fiddle
If you want to extract only the amount without prefix 'R' then, remove the '-1' line from the above query.
Related
I want to count how many columns in a row are not NULL.
The table is quite big (more than 100 columns), therefore I would like to not do it manually or using php (since I dont use php) using this approach Counting how many MySQL fields in a row are filled (or empty).
Is there a simple query I can use in a select like SELECT COUNT(NOT ISNULL(*)) FROM big_table;
Thanks in advance...
Agree with comments above:
There is something wrong in the data since there is a need for such analysis.
You can't completely make it automatic.
But I have a recipe for you for simplifying the process. There are only 2 steps needed to achieve your aim.
Step 0. In the step1 you'll need to get the name of your table schema. Normally, the devs know in what schema does the table reside, but still... Here is how you can find it
select *
from information_schema.tables
where table_name = 'test_table';
Step 1. First of all you need to get the list of columns. Getting just the list of cols won't help you out at all, but this list is all we need to be able to create SELECT statement, right? So, let's make database to prepare select statement for us
select concat('select (length(concat(',
group_concat(concat('ifnull(', column_name, ', ''###'')') separator ','),
')) - length(replace(concat(',
group_concat(concat('ifnull(', column_name, ', ''###'')') separator ','),
'), ''###'', ''''))) / length(''###'')
from test_table')
from information_schema.columns
where table_schema = 'test'
and table_name = 'test_table'
order by table_name,ordinal_position;
Step 3. Execute statement you've got on step 2.
select (length(concat(.. list of cols ..)) -
length(replace(concat(.. list of cols .. ), '###', ''))) / length('###')
from test_table
The select looks tricky but it's simple: first replace all nulls with some symbols that you're sure you'll never get in those columns. I usually do that replacing nulls with "###". that what all that "ifnull"s are here for.
Next, count symbols with "length". In my case it was 14
After that, replace all "###" with blanks and count length again. It's 11 now. For that I was using "length(replace" functions together
Last, just divide (14 - 11) by a length of a replacement string ("###" - 3). You'll get 1. This is exactly amount of nulls in my test string.
Here's a test case you can play with
Do not hesitate to ask if needed
I am trying to replace substrings within one text column in my table using a reference table.
To my knowledge, the replace(column, string1,string2) function will only work with strings as the second and third input.
Here is a visual of what I am trying to do. To be clear, the reference table I need to use is much larger - otherwise, I would use four replace functions.
EDIT: Thank you to everyone who has pointed out how bad this data model is built. Though I am not an expert on building efficient data models, I do know this one is built terribly. However, the structure of this model is completely out of my control. Apologies for not mentioning that from the get-go.
table1
Farms
Animals
Farm1
Cow, Pig
Farm2
Dog, Cow, Cat
Farm3
Dog
referenceTable
refColumn1
refColumn2
Cow
Moo
Pig
Oink
Dog
Bark
Cat
Meow
And here is what I would like the result column to be..
table1
Farms
Animals
Farm1
Moo, Oink
Farm2
Bark, Moo, Meow
Farm3
Bark
First question on stackoverflow so apologies if I missed anything.
Any help is appreciated! Thank you!
To loop over comma (or ', ' in this case) separated values, you can use a double substring_index and a join against a sequence table (where the sequence is <= the number of joined values in a given row, as determined with char_length/replace):
select t1.Farms, group_concat(rt.refColumn2 order by which.n separator ', ') Animals
from table1 t1
join (select 1 n union select 2 union select 3) which
on ((char_length(t1.Animals)-char_length(replace(t1.Animals,', ','')))/char_length(', '))+1 >= which.n
join referenceTable rt on rt.refColumn1=substring_index(substring_index(t1.Animals,', ',which.n),', ',-1)
group by t1.Farms
Here I use an ad hoc sequence table of 1 through 3, assuming no row will have more than 3 animals; expand as necessary or alternatively use a cte.
You have a really lousy data model and you should fix it. You should not be storing multiple values in a string column. Each value pair should be on its own row.
Let me assume that someone else created these tables and you have no choice. If that is the case, MySQL has a solution. I think I would suggest:
select t1.*, -- or whatever columns you want
(select group_concat(rt.refColumn2
order by find_in_set(rt.refColumn1, replace(t1.animals, ', ', ','))
separator ', '
)
from referenceTable rt
where find_in_set(rt.refColumn1, replace(t1.animals, ', ', ',')) > 0
)
from table1 t1
I'm more fluent in Sql Server than MySql, having got a solution working in Sql Server the real challenge was converting to a working MySql version!
See if this meets your needs. It works for your sample data, you may of course need to tweak if it doesn't fully represent your real world data.
with w as (
select *, case when animals like '%' || refcol1 || '%' then locate(refcol1,animals) end pos
from t1
join lateral (select * from t2)t2 on 1=1
)
select farms, group_concat(refcol2 order by pos separator ',') as Animals
from w
where pos>0
group by farms
order by farms
Working DB<>Fiddle
just as the question can we do something to get the length and first 3 characters of the employee name of one column
Please do not mark as answered or duplicate
i have the test tomorrow Advance SQL so I am trying to solve some imp question..
Please answer the problem
thanks again
Hi Shanu, You can use LEN() or LENGTH()(in case of oracle sql) function to get the length of a column.
SELECT LEN(column_name) FROM table_name;
And you can use SUBSTRING or SUBSTR() function go get first three characters of a column.
SUBSTRING( string, start_position, length );
SELECT SUBSTRING( column_name, 1, 3 ) FROM table_name;
To get both together use concatenation operator,
SELECT LEN(column_name)||SUBSTRING( column_name, 1, 3 ) FROM table_name;
Hope you got what you need. Any issues, feel free to ask
We can use SUBSTRING or SUBSTR() function, go get first three characters of a column.
And then try this particular query:
SELECT SUBSTRING(ename,1,3)
FROM emp;
Select len(ename) as Column_Length, left(ename,3) first_three_char from employee; ---------need to code your query. Should not use test format, will be confusing
You can also use substring function instead of left. Query will look like
Select len(ename) as Column_Length,substring(ename,1,3) first_three_char from employee;
SELECT LEN(EMPLOYEE_NAME),LEFT(EMPLOYEE_NAME,3) FROM EMPLOYEE_TABLE;
I have a field called 'areasCovered' in a MySQL database, which contains a string list of postcodes.
There are 2 rows that have similar data e.g:
Row 1: 'B*,PO*,WA*'
Row 2: 'BB*, SO*, DE*'
Note - The strings are not in any particular order and could change depending on the user
Now, if I was to use a query like:
SELECT * FROM technicians WHERE areasCovered LIKE '%B*%'
I'd like it to return JUST Row 1. However, it's returning Row 2 aswell, because of the BB* in the string.
How could I prevent it from doing this?
The key to using like in this case is to include delimiters, so you can look for delimited values:
SELECT *
FROM technicians
WHERE concat(', ', areasCovered, ', ') LIKE '%, B*, %'
In MySQL, you can also use find_in_set(), but the space can cause you problems so you need to get rid of it:
SELECT *
FROM technicians
WHERE find_in_set('B', replace(areasCovered, ', ', ',') > 0
Finally, though, you should not be storing these types of lists as strings. You should be storing them in a separate table, a junction table, with one row per technician and per area covered. That makes these types of queries easier to express and they have better performance.
You are searching wild cards at the start as well as end.
You need only at end.
SELECT * FROM technicians WHERE areasCovered LIKE 'B*%'
Reference:
Normally I hate REGEXP. But ho hum:
SELECT * FROM technicians
WHERE concat(",",replace(areasCovered,", ",",")) regexp ',B{1}\\*';
To explain a bit:
Get rid of the pesky space:
select replace("B*,PO*,WA*",", ",",");
Bolt a comma on the front
select concat(",",replace("B*,PO*,WA*",", ",","));
Use a REGEX to match "comma B once followed by an asterix":
select concat(",",replace("B*,PO*,WA*",", ",",")) regexp ',B{1}\\*';
I could not check it on my machine, but it's should work:
SELECT * FROM technicians WHERE areasCovered <> replace(areaCovered,',B*','whatever')
In case the 'B*' does not exist, the areasCovered will be equal to replace(areaCovered,',B*','whatever'), and it will reject that row.
In case the 'B*' exists, the areCovered will NOT be eqaul to replace(areaCovered,',B*','whatever'), and it will accept that row.
You can Do it the way Programming Student suggested
SELECT * FROM technicians WHERE areasCovered LIKE 'B*%'
Or you can also use limit on query
SELECT * FROM technicians WHERE areasCovered LIKE '%B*%' LIMIT 1
%B*% contains % on each side which makes it to return all the rows where value contains B* at any position of the text however your requirement is to find all the rows which contains values starting with B* so following query should do the work.
SELECT * FROM technicians WHERE areasCovered LIKE 'B*%'
I have the following:
SELECT * FROM users LEFT JOIN user_info ON users.id=user_info.user_id
WHERE
((user_info.tester != 1) OR (user_info.tester is null)) AND
id in (SELECT explicituser_id FROM user_login WHERE (created < '2012-12-17' OR created >= date_add('2012-12-17', interval 1 day))) AND
id IN (SELECT participte_id FROM roster WHERE roster_id IN (6))
order by
substring_index(users.name, ' ', -1)
I'm simply trying to sort by the users' last name.
However, while it can sort by the first name, the last name is buggy. If the user has quotes around their name (ie. "Abigail Martinez" it will make the sorting incorrect. If the user provides only one name, and it's a nickname (ie. Juan), then it will also make it incorrect. And then there's middle initials (ie. Tiffany S Villa or Steve de la Makinov). Unfortunately, this uses only one column for the full name (users.name).
Any help is appreciated. Thanks!
substring_index(TRIM(users.name), ' ', -1) Adding TRIM will remove trailing spaces. After that, sorting occurs as expected.
if it is the case try to get all the rows and then may try with mysql_real_escape_string. so that you will not be having quotes. it'll be something like this.
$row['new_name'] = mysql_real_escape_string($row['name']);
I'm not too sure performance wise whether its good to go for it or not..
Hope this may helps