I'm using Elasticbeanstalk (and a MySQL RDS) for my client application and an EC2 to act as a utility server to keep all individual client databases up to date. Because Elasticbeanstalk is dodgy when it comes to files unless I'm using an S3 I figured it'd be better to use a database only solution. My goal is to create a checksum of each client database and track a running history of expected checksums to know when it's time to update.
My question is: What is the best way to easily determine that a production schema matches the installed schema in MySQL?
I wrote this query to handle it, but are there any pitfalls doing it this way?
SELECT MD5(GROUP_CONCAT(
COALESCE(TABLE_NAME, ''),
COALESCE(COLUMN_NAME, ''),
COALESCE(ORDINAL_POSITION, ''),
COALESCE(COLUMN_DEFAULT, ''),
COALESCE(IS_NULLABLE, ''),
COALESCE(DATA_TYPE, ''),
COALESCE(CHARACTER_MAXIMUM_LENGTH, ''),
COALESCE(CHARACTER_SET_NAME, ''),
COALESCE(COLLATION_NAME, ''),
COALESCE(COLUMN_TYPE, ''),
COALESCE(COLUMN_KEY, ''))),
SHA1(GROUP_CONCAT(
COALESCE(TABLE_NAME, ''),
COALESCE(COLUMN_NAME, ''),
COALESCE(ORDINAL_POSITION, ''),
COALESCE(COLUMN_DEFAULT, ''),
COALESCE(IS_NULLABLE, ''),
COALESCE(DATA_TYPE, ''),
COALESCE(CHARACTER_MAXIMUM_LENGTH, ''),
COALESCE(CHARACTER_SET_NAME, ''),
COALESCE(COLLATION_NAME, ''),
COALESCE(COLUMN_TYPE, ''),
COALESCE(COLUMN_KEY, '')))
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = DATABASE();
Related
I have a query that I'm trying to use to update only cells with empty strings to NULL. But instead, when I run this command ALL of the cells end up being NULL:
UPDATE table_name
SET InvoiceId=NULL,
LinkedAccountId=NULL,
RecordId=NULL,
ProductName=NULL,
RateId=NULL,
SubscriptionId=NULL,
PricingPlanId=NULL,
UsageType=NULL,
Operation=NULL,
AvailabilityZone=NULL,
ReservedInstance=NULL,
UsageStartDate=NULL,
UsageEndDate=NULL,
UsageQuantity=NULL,
BlendedRate=NULL,
UnBlendedRate=NULL,
ResourceId=NULL,
Engagement=NULL,
Name=NULL,
Owner=NULL,
Parent=NULL
WHERE InvoiceId=''
OR LinkedAccountId=''
OR RecordId=''
OR ProductName=''
OR RateId=''
OR SubscriptionId=''
OR PricingPlanId=''
OR UsageType=''
OR UsageEndDate=''
OR Operation=''
OR AvailabilityZone=''
OR ReservedInstance=''
OR UsageStartDate=''
OR UsageEndDate=''
OR UsageQuantity=''
OR BlendedRate=''
OR UnBlendedRate=''
OR ResourceId=''
OR Engagement=''
OR Name=''
OR Owner=''
OR Parent='';
What am I doing wrong?
You are setting ALL fields to NULL where any field = ''
You might need to write as many queries as fields you want to update.
I might be wrong, but I don't think this is possible to do in a single query
UPDATE table_name
SET InvoiceId = NULL
WHERE InvoiceId = ''
And this for every fields
The way to interpret your SQL query is kind of like this:
UPDATE table_name
SET
(all these fields to null)
WHERE
(if ANY Of these conditions is true)
I agree with Cid that it might be the safest to do 1 field per query at a time, but this is how you could write this in a single query:
UPDATE table_name SET
field1 = IF(field1='',NULL,field1),
field2 = IF(field2='',NULL,field2),
/* etc */
I think you're assuming that the terms in your WHERE clause have an implicit correlation with the assignments in your SET clause, so only some columns will be set, depending on which ones satisfy individual terms in the WHERE clause. This is not how SQL works.
What really happens is that the whole condition of the WHERE clause is evaluated to select which rows are affected. Then all the SET assignments are applied, so all columns will be changed on the rows that satisfy the whole WHERE clause condition.
To do what you want in one pass, you could do it this way:
UPDATE table_name
SET InvoiceId=NULLIF(InvoiceId, ''),
LinkedAccountId=NULLIF(LinkedAccountId, ''),
RecordId=NULLIF(RecordId, ''),
ProductName=NULLIF(ProductName, ''),
RateId=NULLIF(RateId, ''),
SubscriptionId=NULLIF(SubscriptionId, ''),
PricingPlanId=NULLIF(PricingPlanId, ''),
UsageType=NULLIF(UsageTYpe, ''),
Operation=NULLIF(Operation, ''),
AvailabilityZone=NULLIF(AvailabilityZone, ''),
ReservedInstance=NULLIF(ReservedInstance, ''),
UsageStartDate=NULLIF(UsageStartDate, ''),
UsageEndDate=NULLIF(UsageEndDate, ''),
UsageQuantity=NULLIF(UsageQuantity, ''),
BlendedRate=NULLIF(BlendedRate, ''),
UnBlendedRate=NULLIF(UnBlendedRate, ''),
ResourceId=NULLIF(ResourceId, ''),
Engagement=NULLIF(Engagement, ''),
Name=NULLIF(Name, ''),
Owner=NULLIF(Owner, ''),
Parent=NULLIF(Parent, '')
The NULLIF() function returns NULL if its two arguments are equal, otherwise it returns the first argument. So in the case that each column is not '' then it is a no-op, setting the column to its own original value.
I have around 200,000 records of data with phone numbers, but the numbers are inconsistent.
for example, some may be 10 digits (missing a 0 at the beginning), some have spaces in there, some have a '-' in the middle and some begin with '+44' instead of 0.
Is there a way in mySQL to condition all these and cleanse the data in one query?
Without sample data and without an example output this is purely speculative and assuming you want the output in the format of 01234567891.
Use a combination of LENGTH, REPLACE' ANDLEFT` functions to resolve the 4 issues you highlighted:
Missing 0 at beggining.
Spaces in the string.
-'s in the string.
+44 rather than 0.
SELECT CASE WHEN LENGTH(REPLACE(REPLACE(numberfield, '-', ''), ' ', '')) = 10
THEN CONCAT('0', REPLACE(REPLACE(numberfield, '-', ''), ' ', ''))
WHEN LEFT(REPLACE(REPLACE(numberfield, '-', ''), ' ', ''), 3) = '+44'
THEN REPLACE(REPLACE(REPLACE(numberfield, '-', ''), ' ', ''), '+44', '0'
END AS Cleannumber
FROM yourtable
Assuming the phone number field is a string - the following should deal with the conditions you specified :
RIGHT( LPAD( REPLACE( REPLACE( REPLACE('phonenumber', '-', ''), '+44', ''), ' ', ''), 11, '0' ), 11 )
first any '-' are removed, then '+44' is removed, then spaces are removed, then 11 '0's are added to the start of the number, finally the rightmost 11 characters are taken.
So you would do an UPDATE query replacing the phonenumber column.
This is the query i execute after insertation in some table in my database. Here i am getting rate_like_id and all other fields i also save them but when i use CONCAT and trying to save them as json it inserts NULL value via trigger.
Is this a wrong way to do this. Is ther any other way i can save this type of data via trigger mysql.
INSERT INTO `log_activity`(`visitor_id`,
`rating_like_id`,
`type`,
`response`,
`created_by`,
`created_date`)
VALUES (new.created_by,
new.rating_like_id,
CASE WHEN new.is_like = NULL THEN "Rating" ELSE "Like" END ,
-- Here is this column Should not be null but it is coming null when INSERTED
CONCAT(
'{"card_id":','"',new.card_id,'"',
',"card_type":','"',new.card_type,'"',
',"user_id":','"',new.user_id,'"',
',"is_like":','"',new.is_like,'"',
',"has_rated":','"',new.has_rated,'"',
',"rate":','"',new.rate,'"',
',"created_by":','"',new.created_by,'"',
',"created_date":','"',new.created_date,'"',
',"card_type":','"',new.card_type,'"','}'),
new.created_by,
new.created_date)
what is wrong with this .. all other fields are saved correctly only CONCAT field is not getting inserted properly.
Yes I am able to insert value like this
CONCAT(" hi "," how "," are "," you ", " ? ")
Is this problem because i am using "new.fieldName" inside CONCAT ... i see THIS question they are doing the same thing .. ofcource it is less complex compare to mine.
If you see comments and conversation between me and marcB.. you will get the answer...
Here i want to show what changes i made to my query to make it work.
marcB said :- Remember that sql null is contagious. if ANY of the fields you're concatting are themselves null, the ENTIRE result becomes null
so you can see my query in question.. i do not have checked for any value is it null or not. So i just had to add the null check.
INSERT INTO `log_activity`(`visitor_id`,
`rating_like_id`,
`type`,
`response`,
`created_by`,
`created_date`)
VALUES (new.created_by,
new.rating_like_id,
CASE WHEN new.is_like = NULL THEN "Rating" ELSE "Like" END,
CONCAT('{"card_id":',
'"',
IFNULL(new.card_id, ''),
'"',
',"card_type":',
'"',
IFNULL(new.card_type, ''),
'"',
',"user_id":',
'"',
IFNULL(new.user_id, ''),
'"',
',"is_like":',
'"',
IFNULL(new.is_like, ''),
'"',
',"has_rated":',
'"',
IFNULL(new.has_rated, ''),
'"',
',"rate":',
'"',
IFNULL(new.rate, ''),
'"',
',"created_by":',
'"',
IFNULL(new.created_by, ''),
'"',
',"created_date":',
'"',
IFNULL(new.created_date, ''),
'"',
',"card_type":',
'"',
IFNULL(new.card_type, ''),
'"',
'}'),
new.created_by,
new.created_date)
It checks for the null value of the field.
I need a little help in queries.
I have a table like this:
id value
1 rs-123
2 rsa-123
I need to get the first row if the user queries in following ways : rs123, rs-123, rs 123 (either using or skipping the space and dash).
WHERE REPLACE(REPLACE(value, '-', ''), ' ', '')
= REPLACE(REPLACE($val , '-', ''), ' ', '')
I have those strings in y table:
abcdefg_1056-DF or
123erttzz-1292 or
gdfgdfg_1056
What I want is only the first part, abcdefg for example for the first string. So I can replace all numbers and all -DF with an empty string, but I dont know how.
Ideas?
I would suggest a regex replace. Have a look here - I think this might help you:
How to do a regular expression replace in MySQL?
If you prever a lazy and ugly method (and not recommended for very, very much rows), you can spare yourself a user defined function like in mwerner's answer and do it simply like this:
select
replace(replace(replace(replace(replace(replace(replace(replace(replace(replace(replace(replace(asdf, '0', ''), '1', ''), '2', ''), '3', ''), '4', ''), '5', ''), '6', ''), '7', ''), '8', ''), '9', ''), '_', ''), '-DF', '')
from
(
select
'abcdefg_1056-DF' as asdf
union select
'123erttzz-1292'
union select
'gdfgdfg_1056'
)q