Sorry for the long post, but the tables involved are quite big.
When I run the query bellow it normally takes about 1m to run. However, when I remove the correlated sub-query I get the query down to 15 seconds. So I think that's the actual problem.
Problem is, I don't really know how to get the SUM results without the correlated query.
QUERY I'm trying to run
SELECT cl_clients.vat as association_vat, cl_clients.name as association_name, cl_clients_with_regions.city_id, cl_clients_with_regions.nut_1, cl_clients_with_regions.nut_2, cl_clients_with_regions.nut_3,
company.vat as company_vat, company.name, company.section, company.division, company.cae,
SUM((
SELECT SUM((COALESCE((cl_finances.sales_community_market), 0) + COALESCE((cl_finances.sales_extra_market), 0)))
from cl_finances
where cl_finances.vat = company.vat
and cl_finances.year = 2018
)) as total_sum
FROM `cl_clients_with_regions`
JOIN `cl_client_intervention_areas` ON `cl_client_intervention_areas`.`city_id` = `cl_clients_with_regions`.`city_id`
JOIN `cl_clients` ON `cl_clients`.`vat` = `cl_client_intervention_areas`.`client_vat`
INNER JOIN cl_clients_with_regions company ON cl_clients_with_regions.vat = company.vat
WHERE `cl_clients_with_regions`.`encrypted_vat` IS NOT NULL
AND `cl_clients_with_regions`.`country` IS NOT NULL
AND `cl_clients_with_regions`.`cae` IS NOT NULL
AND `cl_clients_with_regions`.`cae` != ''
AND `cl_clients_with_regions`.`division` IS NOT NULL
AND `cl_clients_with_regions`.`section` = 'A'
AND `cl_clients_with_regions`.`nut_2` = 'Centro'
GROUP BY association_vat;
CL Clients create table syntax
create table cl_clients
(
vat varchar(20) default '' not null,
encrypted_vat varchar(32) null,
temporary_vat enum ('0', '1', '') default '0' null,
sig_id int null,
phc_id int null,
client_manager int null,
name varchar(300) null,
brand varchar(255) null,
`group` varchar(200) null,
class_internacional varchar(45) null,
logoimage varchar(400) null,
social_capital varchar(100) null,
address varchar(300) null,
gps varchar(25) null,
parish varchar(70) null,
zip_code varchar(10) null,
city int null,
district int null,
country int default 75 not null,
headquarter int null,
person_title varchar(7) null,
person_contact varchar(350) null,
person_phone varchar(20) null,
person_cell varchar(15) null,
person_email varchar(150) null,
person_function varchar(150) null,
language varchar(10) null,
phone varchar(20) null,
mobile_phone varchar(20) null,
fax varchar(20) null,
email varchar(50) null,
email_alternative varchar(50) null,
website varchar(300) null,
pme varchar(500) null,
pme_lider varchar(500) null,
cae varchar(15) null,
cae_2 varchar(15) null,
certified enum ('1', '0', '') null,
sector varchar(350) null,
workers int null,
foundation_date date null,
size enum ('Micro Empresa', 'Pequena empresa', 'Média empresa', 'Grande empresa', 'Não aplicável', 'PME', '') default '' null,
company_type varchar(455) default 'Empresa' null,
type enum ('novo', 'antigo', 'analizar', '') default 'analizar' null,
entity enum ('client', 'prospect', 'suplier', 'potential', '') default 'prospect' null,
client_to_country int null,
partner enum ('1', '0', '') default '0' not null,
partner_dp enum ('0', '1', '') default '0' null,
lucrative enum ('1', '0', '') default '1' not null,
dun varchar(100) null,
debt enum ('50k', '100k', '500k', '1m', '5m', '+5m', '') null,
bankruptcy enum ('1', '0', '') default '0' null,
competitors enum ('1', '0', '') default '0' null,
dun_date date null,
bank_1 varchar(30) null,
bank_2 varchar(30) null,
bank_3 varchar(30) null,
status enum ('0', '1', '') default '1' null,
followup enum ('0', '1', '') default '1' null,
classification enum ('Prestige', 'Premium', 'Current', '') default 'Current' null,
associated enum ('0', '1', '') default '0' null,
industry_id int null,
valid enum ('0', '1', '') default '1' null,
imported_at timestamp null,
modified_by int default 0 null,
responsible_id int null,
partner_type int null,
iberinform_id int null,
iberinform_date timestamp null,
legal_form text null,
going_concern text null,
score int null,
sector_score int null,
tax_status text null,
irc_debtors_ledger enum ('0', '1') null,
ss_debtors_ledger enum ('0', '1') null,
association_class_id int null,
nature enum ('business', 'commercial', 'sectorial', 'other') null,
geographic_scope enum ('national', 'regional', 'local') null,
intervention_nut enum ('nut_1', 'nut_2', 'nut_3') null,
ch_is_associated enum ('0', '1') null,
bi enum ('0', '1') default '0' null,
updated_at timestamp null,
constraint encrypted_vat_UNIQUE
unique (encrypted_vat),
constraint vat
unique (vat),
constraint cl_clients_cl_caes_cae_fk
foreign key (cae) references cl_caes (cae)
on update cascade,
constraint clients_industry_id_fk
foreign key (industry_id) references cl_industries (id),
constraint cltns_asso_clss_id_fk
foreign key (association_class_id) references cl_association_classifications (id)
on update cascade on delete set null,
constraint country_fk_id
foreign key (country) references cl_countries (id)
on update cascade on delete cascade
);
create index cl_client_name_idx
on cl_clients (name);
create index cl_clients_bi_idx
on cl_clients (bi);
create index cl_clients_brand_idx
on cl_clients (brand);
create index cl_clients_company_type_idx
on cl_clients (company_type);
create index cl_clients_geo_index
on cl_clients (city, district);
create index cl_clients_geographic_scope_idx
on cl_clients (geographic_scope);
create index cl_clients_group_idx
on cl_clients (`group`);
create index cl_clients_nature_idx
on cl_clients (nature);
create index cltns_asso_clss_id_fk_idx
on cl_clients (association_class_id);
create index country_idx
on cl_clients (country);
create index fk_clients_industries1_idx
on cl_clients (industry_id);
alter table cl_clients
add primary key (vat);
VIEW being used in the query
create view cl_clients_with_regions as
select `grupoch`.`cl_clients`.`vat` AS `vat`,
`grupoch`.`cl_clients`.`encrypted_vat` AS `encrypted_vat`,
`grupoch`.`cl_clients`.`name` AS `name`,
`grupoch`.`cl_clients`.`brand` AS `brand`,
`grupoch`.`cl_clients`.`country` AS `country`,
`grupoch`.`cl_clients`.`district` AS `district`,
`grupoch`.`cl_clients`.`city` AS `city`,
`grupoch`.`cl_cities`.`id` AS `city_id`,
`grupoch`.`cl_cities`.`name` AS `city_name`,
`grupoch`.`cl_cities`.`nut_1` AS `nut_1`,
`grupoch`.`cl_cities`.`nut_2` AS `nut_2`,
`grupoch`.`cl_cities`.`nut_3` AS `nut_3`,
`grupoch`.`cl_clients`.`cae` AS `cae`,
`grupoch`.`cl_caes`.`description` AS `cae_designation`,
`grupoch`.`cl_caes_divisions`.`division` AS `division`,
`grupoch`.`cl_caes_divisions`.`division_designation` AS `division_designation`,
`grupoch`.`cl_caes_divisions`.`section` AS `section`,
`grupoch`.`cl_caes_divisions`.`section_designation` AS `section_designation`
from (((`grupoch`.`cl_clients` join `grupoch`.`cl_cities` on ((
(`grupoch`.`cl_clients`.`city` = `grupoch`.`cl_cities`.`city_cod`) and (`grupoch`.`cl_clients`.`district` =
`grupoch`.`cl_cities`.`district_id`)))) join `grupoch`.`cl_caes` on ((`grupoch`.`cl_caes`.`cae` = `grupoch`.`cl_clients`.`cae`)))
join `grupoch`.`cl_caes_divisions`
on ((`grupoch`.`cl_caes_divisions`.`division` = `grupoch`.`cl_caes`.`division_id`)))
where ((`grupoch`.`cl_clients`.`country` in (75, 185, 186)) and (`grupoch`.`cl_clients`.`vat` is not null));
CL finances create table syntax
create table cl_finances
(
vat varchar(20) not null,
year int not null,
workers int null,
sells varchar(300) null,
sells_variation varchar(300) null,
international_sells varchar(300) null,
international_sells_variation varchar(300) null,
liquid_results varchar(300) null,
liquid_results_variation varchar(300) null,
capital varchar(300) null,
active_liquid varchar(300) null,
financial_autonomy varchar(300) null,
modified_by int default 0 null,
sales_profitability decimal(13, 2) null,
return_on_capital decimal(13, 2) null,
sales_community_market decimal(13, 2) null,
sales_extra_market decimal(13, 2) null,
created_at timestamp null,
updated_at timestamp null,
primary key (vat, year),
constraint clients_finances_vat
foreign key (vat) references cl_clients (vat)
on update cascade on delete cascade
);
create index cl_finances_year_idx
on cl_finances (year);
create index vat
on cl_finances (vat);
## Create table syntax for cl_client_intervention_areas
create table cl_client_intervention_areas
(
client_vat varchar(20) not null,
city_id int not null,
constraint fk_cl_client_intervention_areas_cl_cities1
foreign key (city_id) references cl_cities (id)
on update cascade on delete cascade,
constraint fk_cl_client_intervention_areas_cl_clients1
foreign key (client_vat) references cl_clients (vat)
on update cascade on delete cascade
);
create index fk_cl_client_intervention_areas_cl_cities1_idx
on cl_client_intervention_areas (city_id);
create index fk_cl_client_intervention_areas_cl_clients1_idx
on cl_client_intervention_areas (client_vat);
The explain result from this query is quite big
I believe the subquery is being executed once per row in the group by clause.
Could you try the following if it makes it any better.
Change i have done is to perform the correlated query as a inline-view and then join with the main table/view -- company.vat
SELECT cl_clients.vat as association_vat, cl_clients.name as association_name, cl_clients_with_regions.city_id, cl_clients_with_regions.nut_1, cl_clients_with_regions.nut_2, cl_clients_with_regions.nut_3,
company.vat as company_vat, company.name, company.section, company.division, company.cae,
SUM(temp_val.sum_val) as total_sum
FROM `cl_clients_with_regions`
JOIN `cl_client_intervention_areas` ON `cl_client_intervention_areas`.`city_id` = `cl_clients_with_regions`.`city_id`
JOIN `cl_clients` ON `cl_clients`.`vat` = `cl_client_intervention_areas`.`client_vat`
INNER JOIN cl_clients_with_regions company ON cl_clients_with_regions.vat = company.vat
LEFT JOIN (SELECT cl_finances.vat
,SUM((COALESCE((cl_finances.sales_community_market), 0) + COALESCE((cl_finances.sales_extra_market), 0))) as sum_val
FROM cl_finances
WHERE 1=1
AND cl_finances.year = 2018
GROUP BY cl_finances.vat
)temp_val
ON company.vat=temp_val.vat
WHERE `cl_clients_with_regions`.`encrypted_vat` IS NOT NULL
AND `cl_clients_with_regions`.`country` IS NOT NULL
AND `cl_clients_with_regions`.`cae` IS NOT NULL
AND `cl_clients_with_regions`.`cae` != ''
AND `cl_clients_with_regions`.`division` IS NOT NULL
AND `cl_clients_with_regions`.`section` = 'A'
AND `cl_clients_with_regions`.`nut_2` = 'Centro'
GROUP BY association_vat;
To further explain the above answer:
Subqueries should never be in the select clause because they execute once per record that is returned / or calculated to return. This causes the query to get slower and slower as the database gets larger.
Instead, the subquery should be in the FROM clause, and the result of the subquery should be joined to and selected.
I have 2 tables 'open_invoices' and 'paid_invoices' with the below structures:
CREATE TABLE `open_invoices` (
`id` int(11) UNSIGNED NOT NULL AUTO_INCREMENT,
`unique_identifier` varchar(255) NOT NULL,
`insert_date` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP,
`cust_nbr` int(11) NOT NULL,
`location` varchar(10) NOT NULL,
`company` varchar(10) NOT NULL,
`invoice_nbr` int(11) NOT NULL,
`future_letter` varchar(1) NOT NULL,
`invoice_total` decimal(8,2) NOT NULL,
`invoice_payments` decimal(8,2) NOT NULL,
`record_type` int(11) NOT NULL,
`terms_desc` varchar(255) NOT NULL,
`due_date` varchar(8) NOT NULL,
`discount_date` varchar(8) NOT NULL,
`orig_disc_avail` decimal(8,2) NOT NULL,
`cust_po` varchar(10) NOT NULL,
`invoice_date` varchar(8) NOT NULL,
`status` int(11) NOT NULL,
`check_no` varchar(10) NOT NULL,
`last_pay_date` varchar(8) NOT NULL,
`as_of_date` varchar(8) NOT NULL,
`as_of_time` varchar(11) NOT NULL,
`remaining` varchar(1) NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `unique_identifier` (`unique_identifier`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
CREATE TABLE `paid_invoices` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`date_change` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP,
`invoice_date` varchar(8) NOT NULL,
`unique_identifier` varchar(255) NOT NULL,
`amount_paid` decimal(8,2) NOT NULL,
`amount_left` decimal(8,2) NOT NULL,
`payment_type` varchar(2) NOT NULL,
`last4` varchar(255) NOT NULL,
`transac_id` varchar(255) NOT NULL,
`customer_id` varchar(255) NOT NULL,
`payment_P_F` varchar(1) NOT NULL,
PRIMARY KEY (`id`),
FOREIGN KEY (unique_identifier)
REFERENCES open_invoices (unique_identifier)
ON DELETE CASCADE
ON UPDATE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
I'm trying to insert a row in 'paid_invoices' like below:
INSERT INTO `paid_invoices` (`invoice_date`, `unique_identifier`, `amount_paid`, `amount_left`, `payment_type`, `last4`, `transac_id`, `customer_id`, `payment_P_F`) VALUES ('07/19/18', '126_89948_2576', 0, '37.45', '', '', '', '13776', 'P')
I get the below error:
Error Number: 1452
Cannot add or update a child row: a foreign key constraint fails (`accounts_receivables`.`paid_invoices`, CONSTRAINT `paid_invoices_ibfk_1` FOREIGN KEY (`unique_identifier`) REFERENCES `open_invoices` (`unique_identifier`))
INSERT INTO `paid_invoices` (`invoice_date`, `unique_identifier`, `amount_paid`, `amount_left`, `payment_type`, `last4`, `transac_id`, `customer_id`, `payment_P_F`) VALUES ('07/19/18', '126_89948_2576', 0, '37.45', '', '', '', '13776', 'P')
If I search on google, it says that the foreign key 126_89948_2576 was not found in the table 'open_invoices'.
When I do SELECT * FROM open_invoices WHERE id = 2576 I see the row with the foreign key 126_89948_2576 as shown in the picture below :
[![query 1][1]][1]
but when I do this query SELECT * FROM open_invoices WHERE unique_identifier = '126_89948_2576' I get no result see capture below:
[![query 2][2]][2]
I'm sure there is no extra blank space in the value saved for foreign key 126_89948_2576.
What is going on please? Is there a bug in the version of MySQL I'm using?
Server: Localhost via UNIX socket
Server type: MySQL
Server version: 5.6.40 - MySQL Community Server (GPL)
Protocol version: 10
Thanks.
[1]: https://i.stack.imgur.com/L2QDX.png
[2]: https://i.stack.imgur.com/T9Fz5.png
It seems you are using wrong divider symbols, "_" instead of "-" in your query. Try unique_identifier = '126-89948-2576'
Because the underscore _ is a wildcard like the percent %, except that it only looks for one character.
SQL pattern matching enables you to use "_" to match any single character and "%" to match an arbitrary number of characters (including zero characters).
you can use
SELECT * FROM open_invoices WHERE unique_identifier like '%126\_89948\_2576%'
This is a GCP MySQL Cloud instance, MySQL 2nd Gen 5.7. Google installed and configured it.
I am logged in as root:
When I run a script to create schemas, tables, and views, I get this error:
Error Code: 1227. Access denied; you need (at least one of) the SUPER privilege(s) for this operation
Here are the permissions assigned to root:
Here is the SQL that fails, in the form of a stored procedure:
CREATE DEFINER=`root`#`%` PROCEDURE `CreateFullCoverageTestCase`()
BEGIN
-- Create all the elements of the Full Coverage Test Case
-- *********************
-- Schema Store
-- *********************
Begin
DROP Schema if exists `store`;
END;
BEGIN
CREATE SCHEMA `store` ;
CREATE TABLE `store`.`city` (
`CityID` int(11) NOT NULL AUTO_INCREMENT,
`City` varchar(45) NOT NULL,
PRIMARY KEY (`CityID`),
UNIQUE KEY `City_UNIQUE` (`City`));
INSERT INTO store.city (City) VALUES('Cincinnati');
INSERT INTO store.city (City) VALUES('Columbus');
INSERT INTO store.city (City) VALUES('Kokomo');
INSERT INTO store.city (City) VALUES('Hillsdale');
INSERT INTO store.city (City) VALUES('Lexington');
INSERT INTO store.city (City) VALUES('Oxford');
CREATE TABLE `store`.`state` (
`StateID` int(11) NOT NULL AUTO_INCREMENT,
`State` varchar(45) NOT NULL,
`StateAbbreviation` varchar(2) NOT NULL,
PRIMARY KEY (`StateID`),
UNIQUE KEY `StateAbbreviation_UNIQUE` (`StateAbbreviation`),
UNIQUE KEY `State_UNIQUE` (`State`),
UNIQUE KEY `StateID_UNIQUE` (`StateID`));
INSERT INTO store.state (State, StateAbbreviation) VALUES('Indiana', 'IN');
INSERT INTO store.state (State, StateAbbreviation) VALUES('Kentucky', 'KY');
INSERT INTO store.state (State, StateAbbreviation) VALUES('Michigan', 'MI');
INSERT INTO store.state (State, StateAbbreviation) VALUES('Ohio', 'OH');
CREATE TABLE `store`.`store` (
`StoreID` int(11) NOT NULL AUTO_INCREMENT,
`StoreNumber` varchar(10) NOT NULL,
`AddressLine1` varchar(45) NOT NULL,
`AddressLine2` varchar(45) DEFAULT NULL,
`CityID` int(11) NOT NULL,
`StateID` int(11) NOT NULL,
`ZipCode` varchar(10) NOT NULL,
PRIMARY KEY (`StoreID`),
UNIQUE KEY `StoreNumber_UNIQUE` (`StoreNumber`));
INSERT INTO store.store (StoreNumber, AddressLine1, CityID, StateID, ZipCode) VALUES('S000000001', '111 Main', 1, 4, '45255-1321');
INSERT INTO store.store (StoreNumber, AddressLine1, CityID, StateID, ZipCode) VALUES('S000000002', '817 Nordyke', 1, 4, '45103-0000');
INSERT INTO store.store (StoreNumber, AddressLine1, CityID, StateID, ZipCode) VALUES('S000000003', '123 Green', 6, 4, '45056-0000');
End;
-- *********************
-- Schema HR (Human Resources)
-- *********************
BEGIN
DROP Schema if exists `hr`;
END;
BEGIN
CREATE SCHEMA `hr` ;
CREATE TABLE `hr`.`employee` (
`EmployeeID` INT NOT NULL AUTO_INCREMENT,
`LastName` VARCHAR(45) NOT NULL,
`FirstName` VARCHAR(45) NOT NULL,
`EmployeeNumber` VARCHAR(10) NOT NULL,
`AddressLine1` VARCHAR(45) NOT NULL,
`AddressLine2` VARCHAR(45) NULL,
`City` VARCHAR(45) NOT NULL,
`State` VARCHAR(2) NOT NULL,
`ZipCode` VARCHAR(10) NOT NULL,
PRIMARY KEY (`EmployeeID`),
UNIQUE INDEX `EmployeeNumber_UNIQUE` (`EmployeeNumber` ASC));
INSERT INTO hr.employee (lastName, firstName, EmployeeNumber, AddressLine1, City, State, ZipCode) VALUES('Cheaney', 'Calbert', 'E000000001', '17th Street', 'Bloomington', 'IN', '47405');
INSERT INTO hr.employee (lastName, firstName, EmployeeNumber, AddressLine1, City, State, ZipCode) VALUES('Reynolds', 'Chris', 'E000000002', 'Fee Street', 'Bloomington', 'IN', '47405');
END;
-- *********************
-- Products for sale
-- *********************
BEGIN
DROP Schema if exists `product`;
END;
BEGIN
CREATE SCHEMA `product` ;
CREATE TABLE `product`.`manufacturer` (
`ManufacturerID` int(11) NOT NULL AUTO_INCREMENT,
`Manufacturer` varchar(45) NOT NULL,
`Comment` varchar(1000) DEFAULT NULL,
`AddressLine01` varchar(45) DEFAULT NULL,
`AddressLine02` varchar(45) DEFAULT NULL,
`City` varchar(45) DEFAULT NULL,
`State` varchar(45) DEFAULT NULL,
`ZipCode` varchar(10) DEFAULT NULL,
`Phone` varchar(13) DEFAULT NULL,
`Manufacturercol` varchar(45) DEFAULT NULL,
PRIMARY KEY (`ManufacturerID`),
UNIQUE KEY `Manufacturer_UNIQUE` (`Manufacturer`));
INSERT INTO product.manufacturer (Manufacturer) VALUES('Smuckers');
INSERT INTO product.manufacturer (Manufacturer) VALUES('Dannon');
INSERT INTO product.manufacturer (Manufacturer) VALUES('Kellogs');
INSERT INTO product.manufacturer (Manufacturer) VALUES('Pepsi');
CREATE TABLE `product`.`unit` (
`UnitID` INT NOT NULL AUTO_INCREMENT,
`Unit` VARCHAR(45) NOT NULL,
PRIMARY KEY (`UnitID`),
UNIQUE INDEX `Unit_UNIQUE` (`Unit` ASC));
INSERT INTO product.unit(Unit) VALUES('Box');
INSERT INTO product.unit(Unit) VALUES('Bag');
INSERT INTO product.unit(Unit) VALUES('Carton');
INSERT INTO product.unit(Unit) VALUES('Jar');
INSERT INTO product.unit(Unit) VALUES('Tube');
INSERT INTO product.unit(Unit) VALUES('Case');
INSERT INTO product.unit(Unit) VALUES('Cup');
CREATE TABLE `product`.`product` (
`ProductID` int(11) NOT NULL AUTO_INCREMENT,
`SKU` varchar(45) NOT NULL,
`Description` varchar(45) NOT NULL,
`ManufacturerID` int(11) NOT NULL,
`UnitCost` decimal(10,3) DEFAULT '0.000',
`UnitPrice` decimal(10,3) DEFAULT '0.000',
`UnitID` int(11) DEFAULT NULL,
PRIMARY KEY (`ProductID`),
UNIQUE KEY `SKU_UNIQUE` (`SKU`),
UNIQUE KEY `Description_UNIQUE` (`Description`),
KEY `ManufacturerID_idx` (`ManufacturerID`),
KEY `UnitID_idx` (`UnitID`),
CONSTRAINT `ManufacturerID` FOREIGN KEY (`ManufacturerID`) REFERENCES `manufacturer` (`ManufacturerID`) ON DELETE NO ACTION ON UPDATE NO ACTION,
CONSTRAINT `UnitID` FOREIGN KEY (`UnitID`) REFERENCES `unit` (`UnitID`) ON DELETE NO ACTION ON UPDATE NO ACTION
);
INSERT INTO product.product (SKU, Description, ManufacturerID, UnitCost, UnitPrice, UnitID) VALUES('P000000001', 'Blueberry Jam', 1, .50, 1.00, 4);
INSERT INTO product.product (SKU, Description, ManufacturerID, UnitCost, UnitPrice, UnitID) VALUES('P000000002', 'Coffee Yogurt', 2, 1.00, 2.00, 7);
INSERT INTO product.product (SKU, Description, ManufacturerID, UnitCost, UnitPrice, UnitID) VALUES('P000000003', '12 Individual Boxes Corn Flakes', 3, 1.00, 5.00, 6);
INSERT INTO product.product (SKU, Description, ManufacturerID, UnitCost, UnitPrice, UnitID) VALUES('P000000004', 'Box Corn Flakes', 3, 1.00, 5.00, 1);
END;
-- *********************
-- Loyalty (Customers)
-- *********************
BEGIN
DROP schema if exists `loyalty`;
END;
BEGIN
CREATE SCHEMA `loyalty`;
CREATE TABLE `loyalty`.`loyalty` (
`LoyaltyID` INT NOT NULL AUTO_INCREMENT,
`LoyaltyNumber` VARCHAR(10) NOT NULL,
`LastName` VARCHAR(45) NULL,
`FirstName` VARCHAR(45) NULL,
`AddressLine1` VARCHAR(45) NULL,
`AddressLine2` VARCHAR(45) NULL,
`City` VARCHAR(45) NULL,
`State` VARCHAR(2) NULL,
`ZipCode` INT NULL,
PRIMARY KEY (`LoyaltyID`),
UNIQUE INDEX `LoyaltyNumber_UNIQUE` (`LoyaltyNumber` ASC));
INSERT INTO loyalty.loyalty (LoyaltyID, LoyaltyNumber, LastName) VALUES(1, 'L000000001', 'Smith');
INSERT INTO loyalty.loyalty (LoyaltyID, LoyaltyNumber) VALUES(2, 'L000000002');
INSERT INTO loyalty.loyalty (LoyaltyID, LoyaltyNumber) VALUES(3, 'L000000003');
INSERT INTO loyalty.loyalty (LoyaltyID, LoyaltyNumber, LastName, FirstName) VALUES(4, 'L000000004', 'Knight', 'Robert');
END;
-- *********************
-- Reconciled Schema
-- *********************
BEGIN
DROP Schema if exists `reconciled`;
END;
BEGIN
CREATE SCHEMA `reconciled` ;
CREATE TABLE `reconciled`.`sale` (
`saleID` int(11) NOT NULL AUTO_INCREMENT,
`EmployeeFirstName` varchar(45) DEFAULT NULL,
`EmployeeLastName` varchar(45) DEFAULT NULL,
`ProductDescription` varchar(45) DEFAULT NULL,
`Unit` varchar(45) DEFAULT NULL,
`SKU` varchar(45) DEFAULT NULL,
`Qty` int(11) DEFAULT NULL,
`UnitCost` decimal(10,3) DEFAULT NULL,
`UnitPrice` decimal(10,3) DEFAULT NULL,
`Manufacturer` varchar(45) DEFAULT NULL,
`EmployeeNumber` varchar(45) DEFAULT NULL COMMENT 'The employee who handled the transaction',
`LoyaltyNumber` varchar(45) DEFAULT NULL,
`StoreNumber` varchar(45) DEFAULT NULL,
`DateOfTransaction` date DEFAULT NULL,
`TimeOfTransaction` time DEFAULT NULL,
`DateOfTransactionString` varchar(45) DEFAULT NULL,
`TimeOfTransactionString` varchar(45) DEFAULT NULL,
`WeekdayOfTransaction` int(11) DEFAULT NULL,
`MonthOfTransaction` int(11) DEFAULT NULL,
`YearOfTransaction` int(11) DEFAULT NULL,
`WeekdayNameOfTransaction` varchar(45) DEFAULT NULL,
`MonthNameOfTransaction` varchar(45) DEFAULT NULL,
`TotalPrice` decimal(10,3) DEFAULT NULL,
`ZipCode` varchar(10) DEFAULT NULL,
PRIMARY KEY (`saleID`)
) ENGINE=InnoDB AUTO_INCREMENT=7 DEFAULT CHARSET=utf8;
CREATE TABLE `reconciled`.`weather` (
`WeatherID` int(11) NOT NULL AUTO_INCREMENT,
`DateStamp` date DEFAULT NULL,
`ZipCode` varchar(10) DEFAULT NULL,
`Temperature` int(11) DEFAULT NULL,
`MonthName` varchar(45) DEFAULT NULL,
`YearNumber` int(11) DEFAULT NULL,
`DayName` varchar(45) DEFAULT NULL,
`MonthNumber` int(11) DEFAULT NULL,
`DayNumber` int(11) DEFAULT NULL,
PRIMARY KEY (`WeatherID`)
) ENGINE=InnoDB AUTO_INCREMENT=17 DEFAULT CHARSET=utf8;
END;
-- *********************
-- Aggregated schema for the data warehouse
-- *********************
BEGIN
DROP Schema if exists dw;
CREATE SCHEMA `dw` ;
CREATE TABLE `dw`.`salesqtybyproductandstore` (
`SalesByProductAndStoreID` int(11) NOT NULL AUTO_INCREMENT,
`SKU` varchar(45) NOT NULL,
`StoreNumber` varchar(45) NOT NULL,
`ProductDescription` varchar(45) DEFAULT NULL,
`Monday` int(11) DEFAULT '0',
`Tuesday` int(11) DEFAULT '0',
`Wednesday` int(11) DEFAULT '0',
`Thursday` int(11) DEFAULT '0',
`Friday` int(11) DEFAULT '0',
`Saturday` int(11) DEFAULT '0',
`Sunday` int(11) DEFAULT '0',
PRIMARY KEY (`SalesByProductAndStoreID`),
UNIQUE KEY `SKU_UNIQUE` (`SKU`,`StoreNumber`)
);
END;
BEGIN
CREATE TABLE `dw`.`monthlysalesqtybyproductandstore` (
`monthlysalesqtybyproductandstoreID` int(11) NOT NULL AUTO_INCREMENT,
`SKU` varchar(45) NOT NULL,
`StoreNumber` varchar(45) NOT NULL,
`ProductDescription` varchar(45) DEFAULT NULL,
`January` int(11) DEFAULT '0',
`February` int(11) DEFAULT '0',
`March` int(11) DEFAULT '0',
`April` int(11) DEFAULT '0',
`May` int(11) DEFAULT '0',
`June` int(11) DEFAULT '0',
`July` int(11) DEFAULT '0',
`August` int(11) DEFAULT '0',
`September` int(11) DEFAULT '0',
`October` int(11) DEFAULT '0',
`November` int(11) DEFAULT '0',
`December` int(11) DEFAULT '0',
PRIMARY KEY (`monthlysalesqtybyproductandstoreID`),
UNIQUE KEY `SKU_UNIQUE` (`SKU`,`StoreNumber`)
) ENGINE=InnoDB AUTO_INCREMENT=18 DEFAULT CHARSET=utf8;
CREATE TABLE `dw`.`weeklysalesqtybyproductandstore` (
`WeeklysalesqtybyproductandstoreID` int(11) NOT NULL AUTO_INCREMENT,
`SKU` varchar(45) NOT NULL,
`StoreNumber` varchar(45) NOT NULL,
`ProductDescription` varchar(45) DEFAULT NULL,
`Monday` int(11) DEFAULT '0',
`Tuesday` int(11) DEFAULT '0',
`Wednesday` int(11) DEFAULT '0',
`Thursday` int(11) DEFAULT '0',
`Friday` int(11) DEFAULT '0',
`Saturday` int(11) DEFAULT '0',
`Sunday` int(11) DEFAULT '0',
PRIMARY KEY (`WeeklysalesqtybyproductandstoreID`),
UNIQUE KEY `SKU_UNIQUE` (`SKU`,`StoreNumber`)
) ENGINE=InnoDB AUTO_INCREMENT=18 DEFAULT CHARSET=utf8;
CREATE TABLE `dw`.`storesalesbytemperature` (
`StoreSalesByTemperatureID` INT NOT NULL AUTO_INCREMENT,
`StoreNumber` VARCHAR(45) NULL,
`Under0` DECIMAL(10,2) NULL,
`0To20` DECIMAL(10,2) NULL,
`21To30` DECIMAL(10,2) NULL,
`31To40` DECIMAL(10,2) NULL,
`41To50` DECIMAL(10,2) NULL,
`51To60` DECIMAL(10,2) NULL,
`61To70` DECIMAL(10,2) NULL,
`71To80` DECIMAL(10,2) NULL,
`81To90` DECIMAL(10,2) NULL,
`Over90` DECIMAL(10,2) NULL,
`StoreSalesByTemperaturecol` VARCHAR(45) NULL,
PRIMARY KEY (`StoreSalesByTemperatureID`),
INDEX `Unique` (`StoreNumber` ASC));
END;
BEGIN
DROP Schema if exists icecream;
CREATE SCHEMA `icecream`;
CREATE TABLE `icecream`.`flavor` (
`FlavorID` INT NOT NULL AUTO_INCREMENT,
`Flavor` VARCHAR(45) NULL,
PRIMARY KEY (`FlavorID`),
UNIQUE INDEX `Flavor_UNIQUE` (`Flavor` ASC));
CREATE TABLE `icecream`.`container` (
`ContainerID` INT NOT NULL AUTO_INCREMENT,
`Container` VARCHAR(45) NULL,
PRIMARY KEY (`ContainerID`),
UNIQUE INDEX `Container_UNIQUE` (`Container` ASC));
CREATE TABLE `icecream`.`icecream` (
`IceCreamID` int(11) NOT NULL AUTO_INCREMENT,
`FlavorID` int(11) DEFAULT NULL,
`ContainerID` int(11) DEFAULT NULL,
`SKU` varchar(45) DEFAULT NULL,
PRIMARY KEY (`IceCreamID`),
UNIQUE KEY `Unique` (`ContainerID`,`FlavorID`),
UNIQUE KEY `SKU_UNIQUE` (`SKU`),
CONSTRAINT `Container` FOREIGN KEY (`ContainerID`) REFERENCES `icecream`.`container` (`ContainerID`) ON DELETE NO ACTION ON UPDATE NO ACTION,
CONSTRAINT `Flavor` FOREIGN KEY (`FlavorID`) REFERENCES `icecream`.`flavor` (`FlavorID`) ON DELETE NO ACTION ON UPDATE NO ACTION
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
INSERT INTO icecream.flavor (flavor) VALUES('Chocolate');
INSERT INTO icecream.flavor (flavor) VALUES('Vanilla');
INSERT INTO icecream.flavor (flavor) VALUES('Strawberry');
INSERT INTO icecream.flavor (flavor) VALUES('Raspberry Chocolate Chip');
INSERT INTO icecream.flavor (flavor) VALUES('Blueberry');
INSERT INTO icecream.container (Container) VALUES('cup');
INSERT INTO icecream.container (Container) VALUES('pint');
INSERT INTO icecream.container (Container) VALUES('quart');
INSERT INTO icecream.container (Container) VALUES('half gallon');
INSERT INTO icecream.container (Container) VALUES('gallon');
INSERT INTO icecream.icecream (flavorID, containerID, SKU) VALUES(1, 2, 'I000000001');
INSERT INTO icecream.icecream (flavorID, containerID, SKU) VALUES(2, 2, 'I000000002');
INSERT INTO icecream.icecream (flavorID, containerID, SKU) VALUES(3, 2, 'I000000003');
INSERT INTO icecream.icecream (flavorID, containerID, SKU) VALUES(1, 3, 'I000000004');
INSERT INTO icecream.icecream (flavorID, containerID, SKU) VALUES(2, 3, 'I000000005');
INSERT INTO icecream.icecream (flavorID, containerID, SKU) VALUES(3, 3, 'I000000006');
INSERT INTO icecream.icecream (flavorID, containerID) VALUES(4, 1);
END;
-- *********************
-- Schema Sales
-- *********************
BEGIN
DROP Schema if exists `weather`;
END;
BEGIN
CREATE SCHEMA `weather` ;
CREATE TABLE `weather`.`weather` (
`weatherText` varchar(100) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
INSERT INTO weather.weather (weatherText) VALUES('2018/01/01 00:00:00.0000,45255,10');
INSERT INTO weather.weather (weatherText) VALUES('2018/01/02 00:00:00.0000,45255,12');
INSERT INTO weather.weather (weatherText) VALUES('2018/01/03 00:00:00.0000,45255,14');
INSERT INTO weather.weather (weatherText) VALUES('2018/01/04 00:00:00.0000,45255,16');
INSERT INTO weather.weather (weatherText) VALUES('2018/01/01 00:00:00.0000,45103,15');
INSERT INTO weather.weather (weatherText) VALUES('2018/01/02 00:00:00.0000,45103,17');
INSERT INTO weather.weather (weatherText) VALUES('2018/01/03 00:00:00.0000,45103,19');
INSERT INTO weather.weather (weatherText) VALUES('2018/01/04 00:00:00.0000,45103,21');
END;
-- *********************
-- Schema Sales
-- *********************
BEGIN
DROP Schema if exists `sales`;
END;
BEGIN
CREATE SCHEMA `sales` ;
CREATE TABLE `sales`.`transaction` (
`TransactionID` int(11) NOT NULL AUTO_INCREMENT,
`DateTimeOfTransaction` datetime NOT NULL,
`LoyaltyNumber` varchar(10) DEFAULT NULL,
`StoreNumber` varchar(10) NOT NULL,
`EmployeeNumber` varchar(10) NOT NULL,
PRIMARY KEY (`TransactionID`),
UNIQUE KEY `NaturalKey` (`DateTimeOfTransaction`,`LoyaltyNumber`,`StoreNumber`,`EmployeeNumber`)
) ENGINE=InnoDB AUTO_INCREMENT=5 DEFAULT CHARSET=utf8;
END;
BEGIN
INSERT INTO `sales`.`transaction` (TransactionID, DateTimeOfTransaction, LoyaltyNumber, StoreNumber, EmployeeNumber) VALUES(1, '2018-01-01 15:00:00', 'L000000001', 'S000000001', 'E000000001');
INSERT INTO `sales`.`transaction` (TransactionID, DateTimeOfTransaction, LoyaltyNumber, StoreNumber, EmployeeNumber) VALUES(2, '2018-02-02 15:00:00', 'L000000002', 'S000000002', 'E000000002');
INSERT INTO `sales`.`transaction` (TransactionID, DateTimeOfTransaction, LoyaltyNumber, StoreNumber, EmployeeNumber) VALUES(3, '2018-02-07 16:00:00', 'L000000002', 'S000000002', 'E000000002');
INSERT INTO `sales`.`transaction` (TransactionID, DateTimeOfTransaction, LoyaltyNumber, StoreNumber, EmployeeNumber) VALUES(4, '2018-02-09 16:16:00', 'L000000002', 'S000000002', 'E000000002');
END;
BEGIN
-- YYYY-MM-DD HH:MM:SS
CREATE TABLE `sales`.`transactiondetail` (
`TransactionDetailID` int(11) NOT NULL AUTO_INCREMENT,
`TransactionID` int(11) NOT NULL,
`SKU` varchar(45) NOT NULL,
`Qty` int(11) NOT NULL,
`TotalPrice` decimal(10,3) NOT NULL,
`Comment` varchar(1000) DEFAULT NULL,
PRIMARY KEY (`TransactionDetailID`),
UNIQUE KEY `NaturalKey` (`TransactionID`,`SKU`),
KEY `TransactionID_idx` (`TransactionID`),
CONSTRAINT `TransactionID` FOREIGN KEY (`TransactionID`) REFERENCES `sales`.`transaction` (`TransactionID`) ON DELETE NO ACTION ON UPDATE NO ACTION);
END;
BEGIN
INSERT INTO sales.transactiondetail (TransactionID, SKU, Qty, TotalPrice) VALUES(1, 'P000000001', 10, 100);
INSERT INTO sales.transactiondetail (TransactionID, SKU, Qty, TotalPrice) VALUES(1, 'P000000002', 5, 25);
INSERT INTO sales.transactiondetail (TransactionID, SKU, Qty, TotalPrice) VALUES(2, 'P000000001', 1, 100);
INSERT INTO sales.transactiondetail (TransactionID, SKU, Qty, TotalPrice) VALUES(2, 'P000000002', 2, 25);
INSERT INTO sales.transactiondetail (TransactionID, SKU, Qty, TotalPrice) VALUES(3, 'P000000002', 40, 40);
INSERT INTO sales.transactiondetail (TransactionID, SKU, Qty, TotalPrice) VALUES(4, 'P000000002', 44, 44);
END;
BEGIN
CREATE
ALGORITHM = UNDEFINED
DEFINER = `root`#`localhost`
SQL SECURITY DEFINER
VIEW `reconciled`.`mapping` AS
SELECT
`hr`.`employee`.`FirstName` AS `EmployeeFirstName`,
`hr`.`employee`.`LastName` AS `EmployeeLastName`,
`product`.`product`.`Description` AS `ProductDescription`,
`product`.`unit`.`Unit` AS `Unit`,
`product`.`product`.`SKU` AS `SKU`,
`sales`.`transactiondetail`.`Qty` AS `Qty`,
`product`.`product`.`UnitCost` AS `UnitCost`,
`product`.`product`.`UnitPrice` AS `UnitPrice`,
`product`.`manufacturer`.`Manufacturer` AS `Manufacturer`,
`sales`.`transaction`.`EmployeeNumber` AS `EmployeeNumber`,
`sales`.`transaction`.`LoyaltyNumber` AS `LoyaltyNumber`,
`sales`.`transaction`.`StoreNumber` AS `StoreNumber`,
`store`.`store`.`zipCode` as `ZipCode`,
CAST(`sales`.`transaction`.`DateTimeOfTransaction`
AS DATE) AS `DateOfTransaction`,
CAST(`sales`.`transaction`.`DateTimeOfTransaction`
AS TIME) AS `TimeOfTransaction`,
CAST(CAST(`sales`.`transaction`.`DateTimeOfTransaction`
AS DATE)
AS CHAR CHARSET UTF8) AS `DateOfTransactionString`,
CAST(CAST(`sales`.`transaction`.`DateTimeOfTransaction`
AS TIME)
AS CHAR CHARSET UTF8) AS `TimeOfTransactionString`,
WEEKDAY(`sales`.`transaction`.`DateTimeOfTransaction`) AS `WeekdayOfTransaction`,
MONTH(`sales`.`transaction`.`DateTimeOfTransaction`) AS `MonthOfTransaction`,
YEAR(`sales`.`transaction`.`DateTimeOfTransaction`) AS `YearOfTransaction`,
DAYNAME(`sales`.`transaction`.`DateTimeOfTransaction`) AS `WeekdayNameOfTransaction`,
MONTHNAME(`sales`.`transaction`.`DateTimeOfTransaction`) AS `MonthNameOfTransaction`,
`sales`.`transactiondetail`.`TotalPrice` AS `TotalPrice`
FROM
((((((`sales`.`transaction`
JOIN `sales`.`transactiondetail` ON ((`sales`.`transaction`.`TransactionID` = `sales`.`transactiondetail`.`TransactionID`)))
JOIN `hr`.`employee` ON ((`hr`.`employee`.`EmployeeNumber` = `sales`.`transaction`.`EmployeeNumber`)))
JOIN `product`.`product` ON ((`product`.`product`.`SKU` = `sales`.`transactiondetail`.`SKU`)))
JOIN `product`.`unit` ON ((`product`.`product`.`UnitID` = `product`.`unit`.`UnitID`)))
JOIN `store`.`store` ON ((`store`.`store`.`StoreNumber` = `sales`.`transaction`.`StoreNumber`)))
JOIN `product`.`manufacturer` ON ((`product`.`product`.`ManufacturerID` = `product`.`manufacturer`.`ManufacturerID`))) ;
CREATE TABLE `reconciled`.`icecream` (
`icecreamid` int(11) NOT NULL AUTO_INCREMENT,
`flavor` varchar(45) DEFAULT NULL,
`container` varchar(45) DEFAULT NULL,
`sku` varchar(45) DEFAULT NULL,
PRIMARY KEY (`icecreamid`),
UNIQUE KEY `Unique` (`flavor`,`container`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
END;
BEGIN
CREATE VIEW `reconciled`.`totalSalesByStoreAndTemperature` AS
SELECT StoreNumber, temperature, SUM(TotalPrice) as `SumOfTotalPrice`
FROM reconciled.sale inner join reconciled.weather on Substring(reconciled.sale.zipCode,1,5) = substring(reconciled.weather.zipCode,1,5)
Group By StoreNumber, temperature
ORDER BY StoreNumber, Temperature;
END;
END
Just want to add the root cause of this issue.
Cloud SQL currently is limited to grant SUPER and FILE privileges according to GCP documentation, so they can't be assigned to any user including root.
As per MySQL 5.7 documentation, the DEFINER attribute requires the user to have the SUPER privilege.
The situation above was the cause of your error and it worked when you took out the DEFINER statement.
In summary any other action that requires SUPER o FILE privileges most probably will fail with the error "Access denied; you need (at least one of) the SUPER privilege(s) for this operation".
I took out the DEFINER clause in the creation of the view reconciled.mapping.
That eliminated the error when I executed the SP.
It wasn't the DEFINER clause of the SP creation script, it was the DEFINER clause in the SP itself.
with my experience in AWS RDS, you don't get SUPER privileges as those are used by the provisioner engine. I am pretty sure that is how it is done in GCP. It could be that something in query make MySQL think it needs SUPER privileges. Posting your query which is giving the error will be helpful.
-------Update after OP posted the query----------
this is the problem. DEFINER. try CREATE PROCEDURE CreateFullCoverageTestCase()