Memsql TPCH queries - mysql

Question

I am trying TPCH DDL queries on memsql. I am new to memsql. I am not able to convert 5 TPCH ddl sql queries to memsql queries. Not able to achieve foreign key relationship using memsql's FOREIGH SHARD KEY concept. Please help me to covert below 5 out of 8 table creation queries into memsql queries. Tried hard but face lot of different issues.
CREATE TABLE REGION ( R_REGIONKEY INTEGER NOT NULL PRIMARY KEY,
R_NAME CHAR(25) NOT NULL,
R_COMMENT VARCHAR(152)
);
CREATE TABLE NATION ( N_NATIONKEY INTEGER NOT NULL PRIMARY KEY,
N_NAME CHAR(25) NOT NULL,
N_REGIONKEY INTEGER NOT NULL REFERENCES REGION(R_REGIONKEY),
N_COMMENT VARCHAR(152)
);
CREATE TABLE PART ( P_PARTKEY INTEGER NOT NULL PRIMARY KEY,
P_NAME VARCHAR(55) NOT NULL,
P_MFGR CHAR(25) NOT NULL,
P_BRAND CHAR(10) NOT NULL,
P_TYPE VARCHAR(25) NOT NULL,
P_SIZE INTEGER NOT NULL,
P_CONTAINER CHAR(10) NOT NULL,
P_RETAILPRICE DECIMAL(15,2) NOT NULL,
P_COMMENT VARCHAR(23) NOT NULL
);
CREATE TABLE SUPPLIER ( S_SUPPKEY INTEGER NOT NULL PRIMARY KEY,
S_NAME CHAR(25) NOT NULL,
S_ADDRESS VARCHAR(40) NOT NULL,
S_NATIONKEY INTEGER NOT NULL REFERENCES NATION(N_NATIONKEY),
S_PHONE CHAR(15) NOT NULL,
S_ACCTBAL DECIMAL(15,2) NOT NULL,
S_COMMENT VARCHAR(101) NOT NULL
);
CREATE TABLE PARTSUPP ( PS_PARTKEY INTEGER NOT NULL REFERENCES PART(P_PARTKEY),
PS_SUPPKEY INTEGER NOT NULL REFERENCES SUPPLIER(S_SUPPKEY),
PS_AVAILQTY INTEGER NOT NULL,
PS_SUPPLYCOST DECIMAL(15,2) NOT NULL,
PS_COMMENT VARCHAR(199) NOT NULL,
PRIMARY KEY (PS_PARTKEY, PS_SUPPKEY)
);
CREATE TABLE CUSTOMER ( C_CUSTKEY INTEGER NOT NULL PRIMARY KEY,
C_NAME VARCHAR(25) NOT NULL,
C_ADDRESS VARCHAR(40) NOT NULL,
C_NATIONKEY INTEGER NOT NULL REFERENCES NATION(N_NATIONKEY),
C_PHONE CHAR(15) NOT NULL,
C_ACCTBAL DECIMAL(15,2) NOT NULL,
C_MKTSEGMENT CHAR(10) NOT NULL,
C_COMMENT VARCHAR(117) NOT NULL
);
CREATE TABLE ORDERS ( O_ORDERKEY INTEGER NOT NULL PRIMARY KEY,
O_CUSTKEY INTEGER NOT NULL REFERENCES CUSTOMER(C_CUSTKEY),
O_ORDERSTATUS CHAR(1) NOT NULL,
O_TOTALPRICE DECIMAL(15,2) NOT NULL,
O_ORDERDATE DATE NOT NULL,
O_ORDERPRIORITY CHAR(15) NOT NULL,
O_CLERK CHAR(15) NOT NULL,
O_SHIPPRIORITY INTEGER NOT NULL,
O_COMMENT VARCHAR(79) NOT NULL
);
CREATE TABLE LINEITEM ( L_ORDERKEY INTEGER NOT NULL REFERENCES ORDERS(O_ORDERKEY),
L_PARTKEY INTEGER NOT NULL REFERENCES PART(P_PARTKEY),
L_SUPPKEY INTEGER NOT NULL REFERENCES SUPPLIER(S_SUPPKEY),
L_LINENUMBER INTEGER NOT NULL,
L_QUANTITY DECIMAL(15,2) NOT NULL,
L_EXTENDEDPRICE DECIMAL(15,2) NOT NULL,
L_DISCOUNT DECIMAL(15,2) NOT NULL,
L_TAX DECIMAL(15,2) NOT NULL,
L_RETURNFLAG CHAR(1) NOT NULL,
L_LINESTATUS CHAR(1) NOT NULL,
L_SHIPDATE DATE NOT NULL,
L_COMMITDATE DATE NOT NULL,
L_RECEIPTDATE DATE NOT NULL,
L_SHIPINSTRUCT CHAR(25) NOT NULL,
L_SHIPMODE CHAR(10) NOT NULL,
L_COMMENT VARCHAR(44) NOT NULL,
PRIMARY KEY (L_ORDERKEY,L_LINENUMBER),
FOREIGN KEY (L_PARTKEY,L_SUPPKEY) REFERENCES PARTSUPP(PS_PARTKEY, PS_SUPPKEY)
);
I am able to create first 3 tables in memsql but not able to remaining tables.1st and 3rd queries are very simple and worked as it is. I am able to create 2nd table but again not sure whether this is right way to achieve.
CREATE TABLE NATION ( N_NATIONKEY INTEGER NOT NULL,
N_NAME CHAR(25) NOT NULL,
N_REGIONKEY INTEGER NOT NULL,
N_COMMENT VARCHAR(152),
FOREIGN SHARD KEY (N_REGIONKEY) REFERENCES REGION (R_REGIONKEY),
PRIMARY KEY (N_NATIONKEY, N_REGIONKEY)
);
Is it possible to create only Replicate table and not partition in memsql? and how?

Answer 1

Since MemSQL does not support referential integrity, foreign shard keys are an optimization aid and not necessary. Foreign shard keys though do allow you to know at table creation time that two tables may be joined locally (no network traffic) on that key. However, the optimizer does not require foreign shard keys to take advantage of this data locality.
Starting with the ORDERS and LINEITEM tables:
CREATE TABLE ORDERS ( O_ORDERKEY INTEGER NOT NULL PRIMARY KEY,
O_CUSTKEY INTEGER NOT NULL,
O_ORDERSTATUS CHAR(1) NOT NULL,
O_TOTALPRICE DECIMAL(15,2) NOT NULL,
O_ORDERDATE DATE NOT NULL,
O_ORDERPRIORITY CHAR(15) NOT NULL,
O_CLERK CHAR(15) NOT NULL,
O_SHIPPRIORITY INTEGER NOT NULL,
O_COMMENT VARCHAR(79) NOT NULL,
KEY (O_CUSTKEY)
);
CREATE TABLE LINEITEM ( L_ORDERKEY INTEGER NOT NULL,
L_PARTKEY INTEGER NOT NULL,
L_SUPPKEY INTEGER NOT NULL,
L_LINENUMBER INTEGER NOT NULL,
L_QUANTITY DECIMAL(15,2) NOT NULL,
L_EXTENDEDPRICE DECIMAL(15,2) NOT NULL,
L_DISCOUNT DECIMAL(15,2) NOT NULL,
L_TAX DECIMAL(15,2) NOT NULL,
L_RETURNFLAG CHAR(1) NOT NULL,
L_LINESTATUS CHAR(1) NOT NULL,
L_SHIPDATE DATE NOT NULL,
L_COMMITDATE DATE NOT NULL,
L_RECEIPTDATE DATE NOT NULL,
L_SHIPINSTRUCT CHAR(25) NOT NULL,
L_SHIPMODE CHAR(10) NOT NULL,
L_COMMENT VARCHAR(44) NOT NULL,
PRIMARY KEY (L_ORDERKEY,L_LINENUMBER),
FOREIGN SHARD KEY (L_ORDERKEY) REFERENCES ORDERS (O_ORDERKEY),
KEY (L_PARTKEY),
KEY (L_SUPPKEY)
);
In this case we know we can take advantage of a local join between ORDERS and LINEITEM, because they are both sharded on ORDERKEY. ORDERS and LINEITEM are the two biggest tables in TPCH, so we want to ensure that they can be joined locally. Since ORDERS' primary key is O_ORDERKEY, I do not need to specify a shard key for ORDERS. MemSQL will shard by O_ORDERKEY automatically.
I've also put secondary indexes on the remaining foreign key columns. This is useful since there will be joins on the foreign keys.
Applying these concepts to PART, PARTSUPP, SUPPLIER, and CUSTOMER:
CREATE TABLE CUSTOMER ( C_CUSTKEY INTEGER NOT NULL PRIMARY KEY,
C_NAME VARCHAR(25) NOT NULL,
C_ADDRESS VARCHAR(40) NOT NULL,
C_NATIONKEY INTEGER NOT NULL,
C_PHONE CHAR(15) NOT NULL,
C_ACCTBAL DECIMAL(15,2) NOT NULL,
C_MKTSEGMENT CHAR(10) NOT NULL,
C_COMMENT VARCHAR(117) NOT NULL,
KEY(C_NATIONKEY)
);
CREATE TABLE SUPPLIER ( S_SUPPKEY INTEGER NOT NULL PRIMARY KEY,
S_NAME CHAR(25) NOT NULL,
S_ADDRESS VARCHAR(40) NOT NULL,
S_NATIONKEY INTEGER NOT NULL,
S_PHONE CHAR(15) NOT NULL,
S_ACCTBAL DECIMAL(15,2) NOT NULL,
S_COMMENT VARCHAR(101) NOT NULL,
KEY(S_NATIONKEY)
);
CREATE TABLE PART ( P_PARTKEY INTEGER NOT NULL PRIMARY KEY,
P_NAME VARCHAR(55) NOT NULL,
P_MFGR CHAR(25) NOT NULL,
P_BRAND CHAR(10) NOT NULL,
P_TYPE VARCHAR(25) NOT NULL,
P_SIZE INTEGER NOT NULL,
P_CONTAINER CHAR(10) NOT NULL,
P_RETAILPRICE DECIMAL(15,2) NOT NULL,
P_COMMENT VARCHAR(23) NOT NULL
);
CREATE TABLE PARTSUPP ( PS_PARTKEY INTEGER NOT NULL,
PS_SUPPKEY INTEGER NOT NULL,
PS_AVAILQTY INTEGER NOT NULL,
PS_SUPPLYCOST DECIMAL(15,2) NOT NULL,
PS_COMMENT VARCHAR(199) NOT NULL,
PRIMARY KEY (PS_PARTKEY, PS_SUPPKEY),
SHARD KEY(PS_PARTKEY),
KEY(PS_SUPPKEY)
);
Although PARTSUPP and PART are both sharded on the same key (PARTKEY), I do not need to specify a foreign shard key in order to take advantage of a local join between them on that key; the optimizer will pick that up automatically.
In response to your final question, MemSQL does allow you create a replicated table instead of a partitioned table. This is called a reference table and will be useful for the NATION and REGION tables since they are very small. Reference tables are not necessary to run distributed queries, but are a useful optimization.
CREATE REFERENCE TABLE REGION ( R_REGIONKEY INTEGER NOT NULL PRIMARY KEY,
R_NAME CHAR(25) NOT NULL,
R_COMMENT VARCHAR(152)
);
CREATE REFERENCE TABLE NATION ( N_NATIONKEY INTEGER NOT NULL PRIMARY KEY,
N_NAME CHAR(25) NOT NULL,
N_REGIONKEY INTEGER NOT NULL,
N_COMMENT VARCHAR(152)
);
For more documentation on everything described, check out:
http://docs.memsql.com/latest/concepts/distributed_sql/