AWS Aurora DB Combine Data from Multiple SQL Statements - mysql

I have three tables with data schema, like:
TABLE user (
user_id BINARY(16) PRIMARY KEY NOT NULL,
created DATETIME NOT NULL,
last_updated DATETIME,
coordinator BINARY(16),
num_updates INT NOT NULL
);
TABLE summary (
user_id BINARY(16) PRIMARY KEY NOT NULL,
calculation_time DATETIME NOT NULL,
calculation_method VARCHAR(25) NOT NULL,
label VARCHAR(50) NOT NULL,
critical_count INT NOT NULL,
median_risk FLOAT(10)
);
TABLE actions(
user_id BINARY(16) PRIMARY KEY NOT NULL,
label VARCHAR(50) NOT NULL,
access_count INT NOT NULL,
median FLOAT(10)
);
The data for all the users (user table) is simply fetched using the lambda handler function in the following manner:
const AWS = require('aws-sdk');
const rdsDataService = new AWS.RDSDataService();
module.exports.hello = async (event, context, callback) => {
const req_id = "5a9dbfca-74d6-471a-af27-31beb4b53bb2";
const sql = 'SELECT * FROM user WHERE user_id=:id';
try {
const params = {
resourceArn: 'arn:aws:rds:us-west-********************',
secretArn: 'arn:aws:secretsmanager:us-west**************',
sql,
database: 'dev_db1',
continueAfterTimeout: true,
includeResultMetadata: true,
parameters: [{ 'name': 'id', 'value': { 'stringValue': `${req_id}` } }]
}
const db_res = await rdsDataService.executeStatement(params).promise();
const convertToJson = (dbresponse) => {
const columns = dbresponse.columnMetadata.map(col => col.name);
const row_data = dbresponse.records.map(row => {
const json_obj = {};
row.map((val, i) => {
json_obj[columns[i]] = Object.values(val)[0];
});
return json_obj;
});
return row_data;
};
const modified_data = convertToJson(db_res);
const response = {
body: {
statusCode: 200,
message: 'Data fetched successfully',
data: modified_data,
}
};
callback(null, response);
} catch (error) {
console.log('Error Received', error);
const error_res = {
body: {
statusCode: error.statusCode,
message: error.message,
data: null
}
}
callback(null, error_res);
}
};
If the same is followed for another table summary or actions, it also works. Now, I need to combine all the columns of these three tables and then return the data (returned rows should match on the basis of req_id).
My working snippet: https://dbfiddle.uk/?rdbms=mysql_5.7&fiddle=016ecc94c792611fbaca810605e81a6a
But the final result obtained contains the column user_id in duplicated form i.e. three times inclusion. I don't need the same column to be repeated thrice.
I am a bit new to handling MySQL queries, so unable to figure out the exact reason for the error even when the table exists. The MYSQL version used in Aurora is 5.7.
Any help to resolve the same is appreciated!

Plan A: Explicitly specify the columns you want. Extra benefit: You can get rid of the ids, which tend to be useless to others reading the output.
Plan B: (This option is not always possible.) Instead of JOIN .. ON t1.a = t2.a, say JOIN .. USING(a)
I like to use short aliases. Here's doing all things together:
SELECT u.last_name, u.first_name,
s.risk_score,
t.likes
FROM user AS u
JOIN summary AS s USING(user_id)
LEFT JOIN test AS t USING(user_id)
In general, it is not wise to have a 1:1 relationship (as you have via user_id); you may as well have all the columns in a single table.

try this
SELECT users.*, summary.* from users, summary WHERE users.user_id = summary.user_id
OR
SELECT * from users, summary WHERE users.user_id = summary.user_id

Related

How can i fetch data from another table referenced by foreign key?

Heres my tables that a wanna fetch the data, i am using express to make a get request from the next app:
model Provider {
id String #id #default(cuid())
name String
email String
password String
phone String
photo String?
service Service? #relation(fields: [serviceId], references: [id])
serviceId String?
location Location? #relation(fields: [locationId], references: [id])
locationId String?
createdAt DateTime #default(now())
text String
starAverage Float
medals Medals[]
comment Comment[]
}
model Service {
id String #id #default(cuid())
type String
provider Provider[]
}
I wanna fetch the type of the service of the services table of a provider, not the serviceId, and this is my route.
router.get('/list', async (req: Request, res: Response) => {
const allClients = await prisma.client.findMany()
res.json({allClients})
})
this is how i am fetching the data of the rest API using axios
const [providers, setProviders] = useState([] as any[])
useEffect(() => {
axios.get('http://localhost:5140/providers/list')//my list of all providers
.then(res => {
console.log(res)
setProviders(res.data)
}).catch(err => {
console.log(err)
})
}, )
const renderedProviders = Object.values(providers).map(provider => {
return (
<div
className="card"
style={{ width: "18rem"}}
key={provider.id}
>
<img className="card-img-top" src="..."/>
<div className="card-body">
<h3>{provider.name}</h3>
<p>{provider.starAverage} estrekas</p>
<p>{provider.serviceId}</p>
</div>
</div>
);
});
return (
<div className="d-flex flex-row flex-wrap justify-content-between">
{renderedProviders}
</div>
)
for now a only get the serviceId of a provider, not the type of the service
To fetch data from another table referenced by a foreign key in a database, you can use a JOIN clause in your SQL query. A JOIN clause allows you to combine rows from two or more tables based on a related column between the tables.
this how you can use a JOIN clause to fetch data from two tables, users and orders, that are related by a foreign key.
SELECT users.*, orders.*
FROM users
JOIN orders ON orders.user_id = users.id
the JOIN clause combines rows from the users and orders tables based on the user_id column in the orders table and the id column in the users table. The SELECT clause specifies the columns to be retrieved from the users and orders tables.
Edited
how you can reference this in the express route and in the http request from axios ?
you can use the sequelize.query(Sequelize is a promise-based Node.js ORM) method to execute a raw SQL query.
app.get('/users/:id', (req, res) => {
const { id } = req.params;
const query = `
SELECT users.*, orders.*
FROM users
JOIN orders ON orders.user_id = users.id
WHERE users.id = :id
`;
const replacements = { id };
sequelize.query(query).then(([results, metadata]) => {
res.send(results);
});
});
the sequelize.query method is used to execute a raw SQL query that includes a JOIN clause to fetch data from the users and orders tables.

mysql2.createPool() & Managed Database Cluster Limits. Should I use mysql2.createConnection() instead?

I've decided to start using mysql2 library lately and created the following example to illustrate what currently works.
A MySQL tempt table is created, a series of queries are run in parallel and then it terminates.
I then found this article MySQL Limits on Digital Ocean.
Specifically the bullet point referring to:
Connection pooling is not supported for MySQL.
My question is, should I use mysql.createConnection() instead of mysql.createPool()? I ask because, both of them work just fine.
// deno run --allow-env --allow-net todo.ts
import * as mysql2 from "https://deno.land/x/mysql2/mod.ts";
const pool = mysql2.createPool({
host: "db.ondigitalocean.com",
port: 12345,
user: "user1",
password: "letmein",
database: "devdb",
connectionLimit: 10,
timezone: "+00:00" // Development
// timezone: "-06:00" // Production
});
/* CREATE temp table tblTodos in database memory */
const sqlCREATE1: string =
`CREATE TEMPORARY TABLE IF NOT EXISTS tblTodos (
id BIGINT UNSIGNED NOT NULL AUTO_INCREMENT,
userId BIGINT UNSIGNED DEFAULT NULL,
CHECK (userId >= 0),
todoCreated TIMESTAMP DEFAULT NULL,
todoModified TIMESTAMP DEFAULT NULL,
todoStatus VARCHAR(16) DEFAULT NULL,
todoTitle VARCHAR(128) DEFAULT NULL,
todoBody VARCHAR(1024) DEFAULT NULL,
PRIMARY KEY (id)
) ENGINE = INNODB
AUTO_INCREMENT=2001
DEFAULT CHARSET = utf8mb4
COLLATE = utf8mb4_0900_ai_ci;`;
/* SELECT 1 shows an empty table */
const sqlSELECT1: string =
`SELECT
*
FROM
tblTodos;`;
/* INSERT 1 adds a Pending todo record to the table */
const sqlINSERT1: string =
`INSERT INTO tblTodos (
id, userId,
todoCreated, todoModified, todoStatus,
todoTitle, todoBody
)
VALUES
(
NULL, 1001,
NOW(), NOW(), 'Pending',
'Shopping List #1', 'Apples & Oranges'
);`;
/* SELECT 2 shows the Pending todo record */
const sqlSELECT2: string =
`SELECT
*
FROM
tblTodos
WHERE
todoStatus = 'Pending';`;
/* UPDATE 1 changes todo status from Pending to Complete */
const sqlUPDATE1: string =
`UPDATE
tblTodos
SET
todoModified = NOW(),
todoStatus = 'Complete'
WHERE
id = 2001
AND userId = 1001;`;
/* SELECT 3 shows the Complete todo record */
const sqlSELECT3: string =
`SELECT
*
FROM
tblTodos
WHERE
todoStatus = 'Complete'
AND userId = 1001;`;
/* DELETE 1 deletes todo from table */
const sqlDELETE1: string =
`DELETE FROM
tblTodos
WHERE
id = 2001
AND userId = 1001;`;
/* SELECT 4 once again shows an empty table */
const sqlSELECT4: string =
`SELECT
*
FROM
tblTodos;`;
/* DROP 1 deletes table tblTodos from database */
const sqlDROP1: string =
`DROP
TEMPORARY TABLE IF EXISTS tblTodos;`;
const connection = await pool.getConnection();
let create1Result, select1Result, insert1Result, select2Result,
update1Result, select3Result, delete1Result, select4Result,
drop1Result;
try {
create1Result = await connection.execute(sqlCREATE1);
if (create1Result) console.log("Table tblToDos created.");
select1Result = await connection.execute(sqlSELECT1);
if (select1Result) console.log("Table tblToDos contains", select1Result[0].length, "records.");
insert1Result = await connection.execute(sqlINSERT1);
if (insert1Result) console.log(insert1Result[0].affectedRows, "record(s) inserted.", "id:", insert1Result[0].insertId);
select2Result = await connection.execute(sqlSELECT2);
if (select2Result) console.log(select2Result[0]);
update1Result = await connection.execute(sqlUPDATE1);
if (update1Result) console.log(update1Result[0].affectedRows, "record(s) updated.");
select3Result = await connection.execute(sqlSELECT3);
if (select3Result) console.log(select3Result[0]);
delete1Result = await connection.execute(sqlDELETE1);
if (delete1Result) console.log(delete1Result[0].affectedRows, "record(s) deleted.");
select4Result = await connection.execute(sqlSELECT4);
if (select4Result) console.log("Table tblToDos contains", select1Result[0].length, "records.");
drop1Result = await connection.execute(sqlDROP1);
if (drop1Result) console.log("Table tblToDos droped.");
} catch(error) {
// Complete MySQL error message.
// console.log(`%cError: ${error.message}`, "color: #e53935");
// Build a smaller MySQL error message.
const errorPart1 = error.message.split(";")[0];
const errorPart2 = error.message.split(" use ")[1];
console.log(`%cError: ${errorPart1} ${errorPart2}`, "color: #e53935");
} finally {
connection.release();
}
// Debug created datetime
console.log(select3Result[0][0].todoCreated.toLocaleString());
// End pool when web server shuts down.
await pool.end();

How to insert some table information to another table column?

I am a beginner in mySQL and I am trying to create a user's table with information about the user (see code) and populate a column with details that I get from a new table that gets created.
Now I want to be able to put some information from the 'creditcards' table like number for example, to the 'users' table which includes a column 'creditcard', so that I can see each user's credit card number.
I am also comparing the name of the user with the name of the credit card owner so it populates the table according to the user.
I couldn't find any information about the specific problem I am having here.
Here's how I am trying to write:
con.query(createNewCreditCard, [name, type, number, expiration, svss], (err, results) => {
if (err) {
console.error(err);
} else {
const JoinCreditCard = 'INSERT INTO users (creditcard) SELECT number,name FROM creditcards WHERE users.name = creditcards.name';
const userCreateModel = `
CREATE TABLE IF NOT EXISTS users (
id INT(11) NOT NULL AUTO_INCREMENT,
name VARCHAR(45) NOT NULL,
username VARCHAR(45) NOT NULL,
email VARCHAR(45) NOT NULL,
phonenumber VARCHAR(45) NOT NULL,
password VARCHAR(255) NOT NULL,
creditcard INT(11),
salt VARCHAR(255),
created_at DATE,
update_at DATE,
deleted_at DATE,
lastSignIn DATE,
PRIMARY key (id)
)
`;
const CreditCardModel = `
CREATE TABLE IF NOT EXISTS creditcards (
id INT(11) NOT NULL AUTO_INCREMENT,
name VARCHAR (25) NOT NULL,
type VARCHAR(10) NOT NULL,
number INT(12) NOT NULL,
expiration DATE,
svss INT(3) NOT NULL,
created_at DATE,
PRIMARY key (id)
)
`;
I am trying to create a user named ula and then a credit card with the name ula (and all other columns) which I am sending from postman.
The credit card part in nodejs looks like this:
const createCreditCard = async (req, res, next) => {
const {
name,
type,
number,
expiration,
svss
}: {
name: String,
type: String,
number: String,
expiration: String,
svss: String
} = req.body;
const createAt = new Date(Date.now());
try {
const createNewCreditCard = 'INSERT INTO creditcards (name, type, number, expiration, svss) VALUES (?,?,?,?,?)';
con.query(createNewCreditCard, [name, type, number, expiration, svss], (err, results) => {
if (err) {
console.error(err);
} else {
const JoinCreditCard = 'UPDATE users SET creditcard=' + number + ' WHERE name="' + name + '"';
console.log(results);
}
});
res.status(201).send({ success: true, message: 'New credit card was created', data: {name, type, number, expiration, svss} });
} catch (error) {
res.status(500).send({ success: false, message: 'Server error' });
}
await next;
}
The server returns 201, I go to mysql, open users, see column creditcard and its NULL.
Because the entry already exists in the database for users table, you should use UPDATE instead of INSERT.
An example that should work with your code (you already know name and number as vars because you just created the credit card info with them, no need to select them again):
con.query(createNewCreditCard, [name, type, number, expiration, svss], (err, results) => {
if (err) {
console.error(err);
} else {
const JoinCreditCard = 'UPDATE users SET creditcard=' + number + ' WHERE name="' + name + '"';
EDIT: this is the new code from your edit. You already use prepared statements, so forget my notice about that. I've updated the query to follow this. What was missing in your code is that you need to actually do the query! Only declaring the constant won't do anything to your database..
try {
const createNewCreditCard = 'INSERT INTO creditcards (name, type, number, expiration, svss) VALUES (?,?,?,?,?)';
con.query(createNewCreditCard, [name, type, number, expiration, svss], (err, results) => {
if (err) {
console.error(err);
} else {
console.log(results);
const JoinCreditCard = 'UPDATE users SET creditcard=? WHERE name=?';
con.query(JoinCreditCard, [number, name], (err, results) => {
if (err) {
console.error(err);
} else {
console.log(results);
}
});
}
});
res.status(201).send({ success: true, message: 'New credit card was created', data: {name, type, number, expiration, svss} });
} catch (error) {
res.status(500).send({ success: false, message: 'Server error' });
}
NOTE: you should know that using the name to reference the credit card will not allow you to have multiple credit cards for one user, and should be careful about users with the same name, or else this query will update both users. It would be safer to always use the user id field in the WHERE clause. (you should know it at this point)
THIS IS WHAT I RECOMMEND:
it's usually better that the creditcard in users only stores the id from the creditcards table. Like this, relations are on the primary key and it's more optimized (you need to get the id after the credit card creation request, in an inner SELECT in following code).
use the ids to identify rows updates, to prevent 2 users to be updated
delete name from creditcards table, it's already in users
having a third table to reference the relations like states user1974729 is not mandatory, however, it will be the case if you conveniently want to be able to have more than one credit card per user or more than one user that share a card (1 to n relation)
code:
//relation based on id instead of number stored in users + name removed. I assume at this point, you know the id of your user (in var "id" used below in `WHERE` clause)
con.query(createNewCreditCard, [type, number, expiration, svss], (err, results) => {
if (err) {
console.error(err);
} else {
const JoinCreditCard = 'UPDATE users SET creditcard=(SELECT id FROM creditcards WHERE type="' + type + '" AND number="' + number + '" AND expiration="' + expiration + '" AND svss="' + svss + '") WHERE id="' + id + '"';
//no change in userCreateModel
//deleted "name" in CreditCardModel
const CreditCardModel = `
CREATE TABLE IF NOT EXISTS creditcards (
id INT(11) NOT NULL AUTO_INCREMENT,
type VARCHAR(10) NOT NULL,
number INT(12) NOT NULL,
expiration DATE,
svss INT(3) NOT NULL,
created_at DATE,
PRIMARY key (id)
)
`;
it is important that the tables are in normalized forms.
There should be 3 tables.
Users -- all the user data
Credit cards -- all the credit card related information.
Users credit card map -- map users to credit card information.

Unknown column in field list nodejs

I have the following code in nodejs. I am using npm's mysql library and I know that all the columns are correct in mysql database but I keep getting the following error: "Unknown column 's3_id' in 'field list'" but when I do select s3_id from custom_videos I get rows back. How can I have an unknown column that exists?
router.post("/submitCustomVideo", async (req, res, next) => {
try {
const data = {};
const {
s3Id,
name,
renderTime,
duration,
description,
selectedCategories,
selectedKeywords,
customFields
} = req.body;
const VALUES = {
s3_id: s3Id,
name,
duration,
description,
render_time: renderTime,
custom_fields: customFields
};
const updateCustomVideoInfoResult = await database.query(
"call updateCustomVideoInfo(?)",
[VALUES]
);
} catch (error) {
console.log(error);
next(error);
}
});
heres my stored procedure
CREATE DEFINER=`mystuff`#`%` PROCEDURE `updateCustomVideoInfo`(s3_id_param varchar(255), name_param varchar(255), duration_param int, description_param varchar(255), render_time_param int, custom_fields_param json)
BEGIN
UPDATE custom_videos SET name = name_param, duration = duration_param, description = description_param, render_time = render_time_param, custom_fields = custom_fields_param WHERE s3_id = s3_id_param;
END
Try to set the columns as a string and also check your datatype for columns.

Multiple mysql queries with Express Node

I am running an Express server and connecting to MySQL Database. I am have two tables which are referenced via the user_id. When the user provides the email address I need to query users to find the user_id then run another query to join the tracking tables. What is the best way to achieve this? Also any best practice recommendations for any part of the code is appreciated!
Thanks
MySQL
CREATE TABLE users (
user_id INT NOT NULL AUTO_INCREMENT UNIQUE,
first_name VARCHAR(100) NOT NULL,
last_name VARCHAR(100) DEFAULT NULL,
email VARCHAR(320) NOT NULL UNIQUE,
date_created TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (user_id)
);
CREATE TABLE tracking (
tracking_id INT NOT NULL AUTO_INCREMENT,
user_id INT NOT NULL,
metric VARCHAR(100) NOT NULL,
unit VARCHAR(100) NOT NULL,
amount DOUBLE DEFAULT NULL,
date_tracked DATE,
date_created TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (tracking_id),
INDEX user_ind (user_id),
FOREIGN KEY (user_id)
REFERENCES users(user_id)
ON DELETE CASCADE
);
Express (Nodejs)
const express = require("express");
const app = express();
const port = 3000;
var mysql = require("mysql");
var connection = mysql.createConnection({
host: "",
user: "",
password: "",
database: "tracker"
});
app.get("/data/:email/metric/:metric", (req, res) => {
console.log(req.params);
let user_id = "";
connection.connect();
connection.query(
`SELECT user_id FROM users WHERE email = '${req.params.email}';`,
function(error, results, fields) {
if (error) throw error;
user_id = results[0].user_id;
}
);
connection.query(
`SELECT users.first_name, users.last_name, users.email, tracking.metric, tracking.unit, tracking.amount, tracking.date_tracked FROM users JOIN tracking ON users.user_id = tracking.user_id WHERE users.user_id = '${user_id}' AND metric = '${
req.params.metric
}';`,
function(error, results, fields) {
if (error) throw error;
res.send(results);
}
);
connection.end();
});
app.listen(port, () => console.log(`Example app listening on port ${port}!`));
You can accomplish this with a single query, just JOIN the tables on user_id
SELECT
u.first_name,
u.last_name,
u.email,
t.metric,
t.unit,
t.amount,
t.date_tracked
FROM users u
INNER JOIN tracking AS t ON t.user_id = u.user_id
WHERE u.email = '<email>' AND t.metric = '<metric>'
Also, FWIW you should use parameterised queries to protect yourself from SQL Injection
const values = [
req.params.email,
req.params.metric
];
connection.query(
'SELECT ... WHERE u.email = ? AND t.metric = ?',
values,
(err, results, fields) => {
if (err) return next(e); // propagate the error to Express, rather than throwing in the callback
res.send(results);
}
);
I need to query users to find the user_id then run another query to join the tracking tables. What is the best way to achieve this?
You can achieve it with only 1 query:
SELECT users.first_name, users.last_name, users.email,
tracking.metric, tracking.unit, tracking.amount,
tracking.date_tracked
FROM users
JOIN tracking
ON users.user_id = tracking.user_id
WHERE users.email = '${req.params.email}' AND
metric = '${req.params.metric}
Also any best practice recommendations for any part of the code is appreciated!
First of all, you should connect to mySql after the server as started .
Connecting&Disconnecting on each request will impact the performance(It's expensive task opening a socket and connecting outside)
so try something like :
CONNECTION.connect().then(() => {
// mysql is ready , let's start express server
// accepting connections
app.listen(port, () => console.log(`ready on ${port}!`));
}).catch(console.error)
Another good practice you should know is running queries in parallel
with Promise.all([]). when you need to run multiple , unrelated queries ,
you can Execute them all one after the other , making the total wait time for all results much smaller .