i'm trying to create a stats bot for discord but i am having some issues trying to get part of it to work.
What i am trying to do is record the number of messages sent per channel at 5 min intervals. In my bot i have the following code:
// on message events
bot.on("message", async message => {
if(message.author.bot) return;
if(message.channel.type === "dm") return;
//update channel stats
connection.query(`SELECT * FROM channel_stats WHERE channel_id = '${message.channel.id}' AND date BETWEEN timestamp(DATE_SUB(NOW(), INTERVAL 5 MINUTE)) AND timestamp(NOW())`, (err, rows) => {
if(err) throw err;
let sql;
if(rows.length < 1) {
sql = `INSERT INTO channel_stats (channel_id, date, channel_name, channel_message_count) VALUES ('${message.channel.id}', NOW(), '${message.channel.name}', 1)`;
} else {
let channel_message_count = rows[0].channel_message_count;
sql = `UPDATE channel_stats SET channel_message_count = ${channel_message_count + 1}, channel_name = '${message.channel.name}' WHERE channel_id = '${message.channel.id}'`;
};
connection.query(sql)
});
however, the bot always inserts a new row and never updates an existing one.
I ran the sql query SELECT * FROM channel_stats WHERE channel_id = 'xxxxxxxxxxxxx' AND date BETWEEN timestamp(DATE_SUB(NOW(), INTERVAL 5 MINUTE)) AND timestamp(NOW()) manually via phpmyadmin and this seems to be working correctly - only returns rows created within the last 5 mins.
I'm struggling to understand why the bot is constantly adding new rows. Any help would be appreciated.
Thanks!
managed to resolve this myself, had the channel_id row as int(11) which was causing the channel_id to be truncated.
changed it to VARCHAR(30) and it's working perfectly now.
Related
I have a Model Charter that hasMany BlackoutRanges. I am trying to setup a scope to return charters that do not have a blackoutRange created for 3 dates. In our system a blackoutRange is a single day.
Pseudocode would look something like this:
// query where doesnt have blackoutDates on all three of the following dates:
// start date === date passed in
// start date === date passed in plus 1 days
// start date === date passed in plus 2 days
I tried to do some logical grouping and came up with this and I also logged my raw sql query to see what it looks like:
$dayTwo = $date->copy()->addDays(1);
$dayThree = $date->copy()->addDays(2);
return $query->whereDoesntHave("blackoutRanges", function(Builder $subQuery) use($date, $dayTwo, $dayThree){
$temp = $subQuery->where(function($queryOne) use($date) {
return $queryOne->whereDate('start', $date)->where('trip_id', NULL);
})->where(function($queryTwo) use($dayTwo) {
return $queryTwo->whereDate('start', $dayTwo)->where('trip_id', NULL);
})->where(function($queryThree) use($dayThree) {
return $queryThree->whereDate('start', $dayThree)->where('trip_id', NULL);
});
logger($temp->toSql());
return $temp;
});
select * from `blackout_ranges` where `charters`.`id` = `blackout_ranges`.`charter_id` and (date(`start`) = ? and `trip_id` is null) and (date(`start`) = ? and `trip_id` is null) and (date(`start`) = ? and `trip_id` is null)
I've also logged the dates passed in and they are:
[2022-06-07 19:00:58] local.DEBUG: 2022-06-09 00:00:00
[2022-06-07 19:00:58] local.DEBUG: 2022-06-10 00:00:00
[2022-06-07 19:00:58] local.DEBUG: 2022-06-11 00:00:00
An example of the start column of a BlackoutRange would be: 2022-07-21 04:00:00
Would the fact that they are not matching timestamps be a reason for this scope not working?
And how do I know it's not working? I added blackoutRanges for a specific charter for the entire month of June and it's still being returned.
I am looking for any advice on this one as I've been plugging away now for a day and a half and have gotten no where.
I've been battling with this problem now for a couple of weeks.
I have a Laravel 7 application that seems to be absolutely hammering the database when performing a certain query. First I'll outline the problem and then dive a bit more into what I've tried to isolate it.
I have an Opportunity model that has a scopeLucrative() scope on the model that filters the opportunities to only show ones between the users defined threshold as per below:
public function scopeLucrative($query)
{
$user = auth()->user();
$threshold = $user->preference('quality.threshold');
$expiredThreshold = $user->preference('quality.expired_threshold');
$hideOwnReports = $user->preference('quality.hide_own_price_changed_reports');
/**
* For an Opportunity to be an lucrative one, the value has to be over the user's threshold.
*/
// Where probability is over the user's threshold
return $query->where('probability', '>=', $threshold)
// And where the number of false reports is less than the user's expired threshold
->whereHas('verifiedPriceReports', function ($report) {
$report->where('correct_price', false)->distinct('user_id')->take(15);
}, '<', $expiredThreshold)
// And when the user has 'hide_own_price_changed_reports' on, hide ones they've marked as incorrect
->when($hideOwnReports, function ($query) use ($user) {
return $query->whereDoesntHave('verifiedPriceReports', function ($report) use ($user) {
$report->where('user_id', $user->id)->where('correct_price', false);
});
});
}
When called like Opportunity::with('history', 'verifiedPriceReports')->lucrative()->orderByDesc('updated_at')->paginate(10)) the database seems to be fetching a large number of rows (and takes 600ms) according to the DigitalOcean control panel despite the query only returning 10 rows as expected due to the pagination.
As you can imagine, this doesn't scale well. With only 5 active users the database queries start taking seconds to return. The query that is performed by that Query Builder is:
SELECT *
FROM `opportunities`
WHERE `probability` >= '-15'
and (SELECT distinct count(*)
FROM `opportunity_reports`
WHERE `opportunities`.`id` = `opportunity_reports`.`opportunity_id`
and `correct_price` = '0'
and `updated_at` >= '2020-09-06 04:20:17') < 3
and not exists(SELECT *
FROM `opportunity_reports`
WHERE `opportunities`.`id` = `opportunity_reports`.`opportunity_id`
and `user_id` = '1'
and `correct_price` = '0'
and `updated_at` >= '2020-09-06 04:20:17')
ORDER BY `probability` DESC
LIMIT 10 offset 0;
It didn't take long to narrow the problem down to scopeLucrative, with a simple call to the model without the lucrative scope like Opportunity::with('history', 'verifiedPriceReports')->orderByDesc('updated_at')->paginate(10)) performing as expected.
I'm at a loose end as to what I can do to fix this. Has anyone experienced anything like this before?
I solved this by replacing whereHas() with whereRaw()
->whereRaw('opportunities.id not in (SELECT opportunity_id
FROM opportunity_reports
WHERE correct_price = false
GROUP BY opportunity_id
HAVING COUNT(*) >= '.$expiredThreshold.'
)')
Our client's online shop uses Opencart, for marketing purposes they wanted to sync Opencart orders automatically with Google Sheets. I modified a script in google sheets to extract customer orders from the past year (based on https://www.transpacific-software.com/blog/opencart-to-google-docs-pull-and-sync-data-auto-through-scripting and https://gist.github.com/pradeepbheron/e6129c9fd9bc74e814d0)
The SQL query is:
SELECT order_id, firstname AS first_name, lastname AS last_name,
email, date_added AS order_date, CONCAT("£", FORMAT(total,2))
order_value, payment_address_1 AS billing_address_1,
payment_address_2 AS billing_address_2, payment_city AS
billing_city, payment_postcode AS billing_postcode,
payment_country AS billing_country, items_purchased FROM
(
SELECT order_id, firstname, lastname, email, date_added,
total, payment_address_1, payment_address_2, payment_city,
payment_postcode, payment_country
FROM ocbw_order
GROUP BY order_id
) AS A
JOIN (
SELECT order_id AS product_order_id, GROUP_CONCAT(name
SEPARATOR ", ") AS items_purchased
FROM ocbw_order_product
GROUP BY product_order_id
) AS B
ON A.order_id=B.product_order_id
WHERE date_added >= DATE_SUB(NOW(),INTERVAL 1 YEAR)
AND firstname != ''
It runs fine in phpMyAdmin but Google Script Editor generates an "Exceeded maximum execution time" error.
It looks like there are 7357 rows (exported from myPhpAdmin). Is there a better way to write the query? Also I am trying to rename the column headers but can only two works, i.e:
GROUP_CONCAT(name SEPARATOR ", ") AS items_purchased
and
CONCAT("£", FORMAT(total,2)) order_value
Any thoughts
QUICK UPDATE: Fri Nov 9 11:54:03 2018
1) As request by #ThisGuyHasTwoThumbs, here is a screenshot of the explain table result
2) I looked into the Best Practices doc mentioned by #bcperth. I tried to rewrite the google sheets script but ran into issues.
Here is the amended script.
function p1MySQLFetchData() {
// Change it as per your database credentials
var conn =
Jdbc.getConnection('jdbc:mysql://[dbHostIp]:3306/[dbName]',
'[dbUsername]', '[dbPassword]');
var stmt = conn.createStatement();
var start = new Date(); // Get script starting time
//change table name as per your database structure
var rs = stmt.executeQuery('[sqlQuery]');
// It sets the limit of the
// maximum nuber of rows in a ResultSet object
// Returns the currently active spreadsheet
var doc = SpreadsheetApp.getActiveSpreadsheet();
var cell = doc.getRange('a1');
var row = 0;
// Mysql table column name count.
var getCount = rs.getMetaData().getColumnCount();
// Create array to hold mysql data
var tempArray = [];
// get row and column count for later
var colCount = getCount;
// ATTEMPT TO GET ROW COUNT 1
//rs.last();
//var rowCount = rs.getRow();
//rs.beforeFirst(); // resets rs cursor
//Logger.log(RowCount);
// DOESN'T WORK! result => ReferenceError: "RowCount" is not
// defined. (line 28, file "Code")
// ATTEMPT TO GET ROW COUNT 2
//var rsCount = stmt.executeQuery('SELECT COUNT(*) AS rowcount FROM
//[sqlQuery]);
// It sets the limit of the maximum number of rows in a ResultSet
// object
//rsCount.next();
//var rowCount = rsCount.getString("rowcount");
//Logger.log(rowCount);
// DOESN'T WORK! result => 0
// Build TempArray using MySQL data
for (var i = 0; i < getCount; i++){
tempArray[0][i] = rs.getMetaData().getColumnName(i+1);
// DOESNT WORK => ERROR
// TypeError: Cannot set property "0.0" of undefined to "order_id".
// (line 39, file "Code")
//Logger.log(rs.getMetaData().getColumnName(i+1));
}
var row = 1;
while (rs.next()) {
for (var col = 0; col < rs.getMetaData().getColumnCount();
col++) {
tempArray[row][col] = rs.getString(col + 1);
//Logger.log(rs.getString(col + 1));
}
row++;
}
// BELOW DOESNT AS I CANT GET A ROW COUNT (SEE ABOVE)
// Fill Spreadsheet from tempArray
//for (var row = 0; row < rowCount; row++) {
//for (var col = 0; col < colCount; col++) {
//cell.offset(row, col).setValue(tempArray[row][col + 1]);
//}
// }
rs.close();
stmt.close();
conn.close();
var end = new Date(); // Get script ending time
Logger.log('Time elapsed: ' + (end.getTime() - start.getTime()));
// To generate script log. To view log click on View -> Logs.
}
But as you can see from the comments, I get loads of errors. Am not sure what to do next.
UPDATE Fri Nov 30 15:26:14 2018
In answer to #Tedinoz comment below.
PhpMyAdmin generates 6862 results and the query took 13.3074 seconds.
When I ran the script in Googles Script Editor, it took around 2 minutes 30 to complete and only pulls 6348 records (92.5%). The records stop after 3rd October 2018.
Regarding your suggestions:
1) I tried running a modified script, setting the INTERVAL to:
1 MONTH => I get no results in Google (it should be 529)
2 MONTH => I get 14 results (should be 1029)
3 MONTH => I get 299 results (should be 1669).
They all took about 4-7 second in myPhpAdmin vs 5 - 20 seconds for Google Script Editor
2) Do you mean exporting a csv from phpMyAdmin and importing to Google Sheets? Well I did that and it works fine.
Another thing I have noticed is that the order_id's in Google Sheets don't match that from phpMyAdmin. Weird.
Currently this is my code, which inserts the new_item correctly
var new_item = {id_user: id, id_restaurant: id_rest, type: 3, date: date};
connection.query("INSERT INTO table SET ?", [new_item], function(err, results) {
if(err){
...
}
...
}
But now I don't want to send the date with it, I want to use SET ? and do something likedate = NOW() basically I want to insert this:
var new_item = {id_user: id, id_restaurant: id_rest, type: 3};
and use date = NOW()-INTERVAL 6 hour to assign the date.
EDIT: I posted I only wanted Now() but actually I want to set Interval it before inserting.
I think it is better to use moment.js.
You can use Date.now which returns current timestamp(milliseconds), so make sure to format it. The code is as follows:
var a = moment(Date.now()).format('YYYY-MM-DD HH:mm:ss');
Please let me know if this works out
new Date().toISOString().slice(0,10) + " " + new Date().toISOString().slice(11, 19)
same as 'YYYY-MM-DD HH:mm:ss'
Using NOW() is executed by the database interpreter and sets the database time.
You could create a new variable to use in your SQL statement and set the wanted content to that one.
var new_item1 = {id_user: id, id_restaurant: id_rest, type: 3, date: date};
var new_item2 = {id_user: new_item1.id_user, ...};
connection.query("INSERT INTO table SET ?", [new_item2], function(err, results) {
if(err){
...
}
...
}
Then you could configure your column to set the NOW() timestamp as default.
With a conditional Insert, I want to know if a row was inserted or if overlap was found.
I have a query that inserts a request into the database if it doesn't overlap with any existing entry. The query works, but how do I know if there's been an collision? The query just executes successfully. Here is my query:
INSERT INTO requests(id, start_time, end_time)
SELECT NULL, 1463631640671,1463636731000,
FROM Dual
WHERE NOT EXISTS (
SELECT * FROM requests
WHERE
start_time < 1463638531000 AND
end_time > 1463636731000
)
For your information. I'm doing this inside Node.js with the mysql package.
and the code looks like this, and it returns 'SUCCESS' every time.
var queryString = "INSERT INTO requests(id, start_time, end_time..."
connection.query(queryString, function(error, results, fields) {
if (!error) {
console.log('SUCCESS!');
}
else {
console.log('Insert into requests unsuccessful.');
console.log('ERROR: ' + error);
}
});
you can inspect results.insertId and see if you have something in it.