I have a database, with a bunch of different columns. Let's say it looks like this:
TABLE: dbtable
__________________________________
| onecolumn | id | twocolumn |
|---------------------|------------|
| some | 1 | matchthis |
| random | 2 | dontmatch |
| thing | 3 | matchthis |
| here | 4 | dontmatch |
|__________________________________|
I also have a Node.js program (using Sequelize), that is supposed to sort through the dbtable table, and find the row that has the highest ID value, as well as has twocolumn equal "matchthis". (all of these values are examples).
The code looks something like this:
const rel = await dbtable.findOne({
where: {
twocolumn: req.params.twocolumn // Url param that equals "matchthis"
},
order: Sequelize.fn('max', Sequelize.col('id')),
});
However, instead of returning {"onecolumn": "thing", "id": "3", "twocolumn": "matchthis"}, it returns this error:
SequelizeDatabaseError: Expression #1 of ORDER BY contains aggregate function and applies to the result of a non-aggregated query
I'm not very good with Databases, and i can't seem to figure out what is going on.
Your query doesn't contain a GROUP BY clause making it a 'non-aggregated query' as the error suggests, making it incompatible with ordering by an aggregating function (MAX()).
You can order directly on the id column instead.
const rel = await dbtable.findOne({
where: {
twocolumn: req.params.twocolumn
},
order: [
[ 'id', 'DESC' ]
]
});
Related
I have the following json data in a mysql column:
{
"components":[
{
"url":"www.google.com",
"order":3,
"accountId":"123",
"autoPilot":true,
},
{
"url":"www.youtube.com",
"order":7,
"accountId":"987",
"autoPilot":false,
},
"addTabs":true,
"activateHomeSection":true
}
I need to update the url attribute based on accountId and autopilot attribute.
For example I pass:
accountId = 123,
autopilot = true,
url = 'facebook.com'
Result: AccountId and autopilot matches -> www.google.com url changes to www.facebook.com
{
"components":[
{
"url":"www.google.com",
"order":3,
"accountId":"123",
"autoPilot":true
},
...
Till this point I managed to write this code:
select j.url from users, json_table(custom_page, '$.components[*]' columns (
autoPilot boolean path '$.autoPilot',
accountId varchar(36) path '$.accountId',
url TEXT path '$.url')) as j
where username='mihai2nd' AND j.autoPilot = true and j.accountId='123';
I was planning to retrieve first the url, and then use a regex function to update it.
But such way is not good in performance, as I will need to send 2 queries.
I tried also other methods with JSON_SEARCH / JSON_CONTAINS / JSON_EXTRACT, but failed.
Do you know guys a nice way to achieve this, without affecting the performance ?
The best way to do this with good performance is to store the attributes in a normal table with rows and columns, instead of storing them in JSON. If you store the data in JSON, it resists being optimized with indexes.
MySQL 8.0.17 and later support Multi-Valued Indexes which allows you to create an index over a JSON array. Unfortunately, you can only reference one column as a multi-valued index, so you need to choose which one of your attributes is most likely to help improve the performance if it is searchable via an index lookup. Let's assume it's accountId.
mysql> create table user (custom_page json);
mysql> alter table user add index bk1 ((CAST(custom_page->'$.components[*].accountId' AS UNSIGNED ARRAY)));
mysql> explain select * from user where 12 member of (custom_page->'$.components[*].accountId');
+----+-------------+-------+------------+------+---------------+------+---------+-------+------+----------+-------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+-------+------------+------+---------------+------+---------+-------+------+----------+-------------+
| 1 | SIMPLE | user | NULL | ref | bk1 | bk1 | 9 | const | 1 | 100.00 | Using where |
+----+-------------+-------+------------+------+---------------+------+---------+-------+------+----------+-------------+
You can see in the above example that the optimizer will use the index I created when I use the MEMBER OF() predicate to search.
That will at least narrow down the search to rows where the value is found. Then you can use your JSON_TABLE() approach to extract exactly which entry has the attributes you want, knowing that the search will be limited to the rows that matched your MEMBER OF() predicate.
select user.id, j.* from user
cross join json_table(custom_page, '$.components[*]' columns(
rownum for ordinality,
autoPilot boolean path '$.autoPilot',
accountId varchar(36) path '$.accountId',
url text path '$.url')
) as j
where 123 member of (custom_page->'$.components[*].accountId');
+----+--------+-----------+-----------+-----------------+
| id | rownum | autoPilot | accountId | url |
+----+--------+-----------+-----------+-----------------+
| 1 | 1 | 1 | 123 | www.google.com |
| 1 | 2 | 0 | 987 | www.youtube.com |
+----+--------+-----------+-----------+-----------------+
Now you can filter these generated rows:
select user.id, j.* from user
cross join json_table(custom_page, '$.components[*]' columns(
rownum for ordinality,
autoPilot boolean path '$.autoPilot',
accountId varchar(36) path '$.accountId',
url text path '$.url')
) as j
where 123 member of (custom_page->'$.components[*].accountId')
and j.autoPilot = true
and j.accountId = 123;
+----+--------+-----------+-----------+----------------+
| id | rownum | autoPilot | accountId | url |
+----+--------+-----------+-----------+----------------+
| 1 | 1 | 1 | 123 | www.google.com |
+----+--------+-----------+-----------+----------------+
That's SELECT, but you need to UPDATE.
with cte as (
select user.id, j.* from user
cross join json_table(custom_page, '$.components[*]' columns(
rownum for ordinality,
autoPilot boolean path '$.autoPilot',
accountId varchar(36) path '$.accountId',
url text path '$.url')
) as j
where 123 member of (custom_page->'$.components[*].accountId')
and j.autoPilot = true
and j.accountId = 123
)
update user cross join cte
set custom_page = json_set(custom_page, concat('$.components[', cte.rownum-1, '].url'), 'facebook.com')
where cte.id = user.id;
Query OK, 1 row affected (0.00 sec)
Rows matched: 1 Changed: 1 Warnings: 0
Then check that it updated the way you intended:
mysql> select id, json_pretty(custom_page) from user;
+----+--------------------------+
| id | json_pretty(custom_page) |
+----+--------------------------+
| 1 | {
"addTabs": true,
"components": [
{
"url": "facebook.com",
"order": 3,
"accountId": "123",
"autoPilot": true
},
{
"url": "www.youtube.com",
"order": 7,
"accountId": "987",
"autoPilot": false
}
],
"activateHomeSection": true
} |
+----+--------------------------+
Frankly, this is needlessly difficult and complex. It's going to be very time-consuming to develop any code to change JSON data if you store it this way. If you need to work with JSON as though it is a set of discrete rows and columns, then you should create a table with normal rows and columns. Then virtually anything you struggled to do with JSON functions becomes simple and straightforward:
CREATE TABLE page_components (
id INT PRIMARY KEY,
accountId VARCHAR(36) NOT NULL,
autoPilot BOOLEAN NOT NULL DEFAULT FALSE,
`order` INT NOT NULL DEFAULT 1,
url TEXT NOT NULL,
KEY (accountId, autoPilot)
);
...populate it with data you would have stored in JSON...
UPDATE page_components
SET url = 'facebook.com'
WHERE accountId = 123 AND autoPilot = true;
The more I see questions on Stack Overflow about JSON in MySQL, the more I conclude that it's the worst feature ever to add to a relational database.
I have this query:
SELECT
id,
name,
lastName
FROM
users
WHERE
name IN (??)
and then use it like this:
import { sqlQuerySeenAbove } from './sql/queries'
async function createReport() {
try {
const arrOfStrings = ['John', 'Laura']
const pool = await getConnection()
const res = pool.query(sqlQuerySeenAbove, conn.escape(arrOfStrings))
console.log('res -->', res)
} catch(e){
console.log(e)
}
}
the error message that I am getting is:
"Unknown column ''John', 'Laura'' in 'where clause'",
name IN (`'John', 'Laura'`)\n"
I think the problem is that within this () section the mysql is placing the back ticks to signal that it is looking for the column rather than the values. the mysql docs say that when escaping characters:
Arrays are turned into list, e.g. ['a', 'b'] turns into 'a', 'b'
also the escaping identifiers has a similar query as mine so I am not sure what the issue is. I am aware that within the docs it says also that the ?? is used as a placeholder for escaped characters, but, if I do:
pool.query(sqlQuerySeenAbove, arrOfStrings)
I get the error:
"Unknown column 'John' in 'where clause'",
note: I understand that escaping has it's own issues in terms of security but this is internal usage, so for argument sake let's say that we aren't worried about any sql injection
what I would like is a return of this:
/------------------------------------\
| id | name | lastName |
|------------------------------------|
| 1 | John | Smith |
|------------------------------------|
| 2 | John | Doe |
|------------------------------------|
| 3 | Laura| Smith |
|------------------------------------|
| 4 | Laura| Doe |
\------------------------------------/
my issue was that I had to change a couple things:
in the query you don't use (??) use only (?) this is because the ?? is short hand for escaping and therefore will look for the column name instead of the value
where executing the command you don't need to get the connection and escape, for the same reason as above so: pool.query(sqlQuerySeenAbove, arrOfStrings)
the arrOfStrings needs to be a nested array, so the final solution is: pool.query(sqlQuerySeenAbove, [arrOfStrings])
the reason for #3 is because if you had multiple values it will iterate through that and search for that value:
[['John', 'Laura'], ['Smith', 'Doe']]
mysql will iterate through the first array, then the second.
I have a fairly simple MySQL situation. There's a table with JSON values in it that I want to find the max of.
Something like this:
ID | data
1 | { value: 1 }
1 | { value: 2 }
1 | { value: 3 }
2 | { value: 4 }
...
When I query via something simple like this:
SELECT id, max(data->'$.value')
FROM table
GROUP BY id
I'm getting simply wrong values. Something like:
ID | max...
1 | 2
2 | 4
When I debug my query by dropping in a group_concat to see what's being aggregating over:
SELECT id, group_concat(data->'$.value'), max(data->'$.value')
FROM table
GROUP BY id
Suddenly the max function returns the right value and the group_concat shows the list of values it theoretically should be maximizing over.
What's going on? A quick search reveals very little. Thanks!
I have an elquent model named Conversation.
In that model I define a hasMany relationship on an eloquent model CallParticipant like so:
public function participants(){
return $this->hasMany('App\Models\Purecloud\Analytics\CallParticipant', 'conversationId', 'conversationId');
}
Now, CallParticipants can be system processes, customers, or agents. I need a list of all the CallParticipants that are considered "agents" so I defined another relation like this:
public function agents(){
return $this->hasMany('App\Models\Purecloud\Analytics\CallParticipant', 'conversationId', 'conversationId')
->whereIn('partPurpose', ['agent','user']);
}
Now, an "agent" can be a "participant" multiple times on a "conversation" that is to say there can be multiple rows for the same agent just with a different participantId, like this:
+----------------+---------------+--------------+-------------+
| conversationId | participantId | partUserName | partPurpose |
+----------------+---------------+--------------+-------------+
| 1 | 100 | Alex | agent |
| 1 | 101 | Mary | agent |
| 1 | 102 | Alex | agent | <-- I want to exlcude this
+----------------+---------------+--------------+-------------+
I need to remove these sortof-duplicates so that the relationship only returns one row for each partUserName (one row for Mary, and one row for Alex).
I tried to do this by adding a groupBy like this:
public function agents(){
return $this->hasMany('App\Models\Purecloud\Analytics\CallParticipant', 'conversationId', 'conversationId')
->whereIn('partPurpose', ['agent','user'])
->groupBy('partUserName');
}
But this produces the error:
SQLSTATE[42000]: Syntax error or access violation: 1055 'analytics.callParticipants.participantId' isn't in GROUP BY
I also tried doing it in the with statement like below, but I get the same error:
$conversations = Conversation::with(
[
'agents' => function($query){
$query->groupBy('partUserName');
}
])
->get();
Is there any way I can limit the relationship to only rows with unique partUserName
I think that the only way to do this is by "hacking" the relation. Could you try:
public function agents(){
return $this->hasOne('App\Models\Purecloud\Analytics\CallParticipant', 'conversationId', 'conversationId')
->whereIn('partPurpose', ['agent','user'])
->latest();
}
I have the following record in artists table:
| id | names |
| 1 | eminem rihanna rita ora|
| 2 | taylor swift beyonce |
I want to search for example using inem and I want the id of this record to be found which is id = 1 in this case. I'm using Full Text Search in MySQL.
Is there a better technique to achieve this?
Update:
| id | video_name | title | artists | search_tags |
| 1 | onajr | o'najr | kida | onajr o'najr kida |
I want to search using this strings for ex. onajr, ona, kida, kid.
So in e few words a user can search using al the search tags including part of a tag.
This is my function in php. :
public function tags_search() {
//echo 'ok';
$db = DB::get_instance();
//single word
$query = "
select * from `videos`
where
match(`search_tags`) against (':video_name' IN BOOLEAN MODE) order by rand()
limit 1
";
try {
$run_query = $db->prepare($query);
$run_query->execute(array(
':string' => $this->string
));
return ($run_query->rowCount() == 1) ? $run_query->fetch(PDO::FETCH_OBJ)->video_name : false;
} catch(PDOException $e) {
echo $e->getMessage();
}
}
You should be able to just use LIKE
SELECT id FROM artists WHERE names LIKE '%inem%';
The %'s are wild cards saying that anything can come before or after them.
Also, it's generally not a good idea to store multiple values in a single field. I'd recommend making id and names into a composite PK, and then only have one name per field.
eg)
| id | name |
| 1 | eminem |
| 1 | rihanna |
| 1 | rita ora |
| 2 | taylor swift |
| 2 | beyonce |
If you plan on adding more fields, they should be in another table. Learn about database normalization for the reason why.
Edit after update:
I still think a LIKE statement would suit your purposes, the only thing you would have to do different is add a AND search_tags LIKE '%ona%' etc for each comma delineated tag word.
Just as an example from what you have above:
SELECT id FROM artist
WHERE search_tags LIKE '%onajr%'
AND search_tags LIKE '%ona%'
AND search_tags LIKE '%kida%'
AND search_tags LIKE '%kid%';
I don't know enough about PHP to actually give you some code, but after a short search, explode() appears to be the function you want to look at. Then just append the extra SQL for each search tag. This method would also work if you were to split your tags into another table. If I'm not mistaken, you should just have to group by the songid if you go that route.
Not sure if all this is better than what you have. As I said, I don't know much PHP, and I haven't messed around with match(...) against(...), but I thought I'd throw in my own 2ยข.