I want to be able to add a little note, at the beginning of each query, so when I see it in the processlist, or "mytop", I can tell where it’s running.
Is something like this possible?
I am not sure this would work, but it is worth trying.
Simply add "/* some comment or tag */ " before whatever SQL query is sent normally.
It is possible that MySQL server will remove this comment as part of its query analysis/preparation, but it may just leave it as well, so it shows as such in logs and other monitoring tools.
In case the comments get stripped away, and assuming SELECT queries, a slight variation on the above would be to add a calculated column as the first thing after SELECT, something like
SELECT IF('some comment/tag' = '', 1, 0) AS BogusMarker, here-start-the-original-select-list
-- or
SELECT 'some [short] comment/tag' AS QueryID, here-start-the-original-select-list
This approach has the drawback of introducing an extra column value, with each of the results row. The latter form, actually uses the "comment/tag" value as this value, which may be helpful for debugging purposes.
I discovered this today:
MySQL supports the strange syntax:
/*!<min-version> code here */
to embed code that will only be interpreted by MySQL, and only MySQL of a specified minimum version.
This is documented at 9.7 Comments (used by mysqldump, for example).
Such comments will not be parsed out, and included in the processlist, unlike normal ones, and that even if the code is not actually executed.
So you can do something like
/*!999999 comment goes here */ select foo from bla;
to have a comment that will show up in the processlist, but not alter the code. (Until MySQL developers decide to bloat their release numbering and release a version numbered over 99.99.99, in which case you will just have to add another digit.)
Related
I am new to using SQL. I was wondering whether there could be a bug in this program.
/* Insert to interface table all atoms that have diffASA>0*/
insert into NinterfaceAtom(PDB,Chain,Residue,ResId,Symbol,atom,diffASA)
select PDB,Chain,Residue,ResId,Symbol,Atom,max(ASA)-min(ASA) from perAtomASA
group by PDB,Chain,Residue,ResId,Symbol,Atom
having stddev(ASA)>0;
/* Insert to interface table all atoms that have enough distance */
insert ignore into NinterfaceAtoms (PDB,Chain,Residue,ResId,Symbol,atom)
select asa.PDB,asa.Chain,asa.Residue,asa.ResId,asa.Symbol,dist.Atom from interfaceDist dist
inner join
perAtomASA asa
on
dist.PDB=asa.PDB and
dist.Chain=asa.Chain and
dist.ResId=asa.ResId and
dist.Symbol=asa.Symbol and
Seperated=0
I am just unsure why the programmmer before me put asa.PDB instead of dist.PDB in the inner join section.
I was thinking the eighth line needed to be changed from:
select asa.PDB,asa.Chain,asa.Residue,asa.ResId,asa.Symbol,dist.Atom from interfaceDist dist
to:
select dist.PDB,dist.Chain,dist.Residue,dist.ResId,dist.Symbol,dist.Atom from interfaceDist dist
Is that correct? Thanks.
You are joining asa and dist. It is perfectly logical that their values are checked to make sure only matching pairs will be in the result. So, unless you have a very good reason to think that in the line
dist.PDB=asa.PDB
you need dist.PDB instead of asa.PDB, the command looks to be correct. And if this would be correct
dist.PDB=dist.PDB
then it would be trivial and it would be pointless to have this part checked at all. When you identify bugs, you either need to see behavioral problems of the software, or to understand the code you are looking at.
EDIT
In the select clause one can use asa.PDB or dist.PDB, because the two are equal because of the on condition which ensured their equality. If the two are different, then the pair will not be in the result. So, in terms of values, it makes no difference. But if it is more intuitive to have dist.PDB in the select, then you might want to change it (there is no harm in it, because the value is exactly the same) so the code will be more readable and later, if the code is changed and the two will no longer have to be equal, you will not have new bugs out of the blue sky.
After I typed 13,10 into the value box as shown in the image1, it comes up with decimal(13,0). What's the problem?
Thanks.
You might be seeing the effect of this bug, which exists with version 4.6.4 and will be fixed with the upcoming version 4.6.5. You can also apply the patch yourself.
The preview may be incorrect, but try saving it and see if that is correct. In my testing this is working, but we may be on different versions. Also, if it saves incorrectly (if there is a bug in phpMyAdmin) you can always do an alter (Change button) and see if that block of code handles it correctly. If all else fails, there's always the command line to sort it out.
The project that I'm working on is using MySQL on RDS (mysql2 gem specifically).
When I use a hash of conditions including a range in a where statement I'm getting a bit of an odd addition to my query.
User.where(id: [1..5])
and
User.where(id: [1...5])
Result in the following queries respectively:
SELECT `users`.* FROM `users` WHERE ((`users`.`id` BETWEEN 1 AND 5 OR 1=0))
SELECT `users`.* FROM `users` WHERE ((`users`.`id` >= 1 AND `users`.`id` < 5 OR 1=0))
The queries work perfectly fine since OR FALSE is effectively a no-op. I'm just wondering why Rails or ARel is adding this snippet into the query.
EDIT
It looks like the line that could explain this is line 26 in ActiveRecord::PredicateBuilder. Still no idea how the hash could be empty? at that point but maybe someone else does.
EDIT 2
This is intersting. I was looking into Filip's comment to see why he made it since it seems just like a clarification but he is correct that 1..5 != [1..5]. The former is an inclusive range from 1 to 5 where as the latter is an array whose first element is the former. I tried putting these into an ARel where call to see the SQL produced and the OR 1=0 is not there!
User.where(id: 1..5) #=> SELECT "users".* FROM "users" WHERE ("users"."id" BETWEEN 1 AND 5)
User.where(id: 1...5) #=> SELECT "users".* FROM "users" WHERE ("users"."id" >= 1 AND "users"."id" < 5)
While I still do not know why ARel is adding the OR 1=0 which will always be false and seemingly unnecessary. It may be due to how Arrays and Ranges are handled differently.
Building on the fact, which you've discovered, that [1..5] is not the correct way to specify the range... I have discovered why [1..5] behaves as it does. To get there, I first found that an empty array in a hash condition produces the 1=0 SQL condition:
User.where(id: []).to_sql
# => "SELECT \"users\".* FROM \"users\" WHERE 1=0"
And, if you check the ActiveRecord::PredicateBuilder::ArrayHandler code, you'll see that array values are always partitioned into ranges and other values.
ranges, values = values.partition { |v| v.is_a?(Range) }
This explains why you don't see the 1=0 when using non-range values. That is, the only way to get 1=0 from an array without including a range is to supply an empty array, which yields the 1=0 condition, as shown above. And when all the array has in it is a range you're going to get the range conditions (ranges) and, separately, an empty array condition (values) executed. My guess is that there isn't a good reason for this... it just simply is easier to let this be than to avoid it (since the result set is equivalent either way). If the partition code was a bit smarter then it wouldn't have to tack on the additional, empty values array and could skip the 1=0 condition.
As for where the 1=0 comes from in the first place... I think that comes from the database adapter, but I couldn't find exactly where. However, I would call it an attempt to fail to find a record. In other words, WHERE 1=0 isn't ever going to return any users, which makes sense over alternative SQL like WHERE id=null which will find any users whose id is null (realizing that this isn't really correct SQL syntax). And this is what I'd expect when attempting to find all Users whose id is in the empty set (i.e. we're not asking for nil ids or null ids or whatever). So, in my mind, leaving the bit about exactly where 1=0 comes from as a black box is OK. At least we now can reason about why the range inside of the array is causing it to show up!
UPDATE
I've also found that, even when using ARel directly, you can still get 1=0:
User.arel_table[:id].in([]).to_sql
# => "1=0"
This is strictly speaking a guess, since I did something similar in a project of my own (although I used AND 1).
For whatever reason, when generating a query, it is easier to always have a WHERE clause containing a no-op than it is to conditionally generate the WHERE clause at all. That is, if you don't include any where sections it will end up generating something still valid.
On the other hand, I'm not sure why it's taking this form: when I did it I use 1 [<AND (generated code)>...] it allowed arbitrary chaining, but I don't see how what you're seeing would allow it. None the less, I still think it likely to be a result of an algorithmic code generation scheme.
Check to see if you are using active_record-acts_as. That was the problem with me.
Add the line below to your Gemfile:
gem 'active_record-acts_as', :git => 'https://github.com/hzamani/active_record-acts_as.git'
This will just pull the latest version of the Gem that will hopefully be fixed. Worked for me.
I think you're seeing side effects of ruby personally.
I think the better way to do what you're doing would be with
2.0.0-p481#meri :008 > [*1..5]
=> [1, 2, 3, 4, 5]
User.where(id: [*1..5]).to_sql
"SELECT `users`.* FROM `users` WHERE `users`.`id` IN (1, 2, 3, 4, 5)"
As this creates an Array vs an Array with element 1 of class Range.
OR
use an explicit Range to trigger the BETWEEN in AREL.
# with end element, i.e. exclude_end=false
2.0.0-p481#meri :013 > User.where(id: Range.new(1,5)).to_sql
=> "SELECT `users`.* FROM `users` WHERE (`users`.`id` BETWEEN 1 AND 5)"
# without end element, i.e. exclude_end=true
2.0.0-p481#meri :022 > User.where(id: Range.new(1, 5, true)).to_sql
=> "SELECT `users`.* FROM `users` WHERE (`users`.`id` >= 1 AND `users`.`id` < 5)"
If you care about having control of the queries you generate and the full power of the SQL language and database features then I would suggest moving from ActiveRecord/Arel to Sequel.
I can honestly say there are a lot more quirks and infuriating times ahead for you with ActiveRecord, especially when you move beyond simple crud like queries. When you start trying to query your data in anger, perhaps needing to join a few join tables here and there and realize you really do need join conditions or union all type queries.
It is also significantly faster and more reliable in its query generation and result handling and much easier to compose the queries you want. It also has real documentation you can actually read unlike arel.
I just wish I had discovered it much earlier rather than persisting with the rails default data access layer.
I have a million odd rows where most start
'http://www.' or 'https://www.'
but occasionally they start with no 'www.' - this may be correct but the website owner wants consistency throughout the data and thus I need to update the table to always have 'www.'
I'm struggling with the SQL to do this. I tried:
select * from the_million where URL like 'http://[!w]'
But that returns 0 records so I've fallen at the first hurdle of building up the SQL. I guess after I've got the records I want I'll then do a replace.
I'm happy to run this in two goes for each of http and https so no need for anything fancy there.
You can try this query:
UPDATE the_million SET url=REPLACE(url, 'http://', 'http://www.')
WHERE url NOT LIKE 'http://www.%' AND url NOT LIKE 'https://www.%'
UPDATE the_million SET url=REPLACE(url, 'https://', 'https://www.')
WHERE url NOT LIKE 'http://www.%' AND url NOT LIKE 'https://www.%'
Search & replace in 2 queries.
try this
select * from the_million where URL not like 'http://www.%'
This condition:
URL like 'http://[!w]'
... is identical to this one:
URL='http://[!w]'
because it doesn't contain any valid wildcard for MySQL LIKE operator. If you check the MySQL manual page you'll see that the only wildcards are % and _.
The W3Schools page where you read that [!charlist] is valid identifies the section as "SQL Wildcards" which is misleading or plain wrong (depending on how benevolent you feel). That's not standard SQL at all. The error messages returned by their "SQL Tryit Editor" suggest that queries run against a Microsoft Access database, thus it's only a (pretty irrelevant) SQL dialect.
My advice:
Avoid W3Schools as reference site. Their info is often wrong and they apparently don't care enough to amend it.
Always use the official manual of whatever DBMS engine you are using.
Last but not least, the good old www prefix is not a standard part of the HTTP protocol URIs (like http://); it's only a naming convention. Preppending it to an arbitrary list of URLs is like adding "1st floor" to all your customer addresses. Make sure your client knows that he's paying money to corrupt his data on purpose. And if he feels generous, you can propose him to replace all https: with http: as well.
I need to take a MySQL query and insert a string before each table name. The solution doesn't need to be one line but obviously it's a regex problem. It will be implemented in PHP so having programming logic is also fine.
Rationale and Background:
I'm revamping my code base to allow for table prefixes (eg: 'nx_users' instead of 'users') and I'd like to have a function that will automate that for me so I don't need to find every query and modify it manually.
Example:
SELECT * FROM users, teams WHERE users.team_id = teams.team_id ORDER BY users.last_name
Using the prefix 'nx_', it should change to
SELECT * FROM nx_users, nx_ teams WHERE nx_ users.team_id = nx_ teams.team_id ORDER BY nx_ users.last_name
Obviously it should handle other cases such as table aliases, joins, and other common MySQL commands.
Has anybody done this?
How big of a code base are we talking about here? A regular expression for something like this is seriously flirting with disaster and I think you're probably better off looking for every mysql_query or whatever in your code and making the changes yourself. It shouldn't take more than the hour you'd spend implementing your regex and fixing all the edge cases that it will undoubtedly miss.
Using a regex to rewrite code is going to be problematic.
If you need to dynamically change this string, then you need to separate out your sql logic into one place, and have a $table_prefix variable that is appropriately placed in every sql query. The variable can then be set by the calling code.
$query = "SELECT foo from " . $table_prefix . "bar WHERE 1";
If you are encapsulating this in a class, all the better.
This example does not take into consideration any escaping or security concerns.
First off, regular expressions alone are not up to the task. Consider things like:
select sender from email where subject like "from users group by email"
To really do this you need something that will parse the SQL, produce a parse tree which you can modify, and then emit the modified SQL from the modified parse tree. With that, it's doable, but not advisable (for the reasons Paolo gave).
A better approach would be to grep through your source looking for either the table names, the function you use to sent SQL, the word from, or something like it at script something to throw you into an editor at those points.