This might have been asked before, but I can't seem to find any relevant answer anywhere.
I wrote a couple of long queries in Access (2003) a couple of weeks ago. I wanted to use those queries today and found that they are been truncated. The query starts as it should and somewhere halfway through it says AS [%$###_Alias] and the rest is missing.
Both query are only a few 100 characters long, so well below the limit allowed by Access.
The structure of both queries is like this
SELECT Some calculated fields with a few IIF statements
FROM
(
SELECT Some fields calculated with lots of IIF statement
FROM joined table
WHERE some condition
)
ORDER BY Some fields
Which I do not is particularly demanding of Access, and should certainly not cause it truncate the query in anger. So why does this happen?
Please don't slam me for putting in Japanese text now, the next part is purely for reference. One of my actual (truncated) queries looks like this:
SELECT
管理番号,
種別,
IIF(種別TYPE='', OTHER種別TYPE, IIF(種別TYPE='その他','その他' & OTHER種別TYPE,種別TYPE)) AS TYPE
FROM [
SELECT
管理番号,
IIF(建物か設備_建物,'建物',IIF(建物か設備_設備, '設備', '種別なし')) AS 種別,
IIF(建物か設備_建物,IIF(建物用途_戸建住宅,'戸建住宅',IIF(建物用途_共同住宅,'共同住宅',IIF(建物用途_教育施設,'教育施設',IIF(建物用途_庁舎,'庁舎',IIF(建物用途_事務所,'事務所',IIF(建物用途_工場,'工場',IIF(建物用途_倉庫,'倉庫',IIF(建物用途_店舗,'店舗',IIF(建物用途_物置小屋,'物置小屋',IIF(建物用途_車庫,'車庫',IIF(建物用途_農作小屋,'農作小屋',IIF(建物用途_その他,'その他')))))))))))),
IIF(建物か設備_設備,IIF(設備種別_物置,'物置',IIF(設備種別_駐車場,'駐車場',IIF([設備種別_運動場/庭等],'運動場/庭等',IIF(設備種別_電気設備等,'電気設備等',IIF(設備種別_畜舎など,'畜舎など',IIF(設備種別_ﾋﾞﾆｰﾙﾊｳｽ等,'ﾋﾞﾆｰﾙﾊｳｽ等',IIF(設備種別_その他,'その他'))))))),'')) AS 種別TYPE,
'（' & REPLACE(REPLACE(TRIM(建物用途_その他記載欄),'(',''),')','') & '）' AS OTHER種別TYPE
FROM
T_メイン INNER JOIN T対象 ON T_メイン.管理番号=T対象.調査番号]. AS [%$###_Alias];
EDIT
I rechecked both of my queries and unlike I said only one of them got truncated. The other one is intact. Extremely weird...
I've noticed that if I don't assign an alias for a subquery, Access' query designer will assign one for me. And those aliases resemble the one you cited, [%$###_Alias]. However, I've not noticed it discard any part of my SQL text when it added such an alias. So although this suggestion may be a long shot, try assigning your own alias, eg FROM ( ... ) AS sub, and see whether the query designer preserves the rest of your SQL text.
Related
My Access 2010 database has a linked table (on a SQL Server 2012 backend), containing fields ID, Qty, PackSize. My query is meant to select records where Qty is not a multiple of the packsize. Normally I achieve this with a where clause like so: "Where Qty/Packsize <> Fix(Qty/Packsize), but although this works on local tables or linked tables that live in other Access databases, on linked tables which live in SQL server, it returns no results.
If I split up the query into two parts, one with no where clause, creating a new table with ColA: Qty/Packsize, ColB: Fix(Qty/PackSize), and then select where ColA <> ColB, it works fine.
Since I don't really care what the values are, just to know whether they're different, I also tried Int() instead of Fix.
Even weirder, "Where Cdbl(Qty/Packsize) = int(Qty/Packsize)" returns all the records, despite showing me that Cdbl(Qty/Packsize) is for example 425.5 and int(Qty/Packsize) is 425.
Any idea what's going on or how I can achieve this another way? It needs to be in a single step though, as it's really the basis for a record selection in VBA. Why would it not work over a SQL linked table? I've tried in separate databases as well and using a different SQL table, in case it was merely a glitch.
Many thanks in advance.
(Also the title of this question is awful. Edits gratefully received.)
It could be a floating point issue, so try with integers only:
Where Qty <> Fix(Qty/Packsize) * Packsize
By the way, Fix and Int behave the same for positive values.
I'm trying to do what I think is a set of simple set operations on a database table: several intersections and one union. But I don't seem to be able to express that in a simple way.
I have a MySQL table called Moment, which has many millions of rows. (It happens to be a time-series table but that doesn't impact on my problem here; however, these data have a column 'source' and a column 'time', both indexed.) Queries to pull data out of this table are created dynamically (coming in from an API), and ultimately boil down to a small pile of temporary tables indicating which 'source's we care about, and maybe the 'time' ranges we care about.
Let's say we're looking for
(source in Temp1) AND (
((source in Temp2) AND (time > '2017-01-01')) OR
((source in Temp3) AND (time > '2016-11-15'))
)
Just for excitement, let's say Temp2 is empty --- that part of the API request was valid but happened to include 'no actual sources'.
If I then do
SELECT m.* from Moment as m,Temp1,Temp2,Temp3
WHERE (m.source = Temp1.source) AND (
((m.source = Temp2.source) AND (m.time > '2017-01-01')) OR
((m.source = Temp3.source) AND (m.time > '2016-11'15'))
)
... I get a heaping mound of nothing, because the empty Temp2 gives an empty Cartesian product before we get to the WHERE clause.
Okay, I can do
SELECT m.* from Moment as m
LEFT JOIN Temp1 on m.source=Temp1.source
LEFT JOIN Temp2 on m.source=Temp2.source
LEFT JOIN Temp3 on m.source=Temp3.source
WHERE (m.source = Temp1.source) AND (
((m.source = Temp2.source) AND (m.time > '2017-01-01')) OR
((m.source = Temp3.source) AND (m.time > '2016-11-15'))
)
... but this takes >70ms even on my relatively small development database.
If I manually eliminate the empty table,
SELECT m.* from Moment as m,Temp1,Temp3
WHERE (m.source = Temp1.source) AND (
((m.source = Temp3.source) AND (m.time > '2016-11-15'))
)
... it finishes in 10ms. That's the kind of time I'd expect.
I've also tried putting a single unmatchable row in the empty table and doing SELECT DISTINCT, and it splits the difference at ~40ms. Seems an odd solution though.
This really feels like I'm just conceptualizing the query wrong, that I'm asking the database to do more work than it needs to. What is the Right Way to ask the database this question?
Thanks!
--UPDATE--
I did some actual benchmarks on my actual database, and came up with some really unexpected results.
For the scenario above, all tables indexed on the columns being compared, with an empty table,
doing it with left joins took 3.5 minutes (!!!)
doing it without joins (just 'FROM...WHERE') and adding a null row to the empty table, took 3.5 seconds
even more striking, when there wasn't an empty table, but rather ~1000 rows in each of the temporary tables,
doing the whole thing in one query took 28 minutes (!!!!!), but,
doing each of the three AND clauses separately and then doing the final combination in the code took less than a second.
I still feel I'm expressing the query in some foolish way, since again, all I'm trying to do is one set union (OR) and a few set intersections. It really seems like the DB is making this gigantic Cartesian product when it seriously doesn't need to. All in all, as pointed out in the answer below, keeping some of the intelligence up in the code seems to be the better approach here.
There are various ways to tackle the problem. Needless to say it depends on
how many queries are sent to the database,
the amount of data you are processing in a time interval,
how the database backend is configured to manage it.
For your use case, a little more information would be helpful. The optimization of your query by using CASE/COUNT(*) or CASE/LIMIT combinations in queries to sort out empty tables would be one option. However, if-like queries cost more time.
You could split the SQL code to downgrade the scaling of the problem from 1*N^x to y*N^z, where z should be smaller than x.
You said that an API is involved, maybe you are able handle the temporary "no data" tables differently or even don't store them?
Another option would be to enable query caching:
https://dev.mysql.com/doc/refman/5.5/en/query-cache-configuration.html
I'm doing a school project about webapp security and vulnerabilities, and for that i've made my own very simple website with a login and a search bar.
Now, i've made the login silent on purpose, while the search bar shows output on SQL injections. So for demonstration purposes I would like to do a time-based completely blind SQL injection attack on the login... But i'm a bit stuck.
I have no idea how I can enumerate how many columns are in the table for the login query, in a time-based situation:
SELECT * FROM customer WHERE cMAIL='' AND cPWD='';
I cannot do ORDER BY n;#, since I get absolutely no error output when something goes wrong. The only way I can get any indication is through SLEEP() or BENCHMARK().
But I cannot put ORDER BY into a SELECT IF() statement. So how can I find out how many columns exist? (it makes unions impossible for me).
Thank you!
EDIT: It might be worth mentioning, the site is very unsecure on purpose (doesn't use mysqli, just mysql). Since it is just for demonstration
Okay, I found an answer to my own question:
Instead of using ORDER BY, I used SELECT and bruteforced my way to see how many columns existed:
SELECT * FROM customer WHERE cMAIL='' AND cPWD='';
Can be enumerated in the dark by doing:
SELECT * FROM customer WHERE cMAIL='' UNION SELECT null,null,null AND SLEEP(5);# AND cPWD=''
You just keep adding more nulls to the select until the database sleeps for 5 seconds. Then you know how many columns are in the given table.
Hope this can help someone else.
I'd like to select * from 2 tables, but have each table's column name be prefixed with a string, to avoid duplicate column name collissions.
For example, I'd like to have a view like so:
CREATE VIEW view_user_info as (
SELECT
u.*,
ux.*
FROM
user u,
user_ex ux
);
where the results all had each column prefixed with the name of the table:
e.g.
user_ID
user_EMAIL
user_ex_ID
user_ex_TITLE
user_ex_SIN
etc.
I've put a sql fiddle here that has the concept, but not the correct syntax of course (if it's even possible).
I'm using MySql, but would welcome generic solutions if they exist!
EDIT: I am aware that I could alias each of the fields, as mentioned in one of the comments. That's what I'm currently doing, but I find at the start of a project I keep having to sync up my tables and views as they change. I like the views to have everything in them from each table, and then I manually select out what I need. Kind of a lazy approach, but this would allow me to iterate quicker, and only optimize when it's needed.
I find at the start of a project I keep having to sync up my tables and views as they change.
Since the thing you're trying to do is not really supported by standard SQL, and you keep modifying database structures in development, I wonder if your best approach would be to write a little script that recreates that SELECT statement for you. Maybe wrap it in a method call in the development language of your choice?
Essentially you'd need to query INFORMATION_SCHEMA for the tables and columns of interest, probably via a join, and write the results out in SQL style.
Then just run the script every time you make database structural changes that are important to you, and watch your code magically keep up.
I will simplify my problem in order to explain it.
I have a table which contains text messages posted by users and another table which contains keywords.
I want to display, for each user, the number of times keywords are found in text messages.
I don't want the result to display a keyword if it's not found in text messages.
I wan't it to be case INSENSITIVE. All keywords are lowered but in messages, you can find lower & upper chars.
Because I'm not sure that my explanation is clear enough, here comes the SQLFiddle.
http://sqlfiddle.com/#!2/c402a
Hope anyone can help me.
I found what I was looking for. It wasn't easy for me but here is my query :
SELECT t_msg.msg_usr,
t_list.list_word,
count(t_list.list_word),
t_msg.msg_text
FROM t_msg
INNER JOIN t_list
ON LOWER(t_msg.msg_text) LIKE CONCAT("%", t_list.list_word, "%")
GROUP BY t_msg.msg_usr, t_list.list_word;
The SQLFiddle is there : http://sqlfiddle.com/#!2/ba052/8
The recommendation would be to not try solving this with a query. It's possible to write a query that will do it, such query will scan the messages table for each keyword separately, and produce a count (or a row that you can group by), but this won't scale, or be reliable in sense of language search.
Here is what you might want to do:
Create a table to map (user_id, keyword_id) to a count of this keyword in messages of this user. Let's call it t_keyword_count.
Each time you receive a message, before you save the message into the database, search it for all the keywords you care about (using whatever good text search libraries that account for misspellings, etc.). You should know the (user_id) for this message.
You will, at that point, be ready to add the message to the database, and will have an array of (keyword_id) with keywords that this message will have.
In a transaction, insert the message into the t_msg table, and run update/insert for (user_id,keyword_id) to have value=value+1 (or +n, if you need to count the same keyword more than once in the same message) for the t_keyword_count table.
If you are trying to solve the problem of having to do the above on existing data, you can do this manually, just to build up that t_keyword_count table first (depends on how many keywords you have in total, but even if there are a lot, this can be scripted). But you should change (or mirror) the t_msg.msg_text field to be a field suitable for text search, and use SQL text search functionality to find the keywords.