i have 2 dbs: one in mySql and one in MongoDB with the same data inside...
i do the follow in mySQL:
Select tweet.testo From tweet Where tweet.testo like ‘%pizza%’
and this is the result:
1627 rows in set (2.79 sec)
but if i exec in mongo:
Db.tweets.find({text: /pizza/ }).explain()
this is the result:
nscannedObjects" : 1606334,
"n" : 1169,
or if i exec:
Db.tweets.find({text: /pizza/i }).explain()
this is the result:
"nscannedObjects" : 1606334,
"n" : 1641,
Why the number of rows/document in mysql/mongo find is different?
Why the number of rows/document in mysql/mongo find is different??
There could be 1000000000000000 reasons including the temperature of the sun on that particular day.
MongoDB and MySQL are two completely separate techs as such if you expect to keep both in synch you will need some kind of replicator between the two. You have not made us aware as to whether this is the case.
Also we have no idea of your coding, server setup, network setup and everything else so really we cannot even begin to answer this.
A good answer would be to say that the reason you are seeing this is because the data between the two is different...
As for the difference between:
Db.tweets.find({text: /pizza/ }).explain()
and
Db.tweets.find({text: /pizza/i }).explain()
This is because MySQL, by default, queries in lower case I believe and MongoDB (I know) does not as such it is case sensitive (this i makes it case insensitive).
However about replicators, here is a good one: https://docs.continuent.com/wiki/display/TEDOC/Replicating+from+MySQL+to+MongoDB
the mysql command
Select tweet.testo From tweet Where tweet.testo like ‘%pizza%’
is equivalent to MongoDB's
Db.tweets.find({text: /pizza/i })
I realized they both contain the same data, but in some cases the text in mysql was cut-off, so it resulted in less rows being returned.
To begin with your SQL query like '%pizza%' may not pickup entries that begin with the string 'pizza' because of the wildcard on the front. Try the following SQL query to rule out any syntactical differences with the matching logic in SQL and the Regex used by MongoDB
Select tweet.testo From tweet Where lower(tweet.testo) like ‘%pizza%’ or lower(tweet.testo) like ‘pizza%’
Disclaimer: I don't have mySQL in front of me just now so can't verify the leading wildcard behaviour described above, however this is consistent with other RDBMS so it's worth checking
Related
I was solving one of TryHackMe's rooms about SQL injection.But I couldn't figured out one thing that came to my mind and after spending lots of time I thought it's best to ask that question here. In the room there is a machine to deploy, after deployed machine it gave me an interface(webapp) that takes inputs from me like so :
And if you give it a value like test. It returns following output:
When I saw it, I thought it was an easy SQLi question so I tried most basic SQLi payloads for common dbms like so:
' ; sleep(1) --
' or 1=1 -- ...
But none of them worked and returned me an error message like that:
After that failure, I run sqlmap and it found 2 types of SQLi payload with following payloads:
Then I realized that in time based SQLi, sqlmap used ||. Then I tried to send '|| (select sleep(2)) -- . And it worked. Now my question is why my first payload(' ; select sleep(2) -- ) didn't work, but this one worked ?
By default, MySQL query interfaces do not support multiple queries separated by semicolon. To get that, the application would have to explicitly request this feature to be enabled when it connects to the database. There's a connect option for that.
So by default, any type of SQL injection that works by trying to append a malicious query after a semicolon doesn't work.
But an SQL injection that works by modifying an expression in the single query can still work.
I'm stumped by this:
Two commands:
SELECT Date,Sentence FROM exampletable;
SELECT Date,Sentence FROM exampletable WHERE MATCH (Sentence) AGAINST ("South" IN NATURAL LANGUAGE MODE );
The first gives me results for the entire database, beginning in 2013. I see that there's an entry sometime in 2018 that contains the word "South", so using a match against in the second command I know I should get at least one result pre 2020. However, the first result is from 2020-01-28 onwards.
This happens in all examples I try. Simply adding a match against limits my returned results to > 2020. Is there some database setting that I'm not aware of? Or just something plainly obvious?
Any help would be appreciated! (I'm using MySQL 5.7)
Your query has no order by. Have you tried something like this?
SELECT Date, Sentence
FROM exampletable
WHERE MATCH (Sentence) AGAINST ("South" IN NATURAL LANGUAGE MODE )
ORDER BY date;
Perhaps the earlier dates are just later in the result set.
I'm using the Wordnet SQL database from here: http://wnsqlbuilder.sourceforge.net
It's all built fine and users with appropriate privileges have been set.
I'm trying to find synonyms of words and have tried to use the two example statements at the bottom of this page: http://wnsqlbuilder.sourceforge.net/sql-links.html
SELECT synsetid,dest.lemma,SUBSTRING(src.definition FROM 1 FOR 60) FROM wordsXsensesXsynsets AS src INNER JOIN wordsXsensesXsynsets AS dest USING(synsetid) WHERE src.lemma = 'option' AND dest.lemma <> 'option'
SELECT synsetid,lemma,SUBSTRING(definition FROM 1 FOR 60) FROM wordsXsensesXsynsets WHERE synsetid IN ( SELECT synsetid FROM wordsXsensesXsynsets WHERE lemma = 'option') AND lemma <> 'option' ORDER BY synsetid
However, they never complete. At least not in any reasonable amount of time and I have had to cancel all of the queries. All other queries seem to work find and when I break up the second SQL example, I can get the individual parts to work and complete in reasonable times (about 0.40 seconds)
When I try and run the full statement however, the MySQL command line client just hangs.
Is there a problem with this syntax? What is causing it to take so long?
EDIT:
Output of "EXPLAIN SELECT ..."
Output of "EXPLAIN EXTENDED ...; SHOW WARNINGS;"
I did more digging and looking into the various statements used and found the problem was in the IN command.
MySQL repeats the statement for every single row in the database. This is the cause of the hang, as it had to run through hundreds of thousands of records.
My remedy to this was to split the command into two separate database calls first getting the synsets, and then dynamically creating a bound SQL string to look for the words in the synsets.
I have a portion of a MySQL query which is designed to save a password using SHA512-CRYPT:
SELECT ENCRYPT('firstpassword', CONCAT('$6$', SUBSTRING(SHA(RAND()), -16)))
I need to port the application to use PostgreSQL, and as such, the statement is not compatible.
My attempt in PostgreSQL is like so:
SELECT CRYPT('firstpassword'::text, CONCAT('$6$', SUBSTRING(ENCODE(DIGEST(RANDOM()::text , 'sha1'), 'hex') FROM '.{16}$')))
When tested in component parts, each of these implementations appear identical, but as completed statements, the output differs.
I have found that the CONCAT statements (that generate the salt) appear to provide identical output.
If I try comparing the output of CRYPT or ENCRYPT using simple plaintext words, the output is identical. However, if I combine it with the output of a salt, the output differs;
MySQL:
SELECT ENCRYPT( 'firstpassword', '$6$ae73a5ca7d3e5b11' )
Produces: $6$ae73a5ca7d3e5b11$v/RbcEEx4VR37VMUF6gBnPNo2ptSyU...
PostgreSQL:
SELECT CRYPT('firstpassword'::text, '$6$ae73a5ca7d3e5b11'::text)
Produces: $6eTK2KpfoaQM
Can someone explain why these statements are diverging or suggest a better way to implement this MySQL query?
They diverge because they use different encryption algorithms.
The first thing you want to do is to get very familiar with the pg_crypto documentation. See http://www.postgresql.org/docs/9.2/static/pgcrypto.html
My suspicion is that you probably want to switch from something like encrypt/crypt to the use of SHA-2 for the actual password hashing so you can specifically control the algorithms and ensure compatibility on both sides.
SHA512-CRYPT is not supported by PostgreSQL core or the pgcrypto extension shipped with the main sources. The crypt function shown in the question comes from pgcrypto and does only support bf, md5, des and xdes (see "supported algorithm" in the documentation).
But you may use the shacrypt extension:
test=# create extension shacrypt ; -- (after installing from the sources)
CREATE EXTENSION
test=# select sha512_crypt('firstpassword', '$6$ae73a5ca7d3e5b11' );
sha512_crypt
------------------------------------------------------------------------------------------------------------
$6$ae73a5ca7d3e5b11$v/RbcEEx4VR37VMUF6gBnPNo2ptSyU3ys1sg6i8hhBrcfBeLY6hpsfvXR67bwwjMainpPEaLkYV6eO0ow0xVH.
which gives the same result as MySQL's SELECT ENCRYPT( 'firstpassword', '$6$ae73a5ca7d3e5b11' )
The following is generated query from Hibernate (except I replaced the list of fields with *):
select *
from
resource resource0_,
resourceOrganization resourceor1_
where
resource0_.active=1
and resource0_.published=1
and (
resource0_.resourcePublic=1
or resourceor1_.resource_id=resource0_.id
and resourceor1_.organization_id=2
and (
resourceor1_.resource_id=resource0_.id
and resourceor1_.forever=1
or resourceor1_.resource_id=resource0_.id
and (
current_date between resourceor1_.startDate and resourceor1_.endDate
)
)
)
Currently I have 200+ records in both the Windows and Linux databases and currently for each record, the following happens to be true:
active = 1
published = 1
resourcePublic = 1
When I run this directly in a SQL client, this SQL query gets me all the matching records on Windows but none on Linux. I've MySQL 5.1 on both Windows and Linux.
If I apply the Boolean logic, (true and true and (true or whatever)), I expect the outcome to be true. It indeed is true on Windows but false on Linux!!!
If I modify the query as the following, it works on both Windows and Linux:
select *
from
resource resource0_
where
resource0_.active=1
and resource0_.published=1
and (
resource0_.resourcePublic=1
)
So, just the presence of conditions related to resourceOrganization is making the query bring 0 results on Linux and I expected that since it is the second part of an 'or' condition whose first part is true, the outcome should be true.
Any idea why this difference in behavior between the 2 OSs and why what should obviously work on Linux doesn't!
Thanks in advance!
Check the case sensitivity and collation sets (Collation issues)
Check the table case sensitivity. In particular note that on windows the table names are case-insensitive and on Linux they are case-sensitive.
Have you tried a simple test case on both system?
Check that current_date() returns the same format in both plataforms
I notice that the second test query only consults the resource table not the resourceOrganisation table.
I suspect that the table resourceOrganisation is populated differently on the two machines, and the corresponding rows may not exist in your Linux MySQL.
What does this query return?
select *
from
resource resource0_,
resourceOrganization resourceor1_
where
resource0_.active=1
and resource0_.published=1
and (
resource0_.resourcePublic=1
or resourceor1_.resource_id=resource0_.id
and resourceor1_.organization_id=2
)
Also don't forget to check the collation and case sensitivity, if one server uses a different collation to the other then you will have this same issue.