I'm using a MySql database and was trying to find a MySQL alternative to tedious.js (a SQL server parameterised query builder).I'm using Node.js for my backend.
I read that the .raw() command from knex.js is susceptible to sql injection, if not used with bindings.
But are the other commands and knex.js as a whole safe to use to prevent sql injection? Or am I barking up the wrong tree?
Read carefully from knex documentation how to pass values to knex raw (http://knexjs.org/#Raw).
If you are passing values as parameter binding to raw like:
knex.raw('select * from foo where id = ?', [1])
In that case parameters and query string are passed separately to database driver protecting query from SQL injection.
Other query builder methods always uses binding format internally so they are safe too.
To see how certain query is passed to database driver one can do:
knex('foo').where('id', 1).toSQL().toNative()
Which will output SQL string and bindings that are given to driver for running the query (https://runkit.com/embed/2yhqebv6pte6).
Biggest mistake that one can do with knex raw queries is to use javascript template string and interpolate variables directly to SQL string format like:
knex.raw(`select * from foo where id = ${id}`) // NEVER DO THIS
One thing to note is that knex table/identifier names cannot be passed as bindings to driver, so with those one should be extra careful to not read table / column names from user and use them without properly validating them first.
Edit:
By saying that identifier names cannot be passed as bindings I mean that when one is using ?? knex -binding for identifier name, that will be rendered as part of SQL string when passed to the database driver.
Is there any equivalent for the Oracle DBMS function DBMS_LOB.SUBSTRING on Mysql?
if not how can I get the text of BLOB in MYSQL server? (I only need to use SQL no other programming languages)
Please try dbms_lob.substr which is equivalent to DBMS_LOB.SUBSTRING
Get the Oracle value passed to dbms_lob.substr and prefix the value with 0x without quotes for the MySQL insert statement.
I did tried for my case and succeeded.
I have a mysql query that looks like this $query="SELECT * FROM #__content".
What does the #__ at the start of the table name mean?
#__ is a prefix of your tables
#__ is simply the database table prefix and is defined in you configuration.php
If it wasn't defined, people would have to manually have to input their prefixes into every extension that requires access to the database, which you can imagine would be annoying.
So for example, if you database table prefix is j25, then:
#__content = j25_content
As others have said, hash underscore sequence '#_' is the prefix used for table names by Joomla!'s JDatabase class. (N.B. there is only one underscore, the second underscore is maintained to for readability in table names.)
When you first setup Joomla! you are given the option of setting a prefix or using the one randomly generated at the time. You can read about how to check the prefix here.
When you access the database using the JDatabase class it provides you with an abstraction mechanism so that you can interact with the database that Joomla is using without you having to code specifically for MySQL or MSSQL or PostgreSQL etc.
When JDatabase prepares a query prior to executing it, it replaces any occurrences #_ in the from segment of the query with the prefix setup when Joomla! was installed. e.g.
// Get the global DB object.
$db = JFactory::getDBO();
// Create a new query object.
$query = $db->getQuery(true);
// Select some fields
$query->select('*');
// Set the from From segment
$query->from('#__myComponents_Table');
Later when you execute the query JDatabase will change the from segment of the SQL from
from #__myComponents_Table to
from jp25_myComponents_Table — if the prefix is jp25, prior to executing it.
I have a column value stored in the database as:
a:2:{i:0;s:2:"US";i:1;s:2:"19";}
I want to unserialize it during the mysql query rather than using the php unserialize function after fetching the data.
I want to do it this way so I can join another table with the serialized value. This would avoid executing a separate query after unserializing it with php, just for the joined data.
MySQL doesn't know what a PHP serialization is. You can't do it.
You can use SUBSTRING_INDEX
For example, if you have a record like this:
a:5:{s:9:"invoiceid";s:1:"8";s:8:"balance";i:5;s:14:"broughtforward";i:3;s:6:"userid";s:5:"13908";s:10:"customerid";s:1:"3";}
You can use the below SELECT statement:
SELECT SUBSTRING_INDEX(SUBSTRING_INDEX(old_data,';',1),':',-1) AS fieldname1,
SUBSTRING_INDEX(SUBSTRING_INDEX(old_data,';',2),':',-1) AS fieldvalue1,
SUBSTRING_INDEX(SUBSTRING_INDEX(old_data,';',3),':',-1) AS fieldname2,
SUBSTRING_INDEX(SUBSTRING_INDEX(old_data,';',4),':',-1) AS fieldvalue2,
SUBSTRING_INDEX(SUBSTRING_INDEX(old_data,';',5),':',-1) AS fieldname3,
SUBSTRING_INDEX(SUBSTRING_INDEX(old_data,';',6),':',-1) AS fieldvalue3,
SUBSTRING_INDEX(SUBSTRING_INDEX(old_data,';',7),':',-1) AS fieldname4,
SUBSTRING_INDEX(SUBSTRING_INDEX(old_data,';',8),':',-1) AS fieldvalue4,
SUBSTRING_INDEX(SUBSTRING_INDEX(old_data,';',9),':',-1) AS fieldname5,
SUBSTRING_INDEX(SUBSTRING_INDEX(old_data,';',10),':',-1) AS fieldvalue5
FROM table;
Check this for reference: How to unserialize data using mysql without using php
How about this? This is a MySQL user-defined function with embedded php:
CREATE FUNCTION unserialize_php RETURNS STRING SONAME 'unserialize_php.so';
Usage example:
SELECT unserialize_php('O:8:"stdClass":2:{s:1:"a";s:4:"aaaa";s:1:"b";s:4:"bbbb";}', "$obj->a")
AS 'unserialized';
+--------------+
| unserialized |
+--------------+
| aaaa |
+--------------+
1 row in set (0.00 sec)
drop function unserialize_php;
Source: https://github.com/junamai2000/mysql_unserialize_php
You can create a MySQL user-defined function and call zend_eval_string inside of the function so that you can bring back PHP variables to a MySQL result. I implemented a sample program. You can try it.
From http://www.blastar.biz/2013/11/28/how-to-use-mysql-to-search-in-php-serialized-fields/
Standard array
SELECT * FROM table WHERE your_field_here REGEXP '.*;s:[0-9]+:"your_value_here".*'
Associative array
SELECT * FROM table WHERE your_field_here REGEXP '.*"array_key_here";s [0-9]+:"your_value_here".*'
It's a very bad practice to add programming language dependent structures to database. If you do so, you always have to rely on that language.
The best approach is to have normalized table structure (different fields or tables).
The next approach is to save data as a delimited string (e.g.: 0,US,1,19). Then you can use MySQL's SUBSTRING() or to use standard serialization mechanisms like JSON encode.
As mentioned by kchteam, MySQLToolBox library comes handy for this purpose using a custom defined MySQL function getPhpSerializedArrayValueByKey available here https://github.com/KredytyChwilowki/MySQLToolBox/blob/master/getPhpSerializedArrayValueByKey.sql.
After adding this function, you can retrieve any value in the serialized array by using the following syntax,
SELECT getPhpSerializedArrayValueByKey(column_name, 'array_key') AS deseializedArrayValue FROM table_name
The given array can be unserialized like,
SELECT getPhpSerializedArrayValueByKey('a:2:{i:0;s:2:"US";i:1;s:2:"19";}
', 'key_to_retrieve') AS key_to_retrieve
For serialized arrays You can use function getPhpSerializedArrayValueByKey from here
You can join your table simply in this way
SELECT
table_to_join.ID as table_to_join_ID ,
serialized_table.ID AS serialized_table_ID,
FROM
table_to_join
LEFT JOIN
serialized_table ON serialized_table.array_field REGEXP CONCAT_WS('','.s:[0-9];s:', table_to_join.ID ,';.') ;
Take mention. I use index from 0 to 9 in table. If you have other indexes you must correct regexp
Some days ago I asked a question about my problem and I was advised to use CONCAT_WS function. I am using CONCAT_WS on my local mysql database and it is working perfectly. But it is not working on server(application hosted) and generate the following error.
FUNCTION test.CONCAT_WS does not exist
Here test in error string is my database name on server.
My query is like this:
SELECT * FROM patient WHERE CONCAT_WS (',', LastName,FirstName,BirthDate ) NOT IN ('Abdul,Quddus,2000-09-30','Wasim,Akram,1993-09-12');
Can someone tell me the problem or suggest me another solution asked in linked question above ?
Thanks
The easiest way to fix it is by removing the whitespace between the function name and the parenthesis, i.e. CONCAT_WS(...) instead of CONCAT_WS (...).
From the MySQL Manual:
By default, there must be no
whitespace between a function name and
the parenthesis following it. This
helps the MySQL parser distinguish
between function calls and references
to tables or columns that happen to
have the same name as a function.
...
You can tell the MySQL server to
accept spaces after function names by
starting it with the
--sql-mode=IGNORE_SPACE option.
Also, this behavior depends on the MySQL version, this is why it works on one server and doesn't work on another, quote from the "Function Name Parsing and Resolution" manual page:
The number of function names affected
by IGNORE_SPACE was reduced
significantly in MySQL 5.1.13, from
about 200 to about 30.