I want to store IP addresses (v4 and v6) in my rails application. I have installed an extension to MySQL adds functions to convert ip strings to binary which will allow me to query by IP range easily.
I can use unescaped sql statements for SELECT type queries, thats easy.
The hard part is that I also need a way to overwrite the way the field is escaped for insert/update statements.
This ActiveRecord statement
new_ip = Ip.new
new_ip.start = '1.2.3.4'
new_ip.save
Should generate the following SQL statement
INSERT INTO ips(start) VALUES(inet6_pton('1.2.3.4'));
Is there a way to do this? I tried many things, including overriding ActiveRecord::Base#arel_attributes_values, without luck : the generated sql is always converted to binary (if that matters, my column is a MySQL VARBINARY(16)).
ActiveRecord::Base.connection.execute("INSERT INTO ips(start) VALUES(inet6_pton('1.2.3.4'))")
Related
Perl Mojolicious use query with placeholder to prevent SQL injection. But the problem is, sometimes I want to see what's the query look like.
Is there a way to print the query with all placeholders replaced with the real values?
I know I can do a replace by myself. But I have to do it every time I want to debug the SQL. It's so cumbersome. If mojolicious has a way like $c->mysql->output_last_sql(), it'll be amazing.
my $sql=q|
SELECT
count(*) cnt
FROM
my_table
WHERE
id= ?
|;
# I know I can do below by myself
print q1|
SELECT
count(*) cnt
FROM
my_table
WHERE
id= $c->param('id')
|;
my $query=$c->mysql->db->query($sql, $c->param('id'));
# how can I print the real SQL with all placeholders replaced?
print $query->hash()->{cnt};
I checked the document of Mojolicious but didn't find anything useful.
https://docs.mojolicious.org/Mojo/Pg/Database
The advantage of query parameters is that they are not simply string-replacements into the SQL query. If they were, they would still cause a risk of SQL injection. The way parameters work is that they are never replaced in your query until after the query is prepared. Parsing occurs during the prepare step, so if parameter values are not combined with the query until after parsing, then there's no way for the values to cause mischief with the SQL syntax.
That means you can't get the SQL combined with its parameters in the client.
The only workaround is to use the query log on the MySQL Server. I give an example here: Getting raw SQL query string from PDO prepared statements. That's about PHP, not Perl, but it works the same regardless of which language you use.
(With exceptions only for client connectors that create fake prepared statements, and actually do interpolate parameters into the SQL string, then send it to the MySQL Server. For example, Python's connector does this by default, for example, but you can optionally make even Python use true prepared statements.)
According to Node.js' mysql package documentation:
In order to avoid SQL Injection attacks, you should always escape any user provided data before using it inside a SQL query. You can do so using the mysql.escape(), connection.escape() or pool.escape() methods.
I cannot find any documentation / reference to using prepared statements with mysql, except for in a reference to using '?' characters. It states the following:
Alternatively, you can use ? characters as placeholders for values you would like to have escaped...
This looks similar to prepared statements in MySQL, however it really
just uses the same connection.escape() method internally.
From my experience with talking to other developers, the general consensus in the developer community is that prepared statements are the ONLY safe way to perform mysql queries from Node.js however, as you can see with the mysql package, there is no obvious support for prepared statements. However, it is indicated that their method for avoiding SQL injection is via the usage of the escape functions.
My Question:
Are the escape functions in the mysql package sufficient enough to securely query a mysql database (without the use of prepared statements)?
Escaping is actually just as safe as using parameterized queries, if you do it consistently.
But it's recommended to use parameters because it makes your code simpler. Therefore developers are probably more likely to do it.
If the Node.js library makes it just as convenient as parameters, but implements it internally by modifying query strings, replacing the ? marks with the parameter values, then you're good.
For what it's worth, Python's MySQL driver does this too, and also PHP's PDO extension when the "emulate prepares" option is in effect. As long as those libraries are well-tested, it should be safe.
FWIW, both escaping and parameters is limited in SQL injection prevention, because in both cases, you can only protect values that you would combine with your SQL query. You cannot protect identifiers (like table names, or column names), or SQL keywords, or expressions, etc. In these cases, just be careful that you have vetted any dynamic content before combining it with your SQL query.
See also:
Preventing SQL injection in Node.js
Difference between real_escape_string and prepare()?
I am using mysqli prepared statements and bound variables.
Then to prevent sql injection, am I need to do anything else(eg: data type validation, filtering, sanitize, string escape etc ) with user input ?
Is there any other way of attacking MySql database other than Sql Injection ?
To prevent SQL injection you have to format your query properly.
Every literal that have to be added to the query dynamically, have to be properly formatted.
Not only data literals like strings and numbers but all of them, including operators and identifiers.
The only proper way to make values formatted is prepared statements.
For the identifiers and operators you will need also filtering, to let only allowed ones into query.
Whatever user input should not be involved at all. It's destination, not source that matters.
Is there any other way of attacking MySql database other than Sql Injection ?
sure thing. But the topic is too broad to make you secured by means of a forum post. Better hire a DBA.
I am creating an ETL in SSIS in which I which I want my data source to be a restricted query, like select * from table_name where id='Variable'. This variable is what I defined as User created variable.
I do not understand how I can have my source query interact with the SSIS scoped Variable.
The only present options are
Table
Table from variable
SQL Command
SQL command from a variable
What I want is to have a SQL statement having a variable as parameter
Simple. Choose SQL command as the Data Access Mode. Enter your query with a question mark as a parameter placeholder. Then click the Parameters button and map your variable to Parameter0 in the Set Query Parameters dialog:
More information is available on MSDN.
An inferior alternative to #Edmund's approach is to use an Expression on another Variable to build your string. Assuming you have #[User::FirstName] already defined, you would then create another variable, #[User::SourceQuery].
In the properties for this variable, set EvaluateAsExpression to True and then set an Expression like "SELECT FirstName, LastName, FROM Person.Person WHERE FirstName = '" + #[User::FirstName] +"'" The double quotes are required because we are building an SSIS String.
There are two big reasons this approach should not be implored.
Caching
This approach is going to bloat your plan cache in SQL Server with N copies of essentially the same query. The first time it runs and the value is "Edmund" SQL Server will create an execution plan and save it (because it can be expensive to build them). You then run the package and the value is "Bill". SQL Server checks to see if it has a plan for this. It doesn't, it only has one for Edmund and so it creates another copy of the plan, this time hard coded to Bill. Lather-rinse-repeat and watch your available memory dwindle until it unloads some plans.
By using the parameter approach, when the plan is submitted to SQL Server, it should be creating a parameterized version of the plan internally and assumes that all parameters supplied will result in equal costing executions. Generally speaking, this is the desired behaviour.
If your database is optimized for ad-hoc workload (it's a setting turned off by default), that should be mitigated as every plan is going to get parameterized.
SQL Injection
The other big nasty you will run into with building your own string is that you open yourself up to SQL Injection attacks or at the least, you can get runtime errors. It's as simple as having a value of "d'Artagnan." That single quote will cause your query to fail resulting in package failure. Changing the value to "';DROP TABLE Person.Person;--" will result in great pain.
You might think it's trivial to safe quote everything but the effort of implementing it consistently everywhere you query is beyond what your employer is paying you. All the more so since there is native functionality provided to do the same thing.
When using OLEDB Connection manager (with SQL Server Native Client 11.0 provider in my case) you can catch an error like this:
Parameters cannot be extracted from the SQL command. The provider
might not help to parse parameter information from the command. In
that case, use the "SQL command from variable" access mode, in which
the entire SQL command is stored in a variable.
So you need to explicitly specify database name in OLEDB Connection manager properties. Otherwise SQL Server Native Client can use different database name then you mean (e.g. master in MSSQL Server).
For some cases you can explicitly specify database name for each database object used in query, e.g.:
select Name
from MyDatabase.MySchema.MyTable
where id = ?
I'm trying to push a brand new Ruby on Rails app to Heroku. Currently, it sits on MySQL. It looks like Heroku doesn't really support MySQL and so we are considering using PostgreSQL, which they DO support.
How difficult should I expect this to be? What do I need to do to make this happen?
Again, please note that my DB as of right now (both development & production) are completely empty.
Common issues:
GROUP BY behavior. PostgreSQL has a rather strict GROUP BY. If you use a GROUP BY clause, then every column in your SELECT must either appear in your GROUP BY or be used in an aggregate function.
Data truncation. MySQL will quietly truncate a long string to fit inside a char(n) column unless your server is in strict mode, PostgreSQL will complain and make you truncate your string yourself.
Quoting is different, MySQL uses backticks for quoting identifiers whereas PostgreSQL uses double quotes.
LIKE is case insensitive in MySQL but not in PostgreSQL. This leads many MySQL users to use LIKE as a case insensitive string equality operator.
(1) will be an issue if you use AR's group method in any of your queries or GROUP BY in any raw SQL. Do some searching for column "X" must appear in the GROUP BY clause or be used in an aggregate function and you'll see some examples and common solutions.
(2) will be an issue if you use string columns anywhere in your application and your models aren't properly validating the length of all incoming string values. Note that creating a string column in Rails without specifying a limit actually creates a varchar(255) column so there actually is an implicit :limit => 255 even though you didn't specify one. An alternative is to use t.text for your strings instead of t.string; this will let you work with arbitrarily large strings without penalty (for PostgreSQL at least). As Erwin notes below (and every other chance he gets), varchar(n) is a bit of an anachronism in the PostgreSQL world.
(3) shouldn't be a problem unless you have raw SQL in your code.
(4) will be an issue if you're using LIKE anywhere in your application. You can fix this one by changing a like b to lower(a) like lower(b) (or upper(a) like upper(b) if you like to shout) or a ilike b but be aware that PostgreSQL's ILIKE is non-standard.
There are other differences that can cause trouble but those seem like the most common issues.
You'll have to review a few things to feel safe:
group calls.
Raw SQL (including any snippets in where calls).
String length validations in your models.
All uses of LIKE.
If you have no data to migrate, it should be as simple as telling your Gemfile to use the pg gem instead, running bundle install, and updating your database.yml file to point to your PostgreSQL databases. Then just run your migrations (rake db:migrate) and everything should work great.
Don't feel you have to migrate to Postgres - there are several MySQL Addon providers available on Heroku - http://addons.heroku.com/cleardb is the one I've had the most success with.
It should be simplicity itself: port the DDL from MySQL to PostgreSQL.
Does Heroku have any schema creation scripts? I'd depend on those if they were available.
MySQL and PostgreSQL are different (e.g. identity type for MySQL, sequences for PostgreSQL). But the port shouldn't be too hard. How many tables? Tens are doable.