I had an model with an "end" column (datetime format), only to discover that Heroku crashes and burns with illogical Active Record errors whenever I attempted to reference the column in a query. I spent two hours trying to debug the extremely simple query, after which point I renamed the column to "end_at" and all of my problems disappeared.
Has anybody else experienced this issue? I'm curious of the reasoning behind this and hope that we can help others avoid the same mistake. A similar question has been asked before, but a clear answer was not presented.
BEGIN and END are reserved words in Oracle and SQL Server, but not sure why MySQL doesn't consider them as such.
However that PGError would appear to indicate that the database engine itself (and not any Ruby-related runtime) has indeed rejected the query because of the "end".
Reserved words (and names containing spaces) can be used if quoted - perhaps Active Record didn't quote the identifiers in the SQL which was generated.
I would look at the log in MySQL (http://dev.mysql.com/doc/refman/5.5/en/query-log.html) and see the statements generated.
And since the PGError means PostGreSQL and you mentioned Heroku (PostgreSQL 8.3) - I think this is because END is indeed a reserved word in PostgreSQL: http://www.postgresql.org/docs/8.3/static/sql-keywords-appendix.html
http://www.petefreitag.com/tools/sql_reserved_words_checker/?word=end
PostgresQL (which Heroku uses) reserves END as a keyword, so it is giving you a syntax error because your syntax is incorrect.
There are two options for fixing it:
If Heroku is breaking because ActiveRecord is not quoting column names, you can rewrite every query that uses that model to explicitly quote the "end" column so PostgresQL doesn't blow up.
Whether or not ActiveRecord quotes that column name by default, renaming the column something more descriptive (end_time, end_date, etc) and also not a reserved word in both the language you are writing the app in as well as a reserved word in the SQL engine you are using.
Related
Say I have a table call ‘users’ with a column ‘order’ and I wish my application code to work across all the database systems out there.
Assume that it is impractical to change the database schema. Even if the schema was change, one of the main database engine will come up with yet another reserved word, so stopping the app working when a database engine is updated.
I would rather not have to process the SQL strings to convert them into the correct form for each database.
'How to find if a column name is a reserved keyword across various databases' partly overlaps with this question.
('Syntax error due to using a reserved word as a table or column name in MySQL' is the reference question for MySQL.
)
You have to test on everything you are going to support, so "all the database systems out there" is not in realty an option.
To be more realistic, you can decide which DBMS you want to support, and write a subset of SQL which they are all happy with, and test them with automated tests.
For example, you might consider:
MySQL
Postgres
MS SQL Server
Oracle
I believe these all support ANSI double-quotes for quoting names, so if you quote all names all the time then you have one less thing to worry about.
I have a large file that contains a mysqldump of a database. I searched for all the semicolons I come up with 378. When I upload the file to phpMyAdmin it reports 379 queries when doing an import. (I know that phpMyAdmin might not be reliable, but I just want to make sure I don't have a problem with the output of mysqldump)
Is there a way to find out what it thinks the extra query is? The dump file is pretty large and I don't want to share the information.
EDIT:
I have posted the database schema at:
http://blastohosting.com/pos_database.sql
It would be great if it could be determined to as why there is an extra query being reported by phpMyAdmin
I can't tell you why, necessarily, but I can say that I got exactly the same results: 378 semicolons, "379 queries executed".
However, here's the weird thing: when I removed the last two lines in the script:
(blank)
-- Dump completed on 2014-02-17 17:52:43
...PMA said "378 queries executed". It's evidently counting that final comment as a "query" even though it's ignored.
A dump file doesn't necessarily contain one semicolon per statement.
Any CREATE TRIGGER or CREATE PROCEDURE or CREATE FUNCTION statements have literal semicolons within their bodies, so it is common practice to change the client's statement terminator to something besides semicolon, to resolve the ambiguity.
So you could be counting semicolons that aren't statement terminators. Likewise, MySQL may recognize more SQL statements executed that didn't use semicolon.
MySQL builtin commands such as DELIMITER, CHARSET, USE, SOURCE, etc. don't need a terminator at all, but they may have one without error. They shouldn't be reported as statements executed by the server, but they may throw off your count of semicolons.
Do you have any semicolons inside string literals or SQL comments?
I have a simple web form and a simple backend table in MySQL. I am seeing syntax errors in certain posts due to the presences of special characters.
I am using stripslashes($_POST['form_field']) but I still get errors writing to the database.
I am looking for some quick help here to see if anyone knows of the right PHP function to better cleanse my incoming data prior to writing to the table.
I prefer not to test my form, one-by-one, with every single special character (32 in all) to catch and isolate those that are the culprits.
Thanks.
have a look at mysql_real_escape_string() - that usually works well.
I'm trying to push a brand new Ruby on Rails app to Heroku. Currently, it sits on MySQL. It looks like Heroku doesn't really support MySQL and so we are considering using PostgreSQL, which they DO support.
How difficult should I expect this to be? What do I need to do to make this happen?
Again, please note that my DB as of right now (both development & production) are completely empty.
Common issues:
GROUP BY behavior. PostgreSQL has a rather strict GROUP BY. If you use a GROUP BY clause, then every column in your SELECT must either appear in your GROUP BY or be used in an aggregate function.
Data truncation. MySQL will quietly truncate a long string to fit inside a char(n) column unless your server is in strict mode, PostgreSQL will complain and make you truncate your string yourself.
Quoting is different, MySQL uses backticks for quoting identifiers whereas PostgreSQL uses double quotes.
LIKE is case insensitive in MySQL but not in PostgreSQL. This leads many MySQL users to use LIKE as a case insensitive string equality operator.
(1) will be an issue if you use AR's group method in any of your queries or GROUP BY in any raw SQL. Do some searching for column "X" must appear in the GROUP BY clause or be used in an aggregate function and you'll see some examples and common solutions.
(2) will be an issue if you use string columns anywhere in your application and your models aren't properly validating the length of all incoming string values. Note that creating a string column in Rails without specifying a limit actually creates a varchar(255) column so there actually is an implicit :limit => 255 even though you didn't specify one. An alternative is to use t.text for your strings instead of t.string; this will let you work with arbitrarily large strings without penalty (for PostgreSQL at least). As Erwin notes below (and every other chance he gets), varchar(n) is a bit of an anachronism in the PostgreSQL world.
(3) shouldn't be a problem unless you have raw SQL in your code.
(4) will be an issue if you're using LIKE anywhere in your application. You can fix this one by changing a like b to lower(a) like lower(b) (or upper(a) like upper(b) if you like to shout) or a ilike b but be aware that PostgreSQL's ILIKE is non-standard.
There are other differences that can cause trouble but those seem like the most common issues.
You'll have to review a few things to feel safe:
group calls.
Raw SQL (including any snippets in where calls).
String length validations in your models.
All uses of LIKE.
If you have no data to migrate, it should be as simple as telling your Gemfile to use the pg gem instead, running bundle install, and updating your database.yml file to point to your PostgreSQL databases. Then just run your migrations (rake db:migrate) and everything should work great.
Don't feel you have to migrate to Postgres - there are several MySQL Addon providers available on Heroku - http://addons.heroku.com/cleardb is the one I've had the most success with.
It should be simplicity itself: port the DDL from MySQL to PostgreSQL.
Does Heroku have any schema creation scripts? I'd depend on those if they were available.
MySQL and PostgreSQL are different (e.g. identity type for MySQL, sequences for PostgreSQL). But the port shouldn't be too hard. How many tables? Tens are doable.
I have a DB called:
MY_XXXX and MY_YYYY.
When I go to the phpMyAdmin, it shows in the tree structure...like this:
So, my questions is, is the MySQL or the phpMyAdmin do this things for me...? Is this have some special meaning for that? Thank you.
This is phpMyAdmin making a (sort of reasonable-ish, a bit, actually largely un-warranted) assumption about a naming convention - it's nothing to do with MySQL.
If your looking for more information on such things - see the MySQL Schema Object Names documentation. (As you'll see - it defines no special meaning to the underscore other than stating that it's considered a valid character.)
THis means you have multiple databases starting with MY .
So rather then showing showing it in different headings as MY_1, MY_2 phpMyAdmin shows them as
MY_
1
2
etc.