I have SQLite and MySQL installed on my local and development machine respectively. Following is working fine on my local machine(with SQLite):
select_single = {'date': "strftime('%%Y-%%m-%%d',projectName_Modelname.created)"}
queryset.extra(select=select_single)
But since strftime doesn't work with MySQL(link), I tried using DATE_FORMAT() as suggested in given link and other places too.
Though now when I execute below:
select_single = {'date': "DATE_FORMAT(projectName_Modelname.created, '%%Y-%%m-%%d')"}
queryset.extra(select=select_single)
Following error comes:
DatabaseError: (1054, "Unknown column 'projectName_Modelname.created' in 'field list'")
where 'created' is Datetime field in Django model 'Modelname' of app 'projectName'
To debug when I replace projectName_Modelname.created with NOW() no error comes. I have also tried just Modelname.created instead of projectName_Modelname.created though with no benefit?
Note: I am using Django1.5.5
I think it should be something like:
date_raw_query = {'date': "date_format(created, '%%Y-%%m-%%d')"}
and then try
queryset.extra(select=date_raw_query)
Hope that works in your setup. I have tried this on Django 1.7 and MySQL and seems to be working.
Also remember that if SQL errors start coming up, you can always do a print queryset.extra(select=date_raw_query).query to see what might be going wrong.
And when it comes to writing compatible code between SQLite and MySQL like this one, writing a custom MySQL function has been suggested here
But I would suggest otherwise. It's better to have a similar dev environment with MySQL setup in local and also, upgrade Django as soon as possible. :P
Related
I'm getting the following error message when trying to reflect any of my SQL views:
sqlalchemy/dialects/mysql/reflection.py", line 306, in _describe_to_create
buffer.append(" ".join(line))
TypeError: sequence item 2: expected str instance, bytes found
I have tried using both the autoload_with and autoload=True options in my select query constructor to no avail.
I have the appropriate permissions on my view. My query is pretty simple:
company_country = Table('company_country', metadata, autoload_with=engine)
query = select(company_country.c.country)
return query
I've tried the inspect utility and it does not list my SQL view, nor does the reflecting all tables described below the views section on this page: https://docs.sqlalchemy.org/en/14/core/reflection.html#reflecting-views
I'm using version SQLAlchemy->1.4.32, Python 3.x and mySQL 8.0.28 on Mac if that's any help
I should add that I can query my SQL views using the text() constructor but it would be far more preferable to use select() if possible.
Any tips appreciated
I was using the mysql-connector client for interop with other code, but after switching to the mysqlclient, I was able to reflect the views.
https://docs.sqlalchemy.org/en/14/dialects/mysql.html#module-sqlalchemy.dialects.mysql.mysqldb
I have a database with a lot of nonmanaged tables which I'm using for a django app. For testing I'm wanting to use the --keepdb option so that I don't have to repopulate these tables every time. I'm using MariaDB for my database. If I don't use the keepdb option everything works fine, the test database gets created and destroyed properly.
But when I try to run the test keeping the database:
$ python manage.py test --keepdb
I get the following error:
Using existing test database for alias 'default'...
Got an error creating the test database: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'CREATE DATABASE IF NOT EXISTS test_livedb ;\n SET sql_note' at line 2")
I assume that this is an issue with a different syntax between MariaDB and MySQL. Is there anyway to get the keepdb option to work with MariaDB?
thanks very much!
For what it's worth: This bug was introduced in Django 2.0.0 and fixed in Django 2.1.3 (https://code.djangoproject.com/ticket/29827)
Two things - check out Factory Boy (for creating test data) and I would suggest checking out Pytest as well. With non-managed tables, the issue I think you'll run into is that (at least in my experience) django won't create them in the test environment and you end up running into issues because there is no migration file to create those tables (since they're unmanaged). Django runs the migration files when creating the test environment.
With Pytest you can run with a --nomigrations flag which builds your test database directly off the models (thus creating the tables you need for your unmanaged models).
If you combine Pytest and Factory Boy you should be able to come up with the ability to setup your test data so it works as expected, is repeatable and testable without issue.
I actually approach it like this (slightly hacky, but it works with our complex setup):
On my model:
class Meta(object):
db_table = 'my_custom_table'
managed = getattr(settings, 'UNDER_TEST', False)
I create the UNDER_TEST variable in settings.py like this:
# Create global variable that will tell if our application is under test
UNDER_TEST = (len(sys.argv) > 1 and sys.argv[1] == 'test')
That way - when the application is UNDER_TEST the model is marked as managed (and Pytest will create the appropriate DB table). Then FactoryBoy handles putting all my test data into that table (either in setUp of the test or elsewhere) so I can test against it.
That's my suggestion - others might have something a little more clear or cleaner.
I'm experiencing a strange bug with the way Django Test framework operates.
When using SQLite Database Backend, all of the tests crash with the following error:
File "[]/core/tests/test_admin.py", line 91, in setUpSomething
content_type = ContentType.objects.get(app_label='core', model='SomeModel')
File "[]/lib/python2.7/site-packages/django/db/models/manager.py", line 151, in get
return self.get_queryset().get(*args, **kwargs)
File "[]/lib/python2.7/site-packages/django/db/models/query.py", line 310, in get
self.model._meta.object_name)
DoesNotExist: ContentType matching query does not exist.
However, the same code executes well under MySQL backend.
Clearly Django should make these functions agnostic of the backend used?
I had the same problem and I have no idea if my solution will be helpful, but it solved my issue, so here goes.
In my application code, I was attempting to query ContentType instances in the following way:
email = ContentType.objects.get(app_label="users", model="EmailAddress")
This worked fine with our actual MySQL database, but failed in test under a SQLite test database. However, if I switched the model definition to lowercase, it worked in both places:
email = ContentType.objects.get(app_label="users", model="emailaddress")
My guess was that this may have to do with the default collation in MySQL of case-insensitivity, so the first query should not have worked if I were comparing case-sensitively.
Indeed, when I looked at my database, all the model labels in the django_content_type table were lowercase and SQLite cares (by default) about case, so my queries in my tests were legitimately failing.
I have the following code that run on heroku inside a controller that intermittently fails. It's a no-brainer that it should work to me, but I must be missing something.
#artist = Artist.find(params[:artist_id])
The parameters hash looks like this:
{"utf8"=>"������",
"authenticity_token"=>"XXXXXXXXXXXXXXX",
"password"=>"[FILTERED]",
"commit"=>"Download",
"action"=>"show",
"controller"=>"albums",
"artist_id"=>"62",
"id"=>"157"}
The error I get looks like this:
ActiveRecord::StatementInvalid: Mysql::Error: : SELECT `artists`.* FROM `artists` WHERE `artists`.`id` = ? LIMIT 1
notice the WHEREartists.id= ? part of the statement? It's trying to find an ID of QUESTION MARK. Meaning Rails is not passing in the params[:artist_id] which is obviously in the params hash. I'm at complete loss.
I get the same error on different pages trying to select the record in a similar fashion.
My environment: Cedar Stack on Heroku (this only happens on Heroku), Ruby 1.9.3, Rails 3.2.8, files being hosted on Amazon S3 (though I doubt it matters), using the mysql gem (not mysql2, which doesn't work at all), ClearDB MySQL database.
Here's the full trace.
Any help would be tremendously appreciated.
try sql?
If it's just this one statement, and it's causing production problems, can you omit the query generator just for now? In other words, for very short term, just write the SQL yourself. This will buy you a bit of time.
# All on one line:
Artist.find_by_sql
"SELECT `artists`.* FROM `artists`
WHERE `artists`.`id` = #{params[:artist_id].to_i} LIMIT 1"
ARel/MySQL explain?
Rails can help explain what MySQL is trying to do:
Artist.find(params[:artist_id]).explain
http://weblog.rubyonrails.org/2011/12/6/what-s-new-in-edge-rails-explain/
Perhaps you can discover some kind of difference between the queries that are succeeding vs. failing, such as how the explain uses indexes or optimizations.
mysql2 gem?
Can you try changing from the mysql gem to the mysql2 gem? What failure do you get when you switch to the mysql2 gem?
volatility?
Perhaps there's something else changing the params hash on the fly, so you see it when you print it, but it's changed by the time the query runs?
Try assigning the variable as soon as you receive the params:
artist_id = params[:artist_id]
... whatever code here...
#artist = Artist.find(artist_id)
not the params hash?
You wrote "Meaning Rails is not passing in the params[:artist_id] which is obviously in the params hash." I don't think that's the problem-- I expect that you're seeing this because Rails is using the "?" as a placeholder for a prepared statement.
To find out, run the commands suggested by #Mori and compare them; they should be the same.
Article.find(42).to_sql
Article.find(params[:artist_id]).to_sql
prepared statements?
Could be a prepared statement cache problem, when the query is actually executed.
Here's the code that is failing-- and there's a big fat warning.
begin
stmt.execute(*binds.map { |col, val| type_cast(val, col) })
rescue Mysql::Error => e
# Older versions of MySQL leave the prepared statement in a bad
# place when an error occurs. To support older mysql versions, we
# need to close the statement and delete the statement from the
# cache.
stmt.close
#statements.delete sql
raise e
end
Try configuring your database to turn off prepared statements, to see if that makes a difference.
In your ./config/database.yml file:
production:
adapter: mysql
prepared_statements: false
...
bugs with prepared statements?
There may be a problem with Rails ignoring this setting. If you want to know a lot more about it, see this discussion and bug fix by Jeremey Cole and Aaron: https://github.com/rails/rails/pull/7042
Heroku may ignore the setting. Here's a way you can try overriding Heroku by patching the prepared_statements setup: https://github.com/rails/rails/issues/5297
remove the query cache?
Try removing the ActiveRecord QueryCache to see if that makes a difference:
config.middleware.delete ActiveRecord::QueryCache
http://edgeguides.rubyonrails.org/configuring.html#configuring-middle
try postgres?
If you can try Postgres, that could clear it up too. That may not be a long term solution for you, but it would isolate the problem to MySQL.
The MySQL statement is obviously wrong, but the Ruby code you mentioned would not produce it. Something is wrong here, either you use a different Ruby code (maybe one from a before_filter) or pass a different parameter (like params[:artist_id] = "?"). Looks like you use nested resources, something like Artist has_many :albums. Maybe the #artist variable is not initialized correctly in the previous action, so that params[:artist_id] has not the right value?
I am using Hibernate3 with spring application.
I introduced a column in my existing table with many to one relationship with the addition in the .hbm file as mentioned below:
<many-to-one name="qualityStatus" column="quality_status"
class="model.PurchaseStatus" lazy="false" fetch="join"/>
The corresponding object to this hbm has a property of type 'PurchaseStatus' and field name as 'qualityStatus'.
Now when i try fetching the data from the table representing the above object with the query:
List<Long> dnIds = session.createQuery("select dnItem.dnId from DeliveryNoteItem dnItem where dnItem.qualityStatus!=? and dnItem.qualityStatus!=? and dnItem.qualityStatus!=?")
.setInteger(0,1)
.setInteger(1,5)
.setInteger(2,7)
.list();
i get an error saying as 'java.sql.Exception: Unknown column 'deliveryno0_.quality_status' in 'where clause'
I get this error when deployed on test server but when i check the same thing on my local server i works fine as well when i just run this query directly on the database on the test server it returns me result. So it confirms that the database on the test server is alright. It is only when it executes through the application on the test server i am getting this error.
Can anybody let me know what can be the reason or how do i debug this out.
Thanks in advance. Looking forward for replies.
If you really have a quality_status column in your database, then you should make sure you have the most recent version of your application deployed on the test server, and make sure that the connection url of hibernate really points to the test database.
This is not a hibernate problem, not a programming one. It is environmental - check your configurations and redeploy.