I have the exact same database definition for multiple databases ( and database servers ). How do I tell Jooq to use the same database as the "Connection" I created to connect to the DB?
Example ( for MySQL ):
jdbc:mysql://localhost:3306/tsm - my development DB ( tsm ), used to generate code
jdbc:mysql://RemoteAmazonDBHost:3306/customer1 - one of my customers
jdbc:mysql://RemoteAmazonDBHost:3306/customer2 - Another customer
All 3 databases have the same definition, the same tables, indexes, etc. The TSM one is the standard our application uses.
Maybe I should be using DSL.using( Connection, Setting ) instead of DSL.using(Connection)? Is that what the manual is implying here?
If I only have one "Input" schema, do I have to specify it? In other words, can I do something like this:
Settings settings = new Settings()
.withRenderMapping(new RenderMapping()
.withSchemata(
new MappedSchema().withOutput(
databaseInfo.getProperties().getProperty("database.db"))));
Or do I have to do something like this:
Settings settings = new Settings()
.withRenderMapping(new RenderMapping()
.withSchemata(
new MappedSchema().withInput("TSM")
.withOutput(databaseInfo.getProperties().getProperty("database.db"))));
I'm assuming you're using code generation. In that case, it might be the easiest to not generate the schema at all but use <outputSchemaToDefault> in the code generation configuration, e.g.
<configuration>
<generator>
<database>
<inputSchema>your_codegen_input_schema_here</inputSchema>
<outputSchemaToDefault>true</outputSchemaToDefault>
</database>
</generator>
</configuration>
See the manual for details: https://www.jooq.org/doc/latest/manual/code-generation/codegen-advanced/codegen-config-database/codegen-database-catalog-and-schema-mapping/
If you want to keep your generated code with schema qualification and map things at runtime, then your second attempt seems correct. Pass this to your Configuration (i.e. the DSL.using() call):
Settings settings = new Settings()
.withRenderMapping(new RenderMapping()
.withSchemata(new MappedSchema()
.withInput("TSM")
.withOutput(databaseInfo.getProperties().getProperty("database.db"))));
More details can be found here: https://www.jooq.org/doc/latest/manual/sql-building/dsl-context/custom-settings/settings-render-mapping
Related
I have a database with a lot of nonmanaged tables which I'm using for a django app. For testing I'm wanting to use the --keepdb option so that I don't have to repopulate these tables every time. I'm using MariaDB for my database. If I don't use the keepdb option everything works fine, the test database gets created and destroyed properly.
But when I try to run the test keeping the database:
$ python manage.py test --keepdb
I get the following error:
Using existing test database for alias 'default'...
Got an error creating the test database: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'CREATE DATABASE IF NOT EXISTS test_livedb ;\n SET sql_note' at line 2")
I assume that this is an issue with a different syntax between MariaDB and MySQL. Is there anyway to get the keepdb option to work with MariaDB?
thanks very much!
For what it's worth: This bug was introduced in Django 2.0.0 and fixed in Django 2.1.3 (https://code.djangoproject.com/ticket/29827)
Two things - check out Factory Boy (for creating test data) and I would suggest checking out Pytest as well. With non-managed tables, the issue I think you'll run into is that (at least in my experience) django won't create them in the test environment and you end up running into issues because there is no migration file to create those tables (since they're unmanaged). Django runs the migration files when creating the test environment.
With Pytest you can run with a --nomigrations flag which builds your test database directly off the models (thus creating the tables you need for your unmanaged models).
If you combine Pytest and Factory Boy you should be able to come up with the ability to setup your test data so it works as expected, is repeatable and testable without issue.
I actually approach it like this (slightly hacky, but it works with our complex setup):
On my model:
class Meta(object):
db_table = 'my_custom_table'
managed = getattr(settings, 'UNDER_TEST', False)
I create the UNDER_TEST variable in settings.py like this:
# Create global variable that will tell if our application is under test
UNDER_TEST = (len(sys.argv) > 1 and sys.argv[1] == 'test')
That way - when the application is UNDER_TEST the model is marked as managed (and Pytest will create the appropriate DB table). Then FactoryBoy handles putting all my test data into that table (either in setUp of the test or elsewhere) so I can test against it.
That's my suggestion - others might have something a little more clear or cleaner.
I have transfered my project from MySQL to PostgreSQL and tried to drop the column as result of previous issue, because after I removed the problematic column from models.py and saved. error didn't even disappear. Integer error transferring from MySQL to PostgreSQL
Tried both with and without quotes.
ALTER TABLE "UserProfile" DROP COLUMN how_many_new_notifications;
Or:
ALTER TABLE UserProfile DROP COLUMN how_many_new_notifications;
Getting the following:
ERROR: relation "UserProfile" does not exist
Here's a model, if helps:
class UserProfile(models.Model):
user = models.OneToOneField(User)
how_many_new_notifications = models.IntegerField(null=True,default=0)
User.profile = property(lambda u: UserProfile.objects.get_or_create(user=u)[0])
I supposed it might have something to do with mixed-case but I have found no solution through all similar questions.
Yes, Postgresql is a case aware database but django is smart enough to know that. It converts all field and it generally converts the model name to a lower case table name. However the real problem here is that your model name will be prefixed by the app name. generally django table names are like:
<appname>_<modelname>
You can find out what exactly it is by:
from myapp.models import UserProfile
print (UserProfile._meta.db_table)
Obviously this needs to be typed into the django shell, which is invoked by ./manage.py shell the result of this print statement is what you should use in your query.
Client: DataGrip
Database engine: PostgreSQL
For me this worked opening a new console, because apparently from the IDE cache it was not recognizing the table I had created.
Steps to operate with the tables of a database:
Database (Left side panel of the IDE) >
Double Click on PostgreSQL - #localhost >
Double Click on the name of the database >
Right click on public schema >
New > Console
GL
I am trying to switch between 2 mysql servers at runtime. I do not need to maintain both connections alive all the time.
This is what I am doing
from django.conf import settings
from django.db import connection
from django.contrib.auth.models import User
connection.close()
setattr(settings, 'DATABASE_HOST', 'mysql1.com')
list1 = User.objects.all()
connection.close()
setattr(settings, 'DATABASE_HOST', 'mysql2.com')
list2 = User.objects.all()
I have the following settings.py:
DATABASE_HOST = '' # localhost
DATABASE_NAME = test
...
The database name is the same on all servers and only the content of each tables differ.
I should get list1 != list2 as the users are different on both servers.
The issue is that I always get the list of users from the default database defined in settings.py (which is running on localhost) instead of the one from mysql 1 server and then from mysql 2 server.
Any idea what I am doing wrong here?
Laurent
My guess, from the information, would be a potential error in your set DATABASE_HOST lines (in yor pseudo code above). read: "setattr(settings..."
Other than that, I'm not sure how you've configured your database to switch based on your criteria, as you've not explained this. If you are doing it by model, it may be worth considering how Django knows this, or even using external connections (manually loading the database driver and running commands by hand prior to the render stage), and using the main.
I'd query the whole approach, but mostly because I'm not sure how you're actually differentiating the two databases, or why. Could you provide a bit more information on how you're doing this? I assume the variables you're pulling in dot-points 2 and 5 above are different. I don't need the values, I'm just making sure you've not used the old code duplication and forgotten to edit it (we've all been there).
Note: I'd post this as a comment if I could, but I think the solution may be in how you're pulling the variables. Finally, you could try adding the database name (just the server IP or whatever) to the output, if you're in 'dev'/debug (offline/non-production) mode, to check if it's actually making it to the second server.
For reference, the Django documentation explicitly states you shouldn't do this -- Altering settings at runetime.
There is a lot of talk within the Django community about the ORM supporting multiple connections/databases at once. There's a lot of good reference info out there on it. Check out this blog post: Easy Multi-Database Support for Django and this Django wiki page Multiple Database Support.
In the blog post, Eric Florenzano does something like this in his settings.py file:
DATABASES = dict(
primary = dict(
DATABASE_NAME=DATABASE_NAME,
# ...
),
secondary = dict(
DATABASE_NAME='secondary.db',
# ...
),
)
I am using LINQ to SQL to connect to database from my application. When I am changing environment from production to staging, I can update my connection string in web.config.
But there is one more value I need to update when environment changes. That is database name. In LINQ to SQL designer file, database name is mentioned as attribute like-
[System.Data.Linq.Mapping.DatabaseAttribute(Name="somedbname")]
How can I pick up value of Name dynamically from some config file?
Any help is really appreciated.
as mentioned on the
http://msdn.microsoft.com/en-us/library/system.data.linq.mapping.databaseattribute.name.aspx
"The DatabaseName is used only if the connection itself does not specify the database name."
so you can delete this attribute and all is gonna work fine!
I've used a wrapper class to provide a context along the lines of
public DataContext Context = new DataContext(SqlConnectionString); //much simplified
I fixed this problem by editing the .dbml file outside of Visual Studio (the designer doesn't seem to allow access to the DatabaseAttribute) and getting rid of the name property here:
<Database Name="BadName" Class="OutputDataContext" xmlns="http://schemas.microsoft.com/linqtosql/dbml/2007">
(note that the accepted answer is no longer correct: this attribute was overriding my connection string)
I'm using VSTS Database Edition GDR Version 9.1.31024.02
I've got a project where we will be creating multiple databases with identical schema, on the fly, as customers are added to the system. It's one DB per customer. I thought I should be able to use the deploy script to do this. Unfortunately I always get the full filenames specified on the CREATE DATABASE statement. For example:
CREATE DATABASE [$(DatabaseName)]
ON
PRIMARY(NAME = [targetDBName], FILENAME = N'$(DefaultDataPath)targetDBName.mdf')
LOG ON (NAME = [targetDBName_log], FILENAME = N'$(DefaultDataPath)targetDBName_log.ldf')
GO
I'd expected something more like this
CREATE DATABASE [$(DatabaseName)]
ON
PRIMARY(NAME = [targetDBName], FILENAME = N'$(DefaultDataPath)$(DatabaseName).mdf')
LOG ON (NAME = [targetDBName_log], FILENAME = N'$(DefaultDataPath)$(DatabaseName)_log.ldf')
GO
Or even
CREATE DATABASE [$(DatabaseName)]
I'm not going to be running this on an on-going basis so I'd like to make it as simple as possible, for the next guy. There are a bunch of options for deployment in the project properties, but I can't get this to work the way I'd like.
Any one know how to set this up?
Better late than never, I know how to get the $(DefaultDataPath)$(DatabaseName) file names from your second example.
The SQL you're showing in your first code snippet suggests that you don't have scripts for creating the database files in your VSTS:DB project, perhaps by deliberately excluded them from any schema comparisons you've done. I found it a little counter-intuitive, but the solution is to let VSTS:DB script the MDF and LDF in you development environment, then edit those scripts to use the SQLCMD variables.
In your database project, go to the folder Schema Objects > Database Level Objects > Storage > Files. In there, add these two files:
Database.sqlfile.sql
ALTER DATABASE [$(DatabaseName)]
ADD FILE (NAME = [$(DatabaseName)],
FILENAME = '$(DefaultDataPath)$(DatabaseName).mdf',
SIZE = 2304 KB, MAXSIZE = UNLIMITED, FILEGROWTH = 1024 KB)
TO FILEGROUP [PRIMARY];
Database_log.sqlfile.sql
ALTER DATABASE [$(DatabaseName)]
ADD LOG FILE (NAME = [$(DatabaseName)_log],
FILENAME = '$(DefaultDataPath)$(DatabaseName)_log.ldf',
SIZE = 1024 KB, MAXSIZE = 2097152 MB, FILEGROWTH = 10 %);
The full database creation script that VSTS:DB, or for that matter VSDBCMD.exe, generates will now use the SQLCMD variables for naming the MDF and LDF files, allowing you to specify them on the command line, or in MSBuild.
We do this using a template database, that we back up, copy, and restore as new customers are brought online. We don't do any of the schema creation with scripts but with a live, empty DB.
Hmm, well it seems that the best answer so far (given the over whelming response) is to edit the file after the fact... Still looking