I want to write integration tests for my application, which is using a MySQL db via jdbc.
(FYI: The application has a user database and a wiki so far, so nothing sophisticated really)
I came across a lot of posts, mostly talking about HSQLDB or H2, as well as some exotics or discontinued (like mxj)
Sadly, most of these threads were made years ago and so many things have changed since then.
I'm looking for people with recent experience (or even better having a similiar setup like I got - that means spring and the need to run these tests both locally and on jenkins)!
I understand that the best way to find the answer is to try it myself and see what works best, but maybe someone has recent experience and is willing to share :)
Technologies used:
MySQL
Spring
TestNG
Jenkins
I'd be very thankful if you can share your experience and advices with me.
Try Maven plugin created exactly for this purpose: jcabi-mysql-maven-plugin. It starts a local MySQL server on pre-integration-test phase and shuts it down on post-integration-test.
So I decided to use h2, which gave me some problems with importing MySQL dumbs. The answer is here: Running h2 in MODE=MySQL doesn't support MySQL dumps
Basically, you have to remove quotes around table-names (they seem to work fine for fields nevertheless). You can use back-ticks ( ` ) or not use any ticks/quotes at all.
Another thing that caused a problem was a "AUTO_INCREMENT = 1" at the end of the table definition. Since h2 uses 1 as default after the start, this command is not needed anyways.
Here's an example how to setup the embedded database:
public class InMemoryTest {
#org.testng.annotations.Test // ofc this can be JUnit #Test as well
public void test() throws Exception {
Class.forName("org.h2.Driver");
Connection conn = DriverManager.
getConnection("jdbc:h2:mem:test;MODE=MySQL;IGNORECASE=TRUE;INIT=RUNSCRIPT FROM 'src/test/resources/test.sql'");
Statement stat = conn.createStatement();
stat.execute("INSERT INTO `usr_avatar` (\"usr_avatar_user_id\") VALUES (1)");
}
}
and the test.sql would be:
DROP TABLE IF EXISTS usr_avatar;
CREATE TABLE IF NOT EXISTS usr_avatar (
"usr_avatar_id" int(11) NOT NULL AUTO_INCREMENT,
"usr_avatar_user_id" int(11) NOT NULL
)
Related
I have a database with a lot of nonmanaged tables which I'm using for a django app. For testing I'm wanting to use the --keepdb option so that I don't have to repopulate these tables every time. I'm using MariaDB for my database. If I don't use the keepdb option everything works fine, the test database gets created and destroyed properly.
But when I try to run the test keeping the database:
$ python manage.py test --keepdb
I get the following error:
Using existing test database for alias 'default'...
Got an error creating the test database: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'CREATE DATABASE IF NOT EXISTS test_livedb ;\n SET sql_note' at line 2")
I assume that this is an issue with a different syntax between MariaDB and MySQL. Is there anyway to get the keepdb option to work with MariaDB?
thanks very much!
For what it's worth: This bug was introduced in Django 2.0.0 and fixed in Django 2.1.3 (https://code.djangoproject.com/ticket/29827)
Two things - check out Factory Boy (for creating test data) and I would suggest checking out Pytest as well. With non-managed tables, the issue I think you'll run into is that (at least in my experience) django won't create them in the test environment and you end up running into issues because there is no migration file to create those tables (since they're unmanaged). Django runs the migration files when creating the test environment.
With Pytest you can run with a --nomigrations flag which builds your test database directly off the models (thus creating the tables you need for your unmanaged models).
If you combine Pytest and Factory Boy you should be able to come up with the ability to setup your test data so it works as expected, is repeatable and testable without issue.
I actually approach it like this (slightly hacky, but it works with our complex setup):
On my model:
class Meta(object):
db_table = 'my_custom_table'
managed = getattr(settings, 'UNDER_TEST', False)
I create the UNDER_TEST variable in settings.py like this:
# Create global variable that will tell if our application is under test
UNDER_TEST = (len(sys.argv) > 1 and sys.argv[1] == 'test')
That way - when the application is UNDER_TEST the model is marked as managed (and Pytest will create the appropriate DB table). Then FactoryBoy handles putting all my test data into that table (either in setUp of the test or elsewhere) so I can test against it.
That's my suggestion - others might have something a little more clear or cleaner.
I have a mysql db on a server where I've bought some space.
Now I want to add a new mvc application to my website to try out some stuff, without buggering up my existing application.
That puts me in a bit of a problem though as I only have one database available, I could just buy a new one, but I don't want to have to buy a new one every time I want to try stuff out.
So I figured, why not just pre-fix all my tables with some unique value for that application, that way I can keep the stored data separate while still using the same db.
That took me on a journey across countless articles both here on stackoverflow and other places on the web.
After about a day I stumbled upon this nugget
<!-- language: c# -->
protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
modelBuilder.Types().Configure(entity => entity.ToTable("pf_" + entity.ClrType.Name));
base.OnModelCreating(modelBuilder);
}
Which does exactly what I want, as long as you either use a mssql database, or you drop your mysql database and create a new one.
However, the database I've been given is not an mssql database, and dropping it is not an option.
When, with an existing mysql db, I try to run
update-database
I get the following error
Can't find file: '.\mytestdb\dbo#002etestitems.frm' (errno: 2 - No such file or directory)
where "mytestdb" is my local db that I'm testing on, "testitems" is the current/previous tablename, .frm is ofcourse the db format, and I have no idea what this "dbo#002e" is, I tried googling it to no avail.
My stack trace can be found here.
http://pastebin.com/yF2dGkm6
Has anyone been in a similar situation, or has ideas as to how I can make it work? =)
dbo#002e stands for "dbo.", looking at your migration file you should discover rows like this one:
RenameTable(name: "dbo.Table", newName: "pf_Table");
Despite everything it's ok with CreateTable with table names prefixed by "dbo.", RenameTable origin the whole "dbo.Name" to be looked for and so the error comes.
My suggestion is to remove manually the "dbo." on the RenameTable's "name:" parameter.
EDIT:
After fixed the problem editing the migration file the best solution is to add this to the Configuration constructor:
public Configuration()
{
AutomaticMigrationsEnabled = false;
....
SetSqlGenerator("MySql.Data.MySqlClient", new MySql.Data.Entity.MySqlMigrationSqlGenerator());
CodeGenerator = new MySql.Data.Entity.MySqlMigrationCodeGenerator();
....
}
It should work since the .Net Connector version 6.7.4 on.
I am working on a big project with a lot of forms using BDE and ODBC to connect to MySQL Database.
I can compile it in Delphi 2009 and it works fine.
When i compile it in Delphi 2010, nothing works because TQuery can not pass parameter values correctly.
Here is an example :
txtUsername.Text = 'Admin';
Query1.Close;
Query1.SQL.Text = 'Select Count(*) From Tbl_User where Username = :username';
Query1.ParamByName('username').AsString = txtUsername.Text;
Query.Open();
The SQL will be sent to MySQL , looks like this :
Select Count(*) From Tbl_User where Username = 'A'
Only first character of parameter will be sent to the server : 'A' instead of 'Admin'
But if i use
Query1.ParamByName('username').AsAnsiString , then
it will works fine and parameter will be sent completely:
Select Count(*) From Tbl_User where Username = 'Admin'
There are huge number of TQuery and TTable in project and its not possible to change all calls of AsString to AsAnsiString.
Is there any solution for this? any settings to make it working fine? probably by making it to use Ansi as default instead of Unicode?
I tried finding some setting in compiling option, and changing ODBC parameters but none of them worked.
Any help would be appreciated.
Thanks
I understand the pressure for quick and cheap solutions, but sometimes they will be quick and cheap only at the first moment, and then will turn (in a near future) into a monster problem with no solution at all.
So, for the first move, I recomend you to read a post on Regex Replace. Use this to replace all your .AsString by .AsAnsiString and quickly get you application running as expected again.
But don´t get too comfortable with this, you have a long term work ahead of you. My suggestion to give your application a better design is:
Remove your datasets from your forms and put them in a DataModule, so you will not mix persistence and business code with presentation code;
Stop working with specific dataset on your business code, go for TClientDataset, which is better than any other dataset and will give you a better application design and complete database and middleware independence;
Create a new database layer, putting all the database and middleware specifics behind an interface, what will give you a incredible level of database abstraction.
This is a time consuming work, of course, so, not a quick and certainly not a cheap solution, but it will give you a lot of insights and improve your techniques. During the process, your application will evolve as never before.
I am having trouble sending a SQL statement through a DbContext using context.Database.ExecuteSqlCommand().
I am trying to execute
CREATE TABLE Phones([Id] [uniqueidentifier] NOT NULL PRIMARY KEY,
[Number] [int],[PhoneTypeId] [int])
GO
ALTER TABLE [dbo].[Phones] ADD CONSTRAINT [DF_Phones_Id]
DEFAULT (newid()) FOR [Id]
GO
This fails with the error string
Incorrect syntax near the keyword 'ALTER'.
Incorrect syntax near 'GO'.
However running that exact statement in SSMS runs without errors? Any issues I need to resolve regarding the default constraint throught the DbContext. I have see problems with people using constraints and not having IsDbGenerated set to true. I am not sure how that would apply here though.
GO is not a part of SQL, so it can't be executed with ExecuteSqlCommand(). Think of GO as a way to separate batches when using Management Studio or the command-line tools. Instead, just remove the GO statements and you should be fine. If you run into errors because you need to run your commands in separate batches, just call ExecuteSqlCommand() once for each batch you want to run.
I know, necroposting is bad maner, but may be this post would save someone's time. As it was mentioned in Dave's post, GO is not a part of SQL, so we can create little workaround to make it work
var text = System.IO.File.ReadAllText("initialization.sql");
var parts = text.Split(new string[] { "GO" }, System.StringSplitOptions.None);
foreach (var part in parts) { context.Database.ExecuteSqlCommand(part); }
context.SaveChanges();
In this case your commands would be splitted and executed without problems
Dave Markle beat me to it. In fact, you can change "GO" to any other string to separate batches.
An alternative implementation here is to use SMO instead of the Entity Framework. There is a useful method there called ExecuteNonQuery that I think will make your life a lot simpler. Here is a good implementation example.
I am using the RJDBC package to connect to a MySQL (Maria DB) database in R on a Windows 7 machine and I am trying a statement like
select a as b
from table
but the column will always continue to be named "a" in the data frame.
This works normally with RODBC and RMySQL but doesn't work with RJDBC. Unfortunately, I have to use RJDBC as this is the only package that has no problem with the encoding of chinese, hebrew and so on letters (set names and so on don't seem to work with RODBC and RMySQL).
Has anybody experienced this problem?
I have run into the same frustrating issue. Sometimes the AS keyword would have its intended effect, but other times it wouldn't. I was unable to identify the conditions to make it work correctly.
Short Answer: (Thanks to Simon Urbanek (package maintainer for RJDBC), Yev, and Sebastien! See the Long Answer.) One thing that you may try is to open your JDBC connection using ?useOldAliasMetadataBehavior=true in your connection string. Example:
drv <- JDBC("com.mysql.jdbc.Driver", "C:/JDBC/mysql-connector-java-5.1.18-bin.jar", identifier.quote="`")
conn <- dbConnect(drv, "jdbc:mysql://server/schema?useOldAliasMetadataBehavior=true", "username", "password")
query <- "SELECT `a` AS `b` FROM table"
result <- dbGetQuery(conn, query)
dbDisconnect(conn)
This ended up working for me! See more details, including caveats, in the Long Answer.
Long Answer: I tried all sorts of stuff, including making views, changing queries, using JOIN statements, NOT using JOIN statements, using ORDER BY and GROUP BY statements, etc. I was never able to figure out why some of my queries were able to rename columns and others weren't.
I contacted the package maintainer (Simon Urbanek.) Here is what he said:
In the vast majority of cases this is an issue in the JBDC driver, because there is really not much RJDBC can do other than to call the driver.
He then recommended that I make sure I had the most recent JDBC driver for MySQL. I did have the most recent version. However, it got me thinking "maybe it IS a bug with the JDBC driver." So, I searched Google for: mysql jdbc driver bug alias.
The top result for this query was an entry at bugs.mysql.com. Yev, using MySQL 5.1.22, says that when he upgraded from driver version 5.0.4 to 5.1.5, his column aliases stopped working. Asked if it was a bug.
Sebastien replied, "No, it's not a bug! It's a documented change of behavior in all subsequent versions of the driver." and suggested using ?useOldAliasMetadataBehavior=true, citing documentation for the JDBC driver.
Caveat Lector: The documentation for the JDBC driver states that
useColumnNamesInFindColumn is preferred over useOldAliasMetadataBehavior unless you need the specific behavior that it provides with respect to ResultSetMetadata.
I haven't had the time to fully research what this means. In other words, I don't know what all of the ramifications are of using useOldAliasMetadataBehavior=true are. Use at your own risk. Does someone else have more information?
I don't know RJDBC, but in some cases when it is necessary to give permanent aliases to columns without renaming them, you can use VIEWs
CREATE OR REPLACE VIEW v_table AS
SELECT a AS b
FROM table
... and then ...
SELECT b FROM v_table
There is a separate function in the ResultSetMetaData interface for retrieving the column label vs the column name:
String getColumnLabel(int column) throws SQLException;
Gets the designated column's suggested title for use in printouts and
displays. The suggested title is usually specified by the SQL AS
clause. If a SQL AS is not specified, the value returned
fromgetColumnLabel will be the same as the value returned by the
getColumnName method.
Using getColumnLabel should resolve this issue (if not, check that your JDBC driver is following this spec).
e.g.
ResultSetMetaData rsmd = rs.getMetaData();
int columnCount = rsmd.getColumnCount();
while(rs.next()) {
for (int i = 1; i < columnCount + 1; i++) {
String label = rsmd.getColumnLabel(i);
System.out.println(rs.getString(label));
}
}
This is the work around we use for R and SAP HANA via RJDBC:
names(result)[1]<-"b"
It's not the nicest work around, but since Aaron's solution does work for us, we went with this "solution".