Fluent NHibernate - always drop table - sql-server-2008

Hi I start learn Fluent NHibernate. I am using this tutorial http://www.d80.co.uk/post/2011/02/20/Linq-to-NHibernate-Tutorial.aspx.
Here is my sample code:
public class Account
{
public virtual int Id { get; set; }
public virtual string Nick { get; set; }
public virtual string Password { get; set; }
}
public class AccountMap:ClassMap<Account>
{
public AccountMap()
{
Id(x => x.Id);
Map(x => x.Nick);
Map(x => x.Password);
}
}
public class NHiberanteHelper
{
private static ISessionFactory _sessionFactory;
private static ISessionFactory SessionFactory
{
get
{
if (_sessionFactory == null)
InitializeSessionFactory();
return _sessionFactory;
}
}
private static void InitializeSessionFactory()
{
_sessionFactory = Fluently.Configure()
//NHibernate bude pouzivat ovladace pre MS SQL 2008
.Database(MsSqlConfiguration.MsSql2008
.ConnectionString(
#"Server=JAN-MSI\SQLEXPRESS;Database=SimpleNHibernate;Trusted_Connection=True;").ShowSql()
)
//urci NHibernatu kde ma hladat mapovacie subory
.Mappings(m=>m.FluentMappings.AddFromAssemblyOf<Account>())
//ak tabs nie su v DB vytvori
.ExposeConfiguration(cfg => new SchemaExport(cfg).Create(true, true))
//vytvori jeden session pre cely life-time apps
.BuildSessionFactory();
}
public static ISession OpenSession()
{
return SessionFactory.OpenSession();
}
}
class Program
{
static void Main(string[] args)
{
using (var session = NHiberanteHelper.OpenSession())
{
using (var trans = session.BeginTransaction())
{
var account = new Account
{
Nick = "dfdwf",
Password = "xxx"
};
session.Save(account);
trans.Commit();
}
}
Console.ReadKey();
}
}
Problem is that this Fluent configuration always drop table in database.
I need only check if table doesnt exist so create table not always when code run drop table.

Your call to new SchemaExport(cfg).Create(true, true) is exporting the configuration to the database. This will drop and re-create it (it's not clever enough to work out the differences and just execute them.
You could use SchemaUpdate, which will update the schema instead. Here's a blog post about it: http://geekswithblogs.net/dotnetnomad/archive/2010/02/22/138094.aspx
I would always prefer to update tables myself or use something like Redgate's SQLCompare to update a schema whilst preserving data.

Related

MySQL and EF Core 6 error The LINQ expression could not be translated

I recently updated our project from EF Core 2.2.6 to 6.x (along with and upgrade from .NET core 3.1 to .NET 6) and now I'm get errors like the one stated in the title whenever the query gets even a little complicated. One of those cases is when you add a GroupBy clause. Below is an example of a failing query.
_context.MyTable
.Where(a => a.Name.Contains("service"))
.GroupBy(ss => ss.IsServiceSpecific)
The entire error is:
The LINQ expression 'DbSet< MyTable >()
.Where(a => a.Name.Contains("service"))
.GroupBy(ss => ss.IsServiceSpecific)' could not be translated. Either rewrite the query in a form that can be translated, or switch
to client evaluation explicitly by inserting a call to 'AsEnumerable',
'AsAsyncEnumerable', 'ToList', or 'ToListAsync'
The setup at this MySQL::Entity Framework Core Support URL is exactly what I did (there are only two steps to set it up). My DI config looks like this:
builder.Services.AddEntityFrameworkMySQL()
.AddDbContext<MydbContext>(options =>
{
options.UseMySQL(builder.Configuration.GetConnectionString("DefaultConnection"));
});
It will execute simple queries but more complex ones always generate this error. It says to rewrite the query and force client side evaluation by using AsEnumerable or ToList but I don't want to drag all that data to the client and I expect that a simple group by can be translated and handled server side.
I did find one article that talks about this problem but I'm not getting if it's suggesting an actual solution.
This shouldn't be this hard and I feel like I'm missing something simple.
Model
internal class Post
{
public int PostId { get; set; }
public string? Title { get; set; }
public string? Content { get; set; }
public int BlogId { get; set; }
}
DBContext
internal class BloggingContext : DbContext
{
public DbSet<Post>? Posts { get; set; }
public string DbPath { get; }
public BloggingContext()
{
var path = Environment.GetFolderPath(Environment.SpecialFolder.Desktop);
DbPath = $"{path}{Path.DirectorySeparatorChar}blogging.db";
}
protected override void OnConfiguring(DbContextOptionsBuilder options)
=> options.UseSqlite($"Data Source={DbPath}");
}
Main
internal class Program
{
static void Main(string[] args)
{
using (var db = new BloggingContext())
{
var posts = db.Posts.Where(s => s.Title.Contains("Hello")).GroupBy(g => g.BlogId == 1994).Select(s => new { Key = s.Key, Counts = s.Count() }).ToList();
foreach (var p in posts)
{
Console.WriteLine(p);
}
}
}
}
Conclusion: You might add Select statement after GroupBy.

how to save apache spark schema output in mysql database

Can anyone please tell me if there is any way in apache spark to store a JavaRDD on mysql database? I am taking input from 2 csv files and then after doing join operations on their contents I need to save the output(the output JavaRDD) in the mysql database. I am already able to save the output successfully on hdfs but I am not finding any information related to apache Spark-MYSQL connection. Below I am posting the code for spark sql. This might serve as a reference to those who are looking for an example for spark-sql.
package attempt1;
import java.io.Serializable;
import org.apache.spark.api.java.JavaRDD;
import org.apache.spark.api.java.JavaSparkContext;
import org.apache.spark.api.java.function.Function;
import org.apache.spark.sql.api.java.JavaSQLContext;
import org.apache.spark.sql.api.java.JavaSchemaRDD;
import org.apache.spark.sql.api.java.Row;
public class Spark_Mysql {
#SuppressWarnings("serial")
public static class CompleteSample implements Serializable {
private String ASSETNUM;
private String ASSETTAG;
private String CALNUM;
public String getASSETNUM() {
return ASSETNUM;
}
public void setASSETNUM(String aSSETNUM) {
ASSETNUM = aSSETNUM;
}
public String getASSETTAG() {
return ASSETTAG;
}
public void setASSETTAG(String aSSETTAG) {
ASSETTAG = aSSETTAG;
}
public String getCALNUM() {
return CALNUM;
}
public void setCALNUM(String cALNUM) {
CALNUM = cALNUM;
}
}
#SuppressWarnings("serial")
public static class ExtendedSample implements Serializable {
private String ASSETNUM;
private String CHANGEBY;
private String CHANGEDATE;
public String getASSETNUM() {
return ASSETNUM;
}
public void setASSETNUM(String aSSETNUM) {
ASSETNUM = aSSETNUM;
}
public String getCHANGEBY() {
return CHANGEBY;
}
public void setCHANGEBY(String cHANGEBY) {
CHANGEBY = cHANGEBY;
}
public String getCHANGEDATE() {
return CHANGEDATE;
}
public void setCHANGEDATE(String cHANGEDATE) {
CHANGEDATE = cHANGEDATE;
}
}
#SuppressWarnings("serial")
public static void main(String[] args) throws Exception {
JavaSparkContext ctx = new JavaSparkContext("local[2]", "JavaSparkSQL");
JavaSQLContext sqlCtx = new JavaSQLContext(ctx);
JavaRDD<CompleteSample> cs = ctx.textFile("C:/Users/cyg_server/Documents/bigDataExample/AssetsImportCompleteSample.csv").map(
new Function<String, CompleteSample>() {
public CompleteSample call(String line) throws Exception {
String[] parts = line.split(",");
CompleteSample cs = new CompleteSample();
cs.setASSETNUM(parts[0]);
cs.setASSETTAG(parts[1]);
cs.setCALNUM(parts[2]);
return cs;
}
});
JavaRDD<ExtendedSample> es = ctx.textFile("C:/Users/cyg_server/Documents/bigDataExample/AssetsImportExtendedSample.csv").map(
new Function<String, ExtendedSample>() {
public ExtendedSample call(String line) throws Exception {
String[] parts = line.split(",");
ExtendedSample es = new ExtendedSample();
es.setASSETNUM(parts[0]);
es.setCHANGEBY(parts[1]);
es.setCHANGEDATE(parts[2]);
return es;
}
});
JavaSchemaRDD complete = sqlCtx.applySchema(cs, CompleteSample.class);
complete.registerAsTable("cs");
JavaSchemaRDD extended = sqlCtx.applySchema(es, ExtendedSample.class);
extended.registerAsTable("es");
JavaSchemaRDD fs= sqlCtx.sql("SELECT cs.ASSETTAG, cs.CALNUM, es.CHANGEBY, es.CHANGEDATE FROM cs INNER JOIN es ON cs.ASSETNUM=es.ASSETNUM;");
JavaRDD<String> result = fs.map(new Function<Row, String>() {
public String call(Row row) {
return row.getString(0);
}
});
result.saveAsTextFile("hdfs://path/to/hdfs/dir-name"); //instead of hdfs I need to save it on mysql database, but I am not able to find any Spark-MYSQL connection
}
}
Here at the end I am saving the result successfully in HDFS. But now I want to save into MYSQL database. Kindly help me out. Thanks
There are two approaches you can use for writing your results back to the database. One is to use something like DBOutputFormat and configure that, and the other is to use foreachPartition on the RDD you want to save and pass in a function which creates a connection to MySQL and writes the result back.
Here is an example using DBOutputFormat.
Create a class that represents your table row -
public class TableRow implements DBWritable
{
public String column1;
public String column2;
#Override
public void write(PreparedStatement statement) throws SQLException
{
statement.setString(1, column1);
statement.setString(2, column2);
}
#Override
public void readFields(ResultSet resultSet) throws SQLException
{
throw new RuntimeException("readFields not implemented");
}
}
Then configure your job and write a mapToPair function. The value doesn't appear to be used. If anyone knows, please post a comment.
String tableName = "YourTableName";
String[] fields = new String[] { "column1", "column2" };
JobConf job = new JobConf();
DBConfiguration.configureDB(job, "com.mysql.jdbc.Driver", "jdbc:mysql://localhost/DatabaseNameHere", "username", "password");
DBOutputFormat.setOutput(job, tableName, fields);
// map your rdd into a table row
JavaPairRDD<TableRow, Object> rows = rdd.mapToPair(...);
rows.saveAsHadoopDataset(job);

NHibernate LinqToHqlGenerator for SQL Server 2008 full text index 'Containing' keyword

I think I'm missing something fundamental when implementing a LinqToHql generator class.
I've successfully registered the SQL Server 2008 contains query using a custom dialect with this registration:
RegisterFunction("contains", new StandardSQLFunction("contains", null));
I have only one class with a full text index to be queried:
public class SearchName
{
public virtual Guid Id {get; set;}
public virtual string Name {get; set;} // this is the search field
}
The contains function works properly in HQL:
var names = Session.CreateQuery("from SearchName where contains(Name,:keywords)")
.SetString("keywords", "john")
.List();
and the generated SQL is perfect:
select searchname0_.Id as Id4_,
searchname0_.Name as Name4_
from Search_Name searchname0_
where contains(searchname0_.Name, 'john' /* #p0 */)
The next challenge was to implement the Linq to HQL generator:
public class MyLinqtoHqlGeneratorsRegistry :
DefaultLinqToHqlGeneratorsRegistry
{
public MyLinqtoHqlGeneratorsRegistry()
{
this.Merge(new ContainsGenerator());
}
}
public class ContainsGenerator : BaseHqlGeneratorForMethod
{
public ContainsGenerator()
{
SupportedMethods = new[] {
ReflectionHelper.GetMethodDefinition<SearchName>(d => d.Name.Contains(String.Empty))
};
}
public override HqlTreeNode BuildHql(MethodInfo method,
System.Linq.Expressions.Expression targetObject,
ReadOnlyCollection<System.Linq.Expressions.Expression> arguments,
HqlTreeBuilder treeBuilder, IHqlExpressionVisitor visitor)
{
return treeBuilder.MethodCall("contains",
visitor.Visit(targetObject).AsExpression(),
visitor.Visit(arguments[0]).AsExpression()
);
}
}
}
Calling the method like this:
var namesLinq = Session.Query<SearchName>().Where(x=> x.Name.Contains("john")).ToList();
Unfortunately, this doesn't seem to override the built-in Contains method, and the generated SQL is wrong:
select searchname0_.Id as Id4_,
searchname0_.Name as Name4_
from Search_Name searchname0_
where searchname0_.Name like ('%' + 'john' /* #p0 */ + '%')
Is it not possible to override the default Contains method, or have I just made a silly mistake?
PS - I'm using NHibernate 3.3.1.4000
OK, I've finally figured it out!
First, I managed to delete the registration code from my configuration:
...
.ExposeConfiguration(cfg =>
{
cfg.LinqToHqlGeneratorsRegistry<MyLinqtoHqlGeneratorsRegistry>();
...
}
Second, don't try to override the existing Linq behaviors. I moved my Contains extension method to the full-text class.
Third, build the Hql tree correctly.
For others trying to implement a SQL 2008 Free-text contains search, here's the complete implementation:
public static class DialectExtensions
{
public static bool Contains(this SearchName sn, string searchString)
{
// this is just a placeholder for the method info.
// It does not otherwise matter.
return false;
}
}
public class MyLinqtoHqlGeneratorsRegistry : DefaultLinqToHqlGeneratorsRegistry
{
public MyLinqtoHqlGeneratorsRegistry()
: base()
{
RegisterGenerator(ReflectionHelper.GetMethod(() =>
DialectExtensions.Contains(null, null)),
new ContainsGenerator());
}
}
public class ContainsGenerator : BaseHqlGeneratorForMethod
{
string fullTextFieldName = "Name";
public ContainsGenerator()
: base()
{
SupportedMethods = new[] {
ReflectionHelper.GetMethodDefinition(() =>
DialectExtensions.Contains(null, null))
};
}
public override HqlTreeNode BuildHql(MethodInfo method,
System.Linq.Expressions.Expression targetObject,
ReadOnlyCollection<System.Linq.Expressions.Expression> arguments,
HqlTreeBuilder treeBuilder, IHqlExpressionVisitor visitor)
{
// cannot figure out how to interrogate the model class to get an
// arbitrary field name...
// perhaps the RegisterGenerator() call above could be used to pass a
// property name to the ContainsGenerator constructor?
// in our case, we only have one full text searchable class, and its
// full-text searchable field is "Name"
HqlExpression[] args = new HqlExpression[2] {
treeBuilder.Ident(fullTextFieldName).AsExpression(),
visitor.Visit(arguments[1]).AsExpression()
};
return treeBuilder.BooleanMethodCall("contains", args);
}
}
For the above to work, you must have declared and used your custom dialect:
public class CustomMsSql2008Dialect : NHibernate.Dialect.MsSql2008Dialect
{
public CustomMsSql2008Dialect()
{
RegisterFunction(
"contains",
new StandardSQLFunction("contains", null)
);
}
}
Then you can use your new contains search this way:
var namesLinq = Session.Query<SearchName>().Where(x => x.Contains("john")).ToList();
... and the resulting SQL is perfect! (at least if you only have one table you're performing full-text searches on)
EDIT: UPDATED IMPLEMENTATION TO SUPPORT MORE THAN ONE FULLTEXT 'Contains' SEARCH PER QUERY.
Here's the revised version:
public static class DialectExtensions
{
public static bool FullTextContains(this string source, string pattern)
{
return false;
}
}
public class MyLinqtoHqlGeneratorsRegistry : DefaultLinqToHqlGeneratorsRegistry
{
public MyLinqtoHqlGeneratorsRegistry()
: base()
{
RegisterGenerator(ReflectionHelper.GetMethod(() => DialectExtensions.FullTextContains(null, null)),
new FullTextContainsGenerator());
}
}
public class FullTextContainsGenerator : BaseHqlGeneratorForMethod
{
public FullTextContainsGenerator()
{
SupportedMethods = new[] { ReflectionHelper.GetMethod(() => DialectExtensions.FullTextContains(null, null)) };
}
public override HqlTreeNode BuildHql(MethodInfo method,
System.Linq.Expressions.Expression targetObject,
ReadOnlyCollection<System.Linq.Expressions.Expression> arguments,
HqlTreeBuilder treeBuilder, IHqlExpressionVisitor visitor)
{
HqlExpression[] args = new HqlExpression[2] {
visitor.Visit(arguments[0]).AsExpression(),
visitor.Visit(arguments[1]).AsExpression()
};
return treeBuilder.BooleanMethodCall("contains", args);
}
}
To use the revised version, the syntax is slightly different:
var namesLinq = Session.Query<SearchName>().Where(x => x.Name.FullTextContains("john")).ToList();

Storing Apache Hadoop Data Output to Mysql Database

I need to store output of map-reduce program into database, so is there any way?
If so, is it possible to store output into multiple columns & tables based on requirement??
please suggest me some solutions.
Thank you..
The great example is shown on this blog, I tried it and it goes really well. I quote the most important parts of the code.
At first, you must create a class representing data you would like to store. The class must implement DBWritable interface:
public class DBOutputWritable implements Writable, DBWritable
{
private String name;
private int count;
public DBOutputWritable(String name, int count) {
this.name = name;
this.count = count;
}
public void readFields(DataInput in) throws IOException { }
public void readFields(ResultSet rs) throws SQLException {
name = rs.getString(1);
count = rs.getInt(2);
}
public void write(DataOutput out) throws IOException { }
public void write(PreparedStatement ps) throws SQLException {
ps.setString(1, name);
ps.setInt(2, count);
}
}
Create objects of previously defined class in your Reducer:
public class Reduce extends Reducer<Text, IntWritable, DBOutputWritable, NullWritable> {
protected void reduce(Text key, Iterable<IntWritable> values, Context ctx) {
int sum = 0;
for(IntWritable value : values) {
sum += value.get();
}
try {
ctx.write(new DBOutputWritable(key.toString(), sum), NullWritable.get());
} catch(IOException e) {
e.printStackTrace();
} catch(InterruptedException e) {
e.printStackTrace();
}
}
}
Finally you must configure a connection to your DB (do not forget to add your db connector on the classpath) and register your mapper's and reducer's input/output data types.
public class Main
{
public static void main(String[] args) throws Exception
{
Configuration conf = new Configuration();
DBConfiguration.configureDB(conf,
"com.mysql.jdbc.Driver", // driver class
"jdbc:mysql://localhost:3306/testDb", // db url
"user", // username
"password"); //password
Job job = new Job(conf);
job.setJarByClass(Main.class);
job.setMapperClass(Map.class); // your mapper - not shown in this example
job.setReducerClass(Reduce.class);
job.setMapOutputKeyClass(Text.class); // your mapper - not shown in this example
job.setMapOutputValueClass(IntWritable.class); // your mapper - not shown in this example
job.setOutputKeyClass(DBOutputWritable.class); // reducer's KEYOUT
job.setOutputValueClass(NullWritable.class); // reducer's VALUEOUT
job.setInputFormatClass(...);
job.setOutputFormatClass(DBOutputFormat.class);
DBInputFormat.setInput(...);
DBOutputFormat.setOutput(
job,
"output", // output table name
new String[] { "name", "count" } //table columns
);
System.exit(job.waitForCompletion(true) ? 0 : 1);
}
}

Using database default values with Linq to SQL codewise

I am using Dynamic Data with linq to SQL and SQL Server 2008.
I have a GUID column that gets his value from the default value with newguid(). When I set IsDbGenerated to true in the designer it works like a charm.
But when I renew table this property is set back to false again. So I added it to the metadata. For some reason it's not being pickup, "00000000-0000-0000-0000-000000000000" is being inserted in database. The displayname and readonly change are being pick up.
What am I missing?
[MetadataType(typeof(CMS_Data_HistoryMetadata))]
public partial class CMS_Data_History
{
}
[TableName("Content")]
public class CMS_Data_HistoryMetadata
{
[DisplayName("Pagina Title")]
public object pageTitleBar { get; set; }
[ReadOnly(true)]
[DisplayName("Versie")]
public object version_date { get; set; }
[ColumnAttribute(IsDbGenerated = true)]
public object entity_id;
}
I solved the problem by extending the partial insert en update class and check there if the guid is filled
partial void
InsertCMS_Data_History(CMS_Data_History
instance)
{
if(instance.entity_id == Guid.Empty)
{
instance.entity_id = Guid.NewGuid();
}
this.ExecuteDynamicInsert(instance);
}
partial void UpdateCMS_Data_History(CMS_Data_History
instance)
{
if (instance.version_date == DateTime.MinValue)
{
instance.version_date = DateTime.Now;
}
this.ExecuteDynamicUpdate(instance);
}