I have a struts2 app with spring transactions and JPA2 over hibernate. The problem is that I have some rows in the database that are changed by an external source (some mysql triggers) and in my front app I have an ajax script that checks this values every 2 seconds. I always need to get the latest value, and not a cached one, and for this I found 2 solutions :
String sql = "FROM MyEntity WHERE xId=:id AND connect!=0 AND complete=0 AND (error=NULL OR error=0)";
Query q = this.em.createQuery(sql).setHint("org.hibernate.cacheable", false).setParameter("agId", agentId);
rs = q.getResultList();
if(rs.size() == 1){
intermedObj = (Intermed) rs.get(0);
}
and the other:
String sql = "FROM MyEntity WHERE xxId=:id AND connect!=0 AND complete=0 AND (error=NULL OR error=0)";
Query q = this.em.createQuery(sql).setParameter("agId", agentId);
rs = q.getResultList();
if(rs.size() == 1){
intermedObj = (Intermed) rs.get(0);
//get latest object from DB
em.refresh(intermedObj);
}
em is a instance of EntityManager which is managed by spring.
So, the question is: which is the best approach from these 2? Or maybe there is a better one ?
So you are right, I used hql there, I still have to learn a lot about hibernate an jpa, and java in general. So I guess that the correct way to write that cod in JPQL would be:
String sql = "SELECT m FROM MyEntity m WHERE m.xxId=:id AND m.connect!=0 AND m.complete=0 AND (m.error!=1)";
Query q = this.em.createQuery(sql).setParameter("agId", agentId);
rs = q.getResultList();
if(rs.size() == 1){
intermedObj = (Intermed) rs.get(0);
//get latest object from DB
em.refresh(intermedObj);
}
So the question would be, is this the proper way to make sure that I got the latest row from DB and not a cached record?
As regarding leve2 cache question I do not know if this is activated. How do I check that?
Related
Similar to the question answered here https://stackoverflow.com/a/42932812/1321510 we need to execute a raw sql query. For the query we don't have any db context model (so any .FromSql answers won't work for us). However we need to execute it within an existing transaction (created with context.Database.BeginTransaction()). All solutions found on SO do not work with existing transactions.
Example:
var connection = context.Database.GetDbConnection();
using (var command = connection.CreateCommand())
{
command.CommandText = sql;
command.Transaction = context.Database.CurrentTransaction.GetDbTransaction();
var executeReader = command.ExecuteReader();
var values = new object[executeReader.FieldCount];
if (!executeReader.Read())
{
return values;
}
executeReader.GetValues(values);
return values;
}
}
Commiting the transaction then throws System.InvalidOperationException: 'This MySqlConnection is already in use. See https://fl.vu/mysql-conn-reuse'.
The provided link in the exception doesn't seem helpful at all, since we're neither using async nor using the connection whilst reading from it.
We're using Pomelo.EntityFrameworkCore.MySql as the database connector.
Hi i have this managed bean where it makes MySQL queries, the problem here is the SQL statement makes a '=' condition instead of 'LIKE'
Here is the code in my managed bean.
Connection con = ds.getConnection();
try{
if (con == null) {
throw new SQLException("Can't get database connection");
}
}
finally {
PreparedStatement ps = con.prepareStatement(
"SELECT * FROM Clients WHERE Machine LIKE '53'");
//get customer data from database
ResultSet result = ps.executeQuery();
con.close();
List list;
list = new ArrayList();
while (result.next()) {
Customer cust = new Customer();
cust.setMachine(result.getLong("Machine"));
cust.setCompany(result.getString("Company"));
cust.setContact(result.getString("Contact"));
cust.setPhone(result.getLong("Phone"));
cust.setEmail(result.getString("Email"));
//store all data into a List
list.add(cust);
}
return list;
Here the SELECT command does not pull all the numbers in 'Machine' column which is like 53, but if i enter a whole value, such as the complete number (53544) in place of 53 then the result is pulled up. I am confused !!
Also if i replace the above select statement with SELECT * FROM Clients the entire database is stored in list. Any ideas ?
Use wildcards:
Like '%53%'
...means everything that contains '53'.
Like '%53' - it ends with 53
LIKE '53%' - it starts with 53
You can also use _ if You want to replace a single character.
You can find a descriptipn HERE
You sql query should be
"SELECT * FROM Clients WHERE Machine LIKE '%53%'
I'm using Linq to query MS CRM 2011 Web Services. I've got a query that results in very poor SQL, it fetches too much intermediary data and its performance is horrible!! I'm new to it, so it may very well be the way I'm using it...
I've got two entities linked via an N-N relationship: Product and SalesLink. I want to recover a bunch of Product from their SerialNumber along with all SalesLink associated to them.
This is the query I have using PredicateBuilder:
// Build inner OR predicate on Serial Number list
var innerPredicate = PredicateBuilder.False<Xrm.c_product>();
foreach (string sn in serialNumbers) {
string temp = sn; // This temp assignement is important!
innerPredicate = innerPredicate.Or(p => p.c_SerialNumber == temp);
}
// Combine predicate with outer AND predicate
var predicate = PredicateBuilder.True<Xrm.c_product>();
predicate = predicate.And(innerPredicate);
predicate = predicate.And(p => p.statecode == (int)CrmStateValueType.Active);
// Inner Join Query
var prodAndLinks = from p in orgContext.CreateQuery<Xrm.c_product>().AsExpandable()
.Where(predicate)
.AsEnumerable()
join link in orgContext.CreateQuery<Xrm.c_saleslink>()
on p.Id equals link.c_ProductSalesLinkId.Id
where link.statecode == (int)CrmStateValueType.Active
select new {
productId = p.Id
, productSerialNumber = p.c_SerialNumber
, accountId = link.c_Account.Id
, accountName = link.c_Account.Name
};
...
Using SQL profiler, I saw that it causes an intermediate SQL query that has no WHERE clause, looking like this:
select
top 5001 "c_saleslink0".statecode as "statecode"
...
, "c_saleslink0".ModifiedOnBehalfByName as "modifiedonbehalfbyname"
, "c_saleslink0".ModifiedOnBehalfByYomiName as "modifiedonbehalfbyyominame"
from
c_saleslink as "c_saleslink0" order by
"c_saleslink0".c_saleslinkId asc
This returns a huge amount of (useless) data. I think the join is done on the client side instead of on the DB side...
How should I improve this query? I runs in around 3 minutes and that's totally unacceptable.
Thanks.
"Solution"
Based on Daryl's answer to use QueryExpression instead of Linq to CRM, I got this which gets the exact same result.
var qe = new QueryExpression("c_product");
qe.ColumnSet = new ColumnSet("c_serialnumber");
var filter = qe.Criteria.AddFilter(LogicalOperator.Or);
filter.AddCondition("c_serialnumber", ConditionOperator.In, serialNumbers.ToArray());
var link = qe.AddLink("c_saleslink", "c_productid", "c_productsaleslinkid");
link.LinkCriteria.AddCondition("statecode", ConditionOperator.Equal, (int)CrmStateValueType.Active);
link.Columns.AddColumns("c_account");
var entities = serviceProxy.RetrieveMultiple(qe).Entities.ToList();;
var prodAndLinks = entities.Select(x => x.ToEntity<Xrm.c_product>()).Select(x =>
new {
productId = x.c_productId
, productSerialNumber = x.c_SerialNumber
, accountId = ((Microsoft.Xrm.Sdk.EntityReference)((Microsoft.Xrm.Sdk.AliasedValue)x["c_saleslink1.c_account"]).Value).Id
, accountName = ((Microsoft.Xrm.Sdk.EntityReference)((Microsoft.Xrm.Sdk.AliasedValue)x["c_saleslink1.c_account"]).Value).Name
}).ToList();
I really would have liked to find a solution using Linq, but it seems to Linq to CRM is just not there yet...
95% of the time when you're having performance issues with a complicated query in CRM, the easiest way to improve the performance is to run a straight SQL query against the database (assuming this is not CRM online of course). This may be one of the 5% of the time.
In your case, the major performance issue you're experiencing is due to the predicate builder forcing a CRM Server (not the SQL database) side join of data. If you used a Query Expression (which is what your link statement get's translated) you can specify a Condition Expression with an IN operator that would allow you to pass in your serialNumbers collection. You could also use FetchXml as well. Both of these methods would allow CRM to perform a SQL side join.
Edit:
This should get you 80% of the way with Query Expressions:
IOrganizationService service = GetService();
var qe = new QueryExpression("c_product");
var filter = qe.Criteria.AddFilter(LogicalOperator.Or);
filter.AddCondition("c_serialnumber", ConditionOperator.In, serialNumbers.ToArray());
var link = qe.AddLink("c_saleslink", "c_productid", "c_productsaleslinkid");
link.LinkCriteria.AddCondition("statecode", ConditionOperator.Equal, (int)CrmStateValueType.Active);
link.Columns.AddColumns("c_Account");
var entities = service.RetrieveMultiple(qe).Entities.ToList();
You will probably find you can get better control by not using Linq to Crm. You could try:
FetchXml, this is an xml syntax, similar in approach to tsql MSDN.
QueryExpression, MSDN.
You could issue a RetrieveRequest, blog.
I'm having trouble with the implementation of SqlDependency in my project.
I'm using SqlDependency in a WCF Service. WCF Service then holds in memory cache all results from all tables in order to have a huge speed gain. Everything seems to be working fine, except when I'm doing a table row update. If I add or delete a row in my table, DataContext is refreshed and cache is invalidated without problems. But when it comes to a table row update, nothing happens, the cache is not invalidated and when I look in debug mode at the content of DataContext, no changes seems to be there.
Here's the code I'm using (note that I'm using the System.Runtime.Caching object) :
public static List<T> LinqCache<T>(this Table<T> query) where T : class
{
ObjectCache cache = MemoryCache.Default;
string tableName =
query.Context.Mapping.GetTable(typeof(T)).TableName;
List<T> result = cache[tableName] as List<T>;
if (result == null)
{
using (SqlConnection conn =
new SqlConnection(query.Context.Connection.ConnectionString))
{
conn.Open();
SqlCommand cmd = new SqlCommand(
query.Context.GetCommand(query).CommandText, conn);
cmd.Notification = null;
cmd.NotificationAutoEnlist = true;
SqlDependency dependency = new SqlDependency(cmd);
SqlChangeMonitor sqlMonitor =
new SqlChangeMonitor(dependency);
CacheItemPolicy policy = new CacheItemPolicy();
policy.ChangeMonitors.Add(sqlMonitor);
cmd.ExecuteNonQuery();
result = query.ToList();
cache.Set(tableName, result, policy);
}
}
return result;
}
I created an extension method so all I have to do is to query any table like that :
List<MyTable> list = context.MyTable.LinqCache();
My DataContext is opened at the Global.asax Application_OnStart and stored in cache, so I can use it whenever I want in my WCF Service. As well at this moment I'm opening the SqlDependency object with
SqlDependency.Start(
ConfigurationManager.ConnectionStrings[myConnectionString].ConnectionString);
So, is that a limitation of SqlDependency, or I'm doing something wrong/missing something in the process?
I think the problem is that although you do all the work in setting up the command object you then do:
cmd.ExecuteNonQuery();
result = query.ToList();
Which is going to use your SQL Command and throw away the results then LINQ to SQL will generate it's own internally via query.ToList(). Thankfully you can ask LINQ to SQL to execute your own command and translate the results for you so try replacing those two lines with:
results = db.Translate<T>(cmd.ExecuteReader());
I am trying to get the records from the 'many' table of a one-to-many relationship and add them as a list to the relevant record from the 'one' table.
I am also trying to do this in a single database request.
Code derived from Linq to Sql - Populate JOIN result into a List almost achieves the intended result, but makes one database request per entry in the 'one' table which is unacceptable. That failing code is here:
var res = from variable in _dc.GetTable<VARIABLE>()
select new { x = variable, y = variable.VARIABLE_VALUEs };
However if I do a similar query but loop through all the results, then only a single database request is made. This code achieves all goals:
var res = from variable in _dc.GetTable<VARIABLE>()
select variable;
List<GDO.Variable> output = new List<GDO.Variable>();
foreach (var v2 in res)
{
List<GDO.VariableValue> values = new List<GDO.VariableValue>();
foreach (var vv in v2.VARIABLE_VALUEs)
{
values.Add(VariableValue.EntityToGDO(vv));
}
output.Add(EntityToGDO(v2));
output[output.Count - 1].VariableValues = values;
}
However the latter code is ugly as hell, and it really feels like something that should be do-able in a single linq query.
So, how can this be done in a single linq query that makes only a single database query?
In both cases the table is set to preload using the following code:
_dc = _db.CreateLinqDataContext();
var loadOptions = new DataLoadOptions();
loadOptions.LoadWith<VARIABLE>(v => v.VARIABLE_VALUEs);
_dc.LoadOptions = loadOptions;
I am using .NET 3.5, and the database back-end was generated using SqlMetal.
This link may help
http://msdn.microsoft.com/en-us/vcsharp/aa336746.aspx
Look under join operators. You'll probably have to change from using extension syntax other syntax too. Like this,
var = from obj in dc.Table
from obj2 in dc.Table2
where condition
select