Is it possible to get Linq2Sql to emit a NOLOCK in its SQL? And if so, how?
Yes it is, so here's the entry from my blog:
The NOLOCK hint is essentially the
same as wrapping a query in a
transaction whose "isolation level" is
set to "read uncommitted". It means
that the query doesn't care if stuff
is in the process of being written to
the rows it's reading from - it'll
read that "dirty" data and return it
as part of the result set.
Turns out that you can do the whole
"read uncommitted" transaction thing
using the old System.Transactions
namespace introduced in .NET 2.0.
Here's some sample code:
using (var txn = new TransactionScope(
TransactionScopeOption.Required,
new TransactionOptions
{
IsolationLevel = IsolationLevel.ReadUncommitted
}
))
{
// Your LINQ to SQL query goes here
}
So I'm creating a new TransactionScope
object and telling it to use a
read-uncommitted isolation level. The
query within the "using" statement now
acts as if all its tables were reading
with the NOLOCK hint.
Here are the first results from a Google search for "linq sql nolock":
InfoQ: Implementing NOLOCK with LINQ to SQL and LINQ to Entities
Matt Hamilton - LINQ to SQL and NOLOCK Hints : Mad Props!
Scott Hanselman's Computer Zen - Getting LINQ to SQL and LINQ to ...
Further to theKing's LinqPad My Extensions addition:
public static IQueryable<T> DumpNoLock<T>(this IQueryable<T> query)
{
using (var txn = GetNewReadUncommittedScope())
{
return query.Dump();
}
}
public static System.Transactions.TransactionScope GetNewReadUncommittedScope()
{
return new System.Transactions.TransactionScope(
System.Transactions.TransactionScopeOption.RequiresNew,
new System.Transactions.TransactionOptions
{
IsolationLevel = System.Transactions.IsolationLevel.ReadUncommitted
});
}
public static IQueryable<T> DumpNoLock<T>(this IQueryable<T> query, string description)
{
using (var txn = GetNewReadUncommittedScope())
{
return query.Dump(description);
}
}
public static List<T> ToListNoLock<T>(this IQueryable<T> query)
{
using (var txn = GetNewReadUncommittedScope())
{
return query.ToList();
}
}
public static U NoLock<T,U>(this IQueryable<T> query, Func<IQueryable<T>,U> expr)
{
using (var txn = GetNewReadUncommittedScope())
{
return expr(query);
}
}
The last one means you can do a NOLOCK on any evaluating queries you haven't a NoLock explicitly written for (like I've got for ToListNoLock above). So, for example:
somequery.NoLock((x)=>x.Count()).Dump();
will evaluate the query with NOLOCK.
Note that you have to ensure you're evaluating the query. E.g. .NoLock((x)=>x.Distinct()).Count().Dump() won't do anything usefully different from .Distinct().Count().Dump().
A simple way may be to run a command on your DataContext class
using (var dataContext = new DataContext())
{
dataContext.ExecuteCommand("SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED");
// Your SQL query
}
Here is an extension method to use with LINQPAD
public static IQueryable<T> Dump2<T>(this IQueryable<T> query)
{
using (var txn = new System.Transactions.TransactionScope(TransactionScopeOption.RequiresNew,
new TransactionOptions
{
IsolationLevel = System.Transactions.IsolationLevel.ReadUncommitted
}))
{
return query.Dump();
}
}
Then you can call it as:
MyTable.Where(t => t.Title = "Blah").Dump2();
In my case, Entity Framework 5 (based on #Soppus answer):
private FoobarEntities db = new FoobarEntities();
public FoobarController()
{
db.Database.ExecuteSqlCommand("SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED");
}
Related
Using EF Core 2.2.6 and Pomelo.EntityFrameworkCore.MySql 2.2.6 (with MySqlConnector 0.59.2)). I have a model for UserData:
public class UserData
{
[DatabaseGenerated(DatabaseGeneratedOption.None)]
public ulong ID { get; private set; }
[Required]
public Dictionary<string, InventoryItem> Inventory { get; set; }
public UserData()
{
Data = new Dictionary<string, string>();
}
}
I have a REST method that can be called that will add items to the user inventory:
using (var transaction = context.Database.BeginTransaction())
{
UserData data = await context.UserData.FindAsync(userId);
// there is code here to detect duplicate entries/etc, but I've removed it for brevity
foreach (var item in items) data.Inventory.Add(item.ItemId, item);
context.UserData.Update(data);
await context.SaveChangesAsync();
transaction.Commit();
}
If two or more calls to this method are made with the same user id then I get concurrent accesses (despite the transaction). This causes the data to sometimes be incorrect. For example, if the inventory is empty and then two calls are made to add items simultaneously (item A and item B), sometimes the database will only contain either A or B, and not both. From logging it appears that it is possible for EF to read from the database while the other read/write is still occurring, causing the code to have the incorrect state of the inventory for when it tries to write back to the db. So I tried marking the isolation level as serializable.
using (var transaction = context.Database.BeginTransaction(System.Data.IsolationLevel.Serializable))
Now I sometimes see an exception:
MySql.Data.MySqlClient.MySqlException (0x80004005): Deadlock found when trying to get lock; try restarting transaction
I don't understand how this code could deadlock... Anyways, I tried to proceed by wrapping this whole thing in a try/catch, and retry:
public static async Task<ResponseError> AddUserItem(Controller controller, MyContext context, ulong userId, List<InventoryItem> items, int retry = 5)
{
ResponseError result = null;
try
{
using (var transaction = context.Database.BeginTransaction(System.Data.IsolationLevel.Serializable))
{
UserData data = await context.UserData.FindAsync(userId);
// there is code here to detect duplicate entries/etc, but I've removed it for brevity
foreach (var item in items) data.Inventory.Add(item.ItemId, item);
context.UserData.Update(data);
await context.SaveChangesAsync();
transaction.Commit();
}
}
catch (Exception e)
{
if (retry > 0)
{
await Task.Delay(SafeRandomGenerator(10, 500));
return await AddUserItem(controller, context, userId, items, retry--);
}
else
{
// store exception and return error
}
}
return result;
}
And now I am back to the data being sometimes correct, sometimes not. So I think the deadlock is another problem, but this is the only method accessing this data. So, I'm at a loss. Is there a simple way to read from the database (locking the row in the process) and then writing back (releasing the lock on write) using EF Core? I've looked at using concurrency tokens, but this seems overkill for what appears (on the surface to me) to be a trivial task.
I added logging for mysql connector as well as asp.net server and can see the following failure:
fail: Microsoft.EntityFrameworkCore.Database.Command[20102]
=> RequestId:0HLUD39EILP3R:00000001 RequestPath:/client/AddUserItem => Server.Controllers.ClientController.AddUserItem (ServerSoftware)
Failed executing DbCommand (78ms) [Parameters=[#p1='?' (DbType = UInt64), #p0='?' (Size = 4000)], CommandType='Text', CommandTimeout='30']
UPDATE `UserData` SET `Inventory` = #p0
WHERE `ID` = #p1;
SELECT ROW_COUNT();
A total hack is to just delay the arrival of the queries by a bit. This works because the client is most likely to generate these calls on load. Normally back-to-back calls aren't expected, so spreading them out in time by delaying on arrival works. However, I'd rather find a correct approach, since this just makes it less likely to be an issue:
ResponseError result = null;
await Task.Delay(SafeRandomGenerator(100, 500));
using (var transaction = context.Database.BeginTransaction(System.Data.IsolationLevel.Serializable))
// etc
This isn't a good answer, because it isn't what I wanted to do, but I'll post it here as it did solve my problem. My problem was that I was trying to read the database row, modify it in asp.net, and then write it back, all within a single transaction and while avoiding deadlocks. The backing field is JSON type, and MySQL provides some JSON functions to help modify that JSON directly in the database. This required me to write SQL statements directly instead of using EF, but it did work.
The first trick was to ensure I could create the row if it didn't exist, without requiring a transaction and lock.
INSERT INTO UserData VALUES ({0},'{{}}','{{}}') ON DUPLICATE KEY UPDATE ID = {0};
I used JSON_REMOVE to delete keys from the JSON field:
UPDATE UserData as S set S.Inventory = JSON_REMOVE(S.Inventory,{1}) WHERE S.ID = {0};
and JSON_SET to add/modify entries:
UPDATE UserData as S set S.Inventory = JSON_SET(S.Inventory,{1},CAST({2} as JSON)) WHERE S.ID = {0};
Note, if you're using EF Core and want to call this using FromSql then you need to return the entity as part of your SQL statement. So you'll need to add something like this to each SQL statement:
SELECT * from UserData where ID = {0} LIMIT 1;
Here is a full working example as an extension method:
public static async Task<UserData> FindOrCreateAsync(this IQueryable<UserData> table, ulong userId)
{
string sql = "INSERT INTO UserData VALUES ({0},'{{}}','{{}}') ON DUPLICATE KEY UPDATE ID = {0}; SELECT * FROM UserData WHERE ID={0} LIMIT 1;";
return await table.FromSql(sql, userId).SingleOrDefaultAsync();
}
public static async Task<UserData> JsonRemoveInventory(this DbSet<UserData> table, ulong userId, string key)
{
if (!key.StartsWith("$.")) key = $"$.\"{key}\"";
string sql = "UPDATE UserData as S set S.Inventory = JSON_REMOVE(S.Inventory,{1}) WHERE S.ID = {0}; SELECT * from UserData where ID = {0} LIMIT 1;";
return await table.AsNoTracking().FromSql(sql, userId, key).SingleOrDefaultAsync();
}
Usage:
var data = await context.UserData.FindOrCreateAsync(userId);
await context.UserData.JsonRemoveInventory(userId, itemId);
I have one very annoying issue, i have read all existing documentation online and read all stackoverflow questions and answers related to this topic, but simply can not make this to work!
I am really desperate and i do not know what i am missing so i will try to give you all what i have so far. Basically what i am trying to do is to save a lot of data with one query instead of multiple queries for each object. As you can suspect i am using Spring Boot, Hibernate and MySql.
So basic facts that i have learned so far based on what i read related to "batch insert using mysql + hibernate" is next:
Mysql does not support Sequence ID, so i can not use it, like i could use it for PostgreSql
Hibernate does not support batch insert out of the box, there are couple of app properties that needs to be added
And this is what i have so far:
Application properties that i added:
spring.datasource.url=jdbc:mysql://localhost:32803/db?rewriteBatchedStatements=true
spring.jpa.properties.hibernate.jdbc.batch_size=50
spring.jpa.properties.hibernate.jdbc.lob.non_contextual_creation=true
spring.jpa.open-in-view=false
spring.jpa.properties.hibernate.show_sql=true
spring.jpa.properties.hibernate.use_sql_comments=true
spring.jpa.properties.hibernate.format_sql=true
spring.jpa.properties.hibernate.order_updates=true
spring.jpa.properties.hibernate.order_inserts=true
spring.jpa.properties.hibernate.batch_versioned_data=true
spring.jpa.properties.hibernate.id.new_generator_mappings=false
spring.jpa.show-sql=true
spring.jpa.properties.hibernate.type=trace
spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.MySQL5Dialect
spring.jpa.properties.hibernate.current_session_context_class=org.springframework.orm.hibernate5.SpringSessionContext
#Entity
data class Person (
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
val id: Long?,
var firstName: String,
var lastName: String,
var country: String,
var org: Int
)
What i want is to save a lot of Persons at once, as you can see i added batch size 50, if i understood correctly that means i will do one database hit per 50 persons while saving. (correct me if i am wrong)
And at the end i have Repository where i execute that batch insert:
#Repository
class PersonRepositoryCustomImpl : PersonRepositoryCustom {
#PersistenceContext
private val entityManager: EntityManager? = null
#Transactional
override fun batchSave2(persons: Set<Person>) {
val session = entityManager!!.unwrap(Session::class.java)
persons.forEachIndexed { index, person ->
if ( index % 50 == 0 ) {
session!!.flush()
session.clear()
}
session!!.save(person)
}
session.close()
}
#Transactional
override fun <T : Person?> batchSave(entities: Collection<T>): Collection<T>? {
val savedEntities: MutableList<T> = ArrayList(entities.size)
var i = 0
for (t in entities) {
savedEntities.add(persistOrMerge(t))
i++
if (i % 50 == 0) { // Flush a batch of inserts and release memory.
entityManager!!.flush()
entityManager.clear()
}
}
return savedEntities
}
private fun <T : Configuration?> persistOrMerge(t: T): T {
return if (t!!.id == null) {
entityManager!!.persist(t)
t
} else {
entityManager!!.merge(t)
}
}
}
So here you can see that i have tried to make this works on 2 almost the same ways, but of course both of them seems not to work.
In order to confirm that i am actually doing batch insert i am looking at this:
https://tableplus.com/blog/2018/10/how-to-show-queries-log-in-mysql.html
so basically that should show me queries that are being executed on DB, and there i can see that for each person object i have one insert statement.
Basically that result of this query:
SELECT
*
FROM
mysql.general_log;
And there i can clearly see that i have multiple insert statements that do one query per object (person).
Edit:
https://blog.arnoldgalovics.com/configuring-a-datasource-proxy-in-spring-boot/
I have also implemented datasource proxy, which proved me that i am not doing batch insert:
Name:, Time:1, Success:True, Type:Prepared, Batch:False, QuerySize:1, BatchSize:0, Query:["insert into person(firstName, lastName, country, org) values (?, ?, ?, ?)"], Params:[(10,John,Johny,USA,ORG)]
i have multiple of records like this one.
Thanks in advance for any kind of help!
Just to give an answer in case someone needs it:
So long story short i was not able to make MySql + hibernate batch processing work, for sake of testing i actually was able to make it work with PostgreSQL.
But anyway if anyone needs this with MySql there is a way using JDBC batch processing, and code more or less is very straight forward:
private String INSERT_SQL_PARAMS = "INSERT INTO item_params(p_key, p_value, item_id) values (?,?,?)"
override fun saveParams(configParams: Set<ItemParam>) {
jdbcTemplate!!.batchUpdate(INSERT_SQL_PARAMS , configParams, 3000) { ps, argument ->
ps.setLong(1, argument.pKey)
ps.setString(2, argument.pValue)
ps.setString(3, argument.itemId)
}
}
mysql doesn't support sequences so you can generate one using this strategy in your entity class:
#Id
#GeneratedValue(generator = "voucher_sequence")
#GenericGenerator(name = "voucher_sequence",strategy = "increment")
and also in your mysql url enable batching:
jdbc:mysql://localhost:3306/db?rewriteBatchedStatements=true
and in your application.properties:
spring.jpa.properties.hibernate.jdbc.batch_size=20
spring.jpa.properties.hibernate.order_inserts=true
spring.jpa.properties.hibernate.order_updates=true
then use saveAll() method to persist list of entities
I'm working in report module, in order to do that I'm creating different stored procedures. I create the procedure with in parameters and then create a class to map the row (resultSet)
I think that's the best way to work arround performance and clarity.(what do you think about that?)
I'm using play framework and ebean orm (2.7.7)
I'm calling the store procedure and getting the resultSet, but I would like to use ebean in order to cast automaticly the row to model... other option is take the row-cell and cast it in a property but I'm trying to avoid it.
This is the current approach
Is this the best way to call an stored procedure?
Transaction tx = Ebean.beginTransaction();
String sql = "{CALL report(?, ?, ?, ?, ?, ?)}";
CallableStatement callableStatement = null;
try {
Connection dbConnection = tx.getConnection();
callableStatement = dbConnection.prepareCall(sql);
callableStatement.setInt(1, 3);
callableStatement.setInt(2, 5);
callableStatement.setInt(3, 2013);
callableStatement.setInt(4, 1);
callableStatement.setInt(5, 2014);
callableStatement.setInt(6, 5);
ResultSet rs = callableStatement.executeQuery(sql);
while (rs.next()) {
//HOW TO CONVER row -> model ?
}
Ebean.commitTransaction();
} catch (SQLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
I've discarded RawSQL and Query because received an error
RuntimeException: Error parsing sql, can not find SELECT keyword in: xxxxx
Also I found other option... using CallableSql
String sql = "{call sp_order_mod(?,?)}";
CallableSql cs = Ebean.createCallableSql(sql);
cs.setParameter(1, "turbo");
cs.registerOut(2, Types.INTEGER);
Ebean.execute(cs);
// read the out parameter
Integer returnValue = (Integer) cs.getObject(2);
but in this case I need to return a ResultSet not simply parameter.
I'm going to share my own solution.
I get a class called ResultSetUtils.(you can google it some implementation)
I added a generic method in order to return a typed list from resultset
public static <T> List<T> populateInList(Class<T> c, final ResultSet rs) {
List<T> listTyped = new ArrayList<T>();
try {
if (rs != null) {
while (rs.next()) {
T o = c.newInstance();
// MAGIC LINE
populate(o, rs);
listTyped.add(o);
}
rs.close();
}
} catch (final Exception e) {
// TODO Auto-generated catch block
System.err.println(e.getMessage());
}
return listTyped;
}
This class to do the population use org.apache.commons.beanutils package
BeanUtils.populate(bean, propertiesRealName);
Using
private static void callingProcedureTest() {
Logger.debug("Init callingProcedureTest");
Transaction tx = Ebean.beginTransaction();
// String sql = "{CALL sp_report_test(3, 5, 2013, 1, 2014, 5)}";
String sql = "CALL sp_report_test(?, ?, ?, ?, ?, ?);";
try {
Connection dbConnection = tx.getConnection();
CallableStatement callableStatement = dbConnection.prepareCall(sql);
callableStatement.setInt(1, 3);
callableStatement.setInt(2, 5);
callableStatement.setInt(3, 2013);
callableStatement.setInt(4, 1);
callableStatement.setInt(5, 2014);
callableStatement.setInt(6, 5);
Logger.debug("SQL > " + sql);
ResultSet rs = callableStatement.executeQuery();
Class<ReportTestResult> c = ReportTestResult.class;
//************** MAGIC LINE, converting ResultSet to Model
List<ReportTestResult> listResult = ResultSetUtils.populateInList(c, rs);
for (ReportTestResult item : listResult) {
Logger.debug("item.firstName> " + item.firstName);
Logger.debug("item.lastName > " + item.lastName);
Logger.debug("item.year > " + item.year);
}
Ebean.commitTransaction();
} catch (Exception e) {
Ebean.rollbackTransaction();
// TODO Auto-generated catch block
e.printStackTrace();
}finally{
Ebean.endTransaction();
}
}
Plus about architecture and implementation
For each report I'm going to create:
a Result class (eg ReportTestResult)
intention: represent a row of report | simple DTO
a Param class (eg ReportTestParam),
intention: represent the parameters (inputs / ouputs), filters of the report
This class should implements
public interface ReportParam {
public int countParameteres();
public void setParametersInCallableStatement(CallableStatement callableStatement) throws SQLException;
}
a Report class (eg ReportTestReport) this class should extends ReportBase
intention: Knows the stored procedure's name, parameters and dto result
public class ReportTestReport extends ReportBase<ReportTestResult, ReportTestParam> {
#Override
protected String getProcedureName() {
return STORED_NAME;
}
}
many Adapters...
Each report could displayed in different charts, In this case I'm using HighCharts. Order to arrange it, I'm creating different adapters to do that.
EG:
class ReportTestHighChartsAdapter
intention: convert a list of ReportTestResult to series and configure different options of report (eg, title, xAxis etc)
public OptionsHC buildColumnReportV1(){
OptionsHC optionChart = new OptionsHC();
optionChart.chart = new ChartHC("column");
this.setTitle(optionChart);
optionChart.yAxis = new AxisHC(new TitleHC("Fruit eaten"));
.....
return optionChart;
}
OptionsHC is a class that represent option obj in the HighCharts framework.
The final step is converting OptionHC to Json and use it in JavaScript (common use of highCharts)
What's ReportBase?
ReportBase class has the strategy to implements the final called to DB, also manage the transaction
public class ReportTestReport extends ReportBase<ReportTestResult, ReportTestParam> {
...
protected List<TResult> execute(Class<TResult> classT) {
List<TResult> resultDTO = null;
CallableStatement callableStatement = null;
Logger.debug("Running procedure> " + this.getProcedureName());
Transaction tx = Ebean.beginTransaction();
String sql = ProcedureBuilder.build(this.getProcedureName(), this.countParameters());
Logger.debug("SQL > " + sql);
try {
Connection dbConnection = tx.getConnection();
callableStatement = dbConnection.prepareCall(sql);
this.getFilter().setParametersInCallableStatement(callableStatement);
ResultSet rs = callableStatement.executeQuery();
resultDTO = ResultSetUtils.populateInList(classT, rs);
Ebean.commitTransaction();
Logger.debug("commitTransaction > " + sql);
} catch (Exception e) {
Ebean.rollbackTransaction();
Logger.debug("rollbackTransaction > " + sql);
// TODO Auto-generated catch block
e.printStackTrace();
}finally{
Ebean.endTransaction();
}
return resultDTO;
}
...
}
Currently the support for stored procedures in Ebean is not orientated to what you are trying to do. Hence you are not going to get much joy from using CallableSql or RawSql.
>> a class to map the row (resultSet) I think that's the best way to work around performance and clarity
Yes, I can understand your motivation.
>> How to convert ResultSet into model
Currently there is no good solution. The best solution would be to enhance RawSql so that you can set a ResultSet onto it. One of the things RawSql does is provide the mapping of resultSet columns to model properties and that is what Ebean needs internally. The enhancement/code change would be to be able to set a resultSet onto the RawSql object ... and get Ebean internally to skip the creation of the resultSet ( preparedStatement, binding parameters and executeQuery()). In terms of Ebean internals this is all done in the CQuery.prepareBindExecuteQueryWithOption() method. That is, if the RawSql has already provided a resultSet skip those things.
The big benefit of doing this rather than just rolling your own row -> model mapping code is that the resulting beans would all still have lazy loading / partial object knowledge etc. They would behave exactly like any other beans that Ebean builds as part of it query mechanism.
So that said, I'm personally away for a week ... so you aren't going to hear back from me until after that. If you want to get into it yourself then internally CQuery.prepareBindExecuteQueryWithOption() is the code you will need to modify.
If you have been following the ebean google group you'll know that but just in case you have not been note that the Model and Finder objects from Play have been incorporated into Ebean just in the last week. This helps both projects ... reduces confusion etc. The Ebean source in github master is at 4.0.4 and the bytecode enhancement in 4.x is different and I don't believe supported in Play.
I'm basically going offline for a week now so I'll look back into this after that.
Cheers, Rob.
I want to make several entries into a MySQL database. Because Some of the tables reference others via foreign key I have to get back the inserted ID to inject them in my next statements.
I have 4 classes:
LodgerFormTest
RentForm
RentObject
House
and the class which inserts the MySQL statements into the db: sql_statements
When I want to send a SQL statement I am getting a nullPointer Exception!
The Action listener of the House-class (this is the first sql-statement I have to send) looks like this:
saveButton.addActionListener(new ActionListener(){
public void actionPerformed(ActionEvent e){
System.out.println("saveButton");
// sql_statements statements = new sql_statements();
sql_statements.performHouse(strasse.getText(), plz.getText(), ort.getText());
mainmenu.create();
rentnerFrame.dispose();
}
});
all methods and variables I am using in sql_statements are static! Therefore I am not instantiating an object.
here is the method "performHouse" in sql_statements
public static void performHouse(String strasse, String plz, String ort) {
String sql = "insert into haus(strasse, plz, ort) values (?,?,?)";
System.out.println(sql);
try{
ps = connect.prepareStatement(sql, ps.RETURN_GENERATED_KEYS);
ps.setString(1, strasse);
ps.setString(2, plz);
ps.setString(3, ort);
ps.execute();
rs = ps.getGeneratedKeys();
if(rs != null && rs.next()) {
// Retrieve the auto generated key(s).
key_idhaus = rs.getLong(1);
System.out.println("idhaus: " + key_idhaus);
}
}catch(Exception ex){
System.out.println(ex);
}
} // close performHouse-methode
I cant debug because I get a "Source not found." error in the debug view.
Can anybody help please?
okay I just wanted to finish the task so I implemented a solution which surely is all but not OO:
I implemented redundant code into all of the 4 classes and every class implements their own sql statements. quick and dirty ;)
I have a Linq query that looks something like this:
var query = from x in table where SomeFunctionReturnsBool() select;
private bool SomeFunctionReturnsBool()
{
return true;
}
This returns and exception that says "SomeFunctionReturnsBool has no supported translation to SQL". I get that this is because it wants to treat "SomeFunctionReturnsBool" as an expression to evaluate as SQL, but it can't.
Although this Linq query isn't complicated, the real ones are. How can I accomplish what I'm trying to do here, which is to break out pieces of the query to hopefully make it more readable?
Jeff
UPDATE
Good answers. I am trying now to work with expressions instead, but this code gets me "cannot resolve method Where(lambda expression)":
var query = from x in table where SomeFunctionReturnsBool() select x;
private Expression<Func<EligibilityTempTable, bool>> SomeFunctionReturnsBool
{
return (x) => true;
}
Another way is to use Expression<Func<YourType, bool>> predicate...
var query = from x in table where SomeFunctionReturnsBool() select;
Edit: I don't usually do it the way I've shown above... I was just getting that from the code above. Here is the way I usually implement it. Because then you can tack on additional Enumerable methods or comment them out during debugging.
var results = table.Where(SomeFunctionReturnsBool())
.OrderBy(yt => yt.YourProperty)
//.Skip(pageCount * pageSize) //Just showing how you can easily comment out parts...
//.Take(pageSize)
.ToList(); //Finally executes the query...
private Expression<Func<YourType, boo>> SomeFunctionReturnsBool()
{
return (YourType yt) => yt.YourProperty.StartsWith("a")
&& yt.YourOtherProperty == true;
}
I prefer to use the PredicateBuilder which allows you to build an expression to be used in your Where...
You can do this in LINQ-to-SQL by creating a UDF mapped to the data-context; this involves writing TSQL, and use ctx.SomeFunctionblah(...).
The alternative is to work with expression trees - for example, it could be:
Expression<Func<Customer, bool>> SomeFunc() {
return c => true; // or whatever
}
and use .Where(SomeFunc()) - is that close enough? You can't use the query syntax in this case, but it gets the job done...
Added dodgy Where method to show how you might use it in query syntax. I don't suggest this is fantastic, but you might find it handy.
using System;
using System.Linq;
using System.Linq.Expressions;
static class Program
{
static void Main()
{
using (var ctx = new NorthwindDataContext())
{
ctx.Log = Console.Out;
// fluent API
var qry = ctx.Customers.Where(SomeFunc("a"));
Console.WriteLine(qry.Count());
// custom Where - purely for illustration
qry = from c in ctx.Customers
where SomeFunc("a")
select c;
Console.WriteLine(qry.Count());
}
}
static IQueryable<T> Where<T>(this IQueryable<T> query,
Func<T, Expression<Func<T, bool>>> predicate)
{
if(predicate==null) throw new ArgumentNullException("predicate");
return query.Where(predicate(default(T)));
}
static Expression<Func<Customer, bool>> SomeFunc(string arg)
{
return c => c.CompanyName.Contains(arg);
}
}
Basically, "out of the box", you can't have LINQ-to-SQL execute queries that have custom functions in them. In fact only some native methods that can be translated to SQL can be used.
The easiest way around this can unfortunately affect performance depending on how much data you're bringing back from the DB.
Basically, you can only use custom functions in WHERE statments if the data has already been loaded into memory, i.e, SQL have already executed.
The quickest fix for your example would look like this:
var query = from x in table.ToList() where SomeFunctionReturnsBool() select;
Notice the ToList(). It executes the SQL and puts the data into memory. You can now do whatever you want in the WHERE statement/method.
I would just break them out like so:
Expression<Func<Table, bool>> someTreeThatReturnsBool = x => true;
var query = from x in table where someTreeThatReturnsBool select x;
You could create functions that pass around expression trees.
Don't use query syntax for this.
var query = table.Where( x => SomeFunction(x) );