In .NET 4 and multicore environment, does the linq to sql datacontext object take advantage of the new parallels if we use DataLoadOptions.LoadWith?
EDIT
I know linq to sql does not parallelize ordinary queries. What I want to know is when we specify DataLoadOption.LoadWith, does it use parallelization to perform the match between each entity and its sub entities?
Example:
using(MyDataContext context = new MyDataContext())
{
DataLaodOptions options =new DataLoadOptions();
options.LoadWith<Product>(p=>p.Category);
return this.DataContext.Products.Where(p=>p.SomeCondition);
}
generates the following sql:
Select Id,Name from Categories
Select Id,Name, CategoryId from Products where p.SomeCondition
when all the products are created, will we have a
categories.ToArray();
Parallel.Foreach(products, p =>
{
p.Category == categories.FirstOrDefault(c => c.Id == p.CategoryId);
});
or
categories.ToArray();
foreach(Product product in products)
{
product.Category = categories.FirstOrDefault(c => c.Id == product.CategoryId);
}
?
No, LINQ to SQL does not. There is little to parallelize on the .NET side. All LINQ to SQL does is translating expression trees to SQL queries. SQL Server will execute those SQL statements, and is able to do this in parallel.
Don't forget that while you can do something like this with your LINQ to SQL LINQ query, it isn't a good idea:
// BAD CODE!!! Don't parallelize a LINQ to SQL query
var q =
from customer in db.Customers.AsParallel()
where customer.Id == 5
select customer;
While this yields the correct results, you won't get the performance improvement you are hoping for. PLINQ isn't able to handle IQueryable objects. Therefore, it will just handle an IQueryable as an IEnumerable (thus iterating it). It will process the db.Customers collection in parallel and use multiple threads to filter the customers. While this sounds okay, this means it will retrieve the complete collection of customers from the database! Without the AsParallel construct, LINQ to SQL would be able to optimize this query by adding the WHERE id = #ID to the SQL. SQL Server would than be able to use indexes (and possibly multiple-threads) to fetch the values.
While there is some room for optimization inside the LINQ to SQL engine when it comes to matching entities to its sub entities, there doesn't seem such optimization in the framework currently (or at least, I wasn't able to find any using Reflector).
Related
I am new in Code Igniter and like its active record feature now is there any useful steps or tips or any guidness how do i convert my pervoiusly written simple SQL Queries in CI style like this is my perviouly written simple query
SELECT *
FROM hs_albums
WHERE id NOT IN (SELECT album_id
FROM hs_delete_albums
WHERE user_id = 72
AND del_type = 1)
AND ( created = 72
OR club_id IN (SELECT cbs.id
FROM hs_clubs cbs
INNER JOIN hs_club_permissions cbp
ON cbs.id = cbp.club_id
WHERE cbp.user_id = 72
AND cbp.status = 2)
OR group_id IN (SELECT gps.id
FROM hs_groups gps
INNER JOIN hs_group_permissions grp
ON gps.id = grp.group_id
WHERE grp.user_id = 72
AND grp.status = 2)
OR comp_id IN (SELECT cmp.id
FROM hs_companies cmp
INNER JOIN hs_comp_permissions comp
ON cmp.id = comp.comp_id
WHERE comp.user_id = 72
AND comp.status = 2) )
The short answer is: You don't.
CodeIgniter's Active Record implementation is basically a layer on top of SQL that makes writing queries easier by:
Automatically escaping values
Automatically generating the appropriate query syntax for the database, so that the application can be more easily ported between databases (for instance, if you didn't use Active Record to write a query, and then wanted to move from MySQL to PostgreSQL, then you might well need to rewrite the query to make it work with PostgreSQL)
Providing a syntax for queries in PHP directly, thus avoiding the context switching between PHP and SQL.
However, it can't do everything SQL can do, and while I would always try to use ActiveRecord where possible, there comes a point where you're better off forgetting about using it and just using $this->db->query() to write your query directly. In this case, as mamdouh alramadan has said, CodeIgniter doesn't support subqueries so you can't replicate this query using ActiveRecord anyway.
The thing to remember is that ActiveRecord is a tool, not a requirement. If you're using CodeIgniter and aren't using an ORM instead, you should use it for the reasons mentioned above. However, once it starts getting in the way, you should consider whether it would be better practice to write your query manually instead.
a quick question which may or may not be easily answered.
Currently, in order to return a limited result set of data to my calling reference using SubSonic I use a similar function as below:
_DataSet = from CatSet in t2_aspnet_shopping_item_category.All()
join CatProdAssignedLink in t2_aspnet_shopping_link_categoryproduct.All() on CatSet.CategoryID equals CatProdAssignedLink.CategoryID
join ProdSet in t2_aspnet_shopping_item_product.All() on CatProdAssignedLink.ProductID equals ProdSet.ProductID
where ProdSet.ProductID == __ProductID
orderby CatProdAssignedLink.LinkID ascending
select CatSet;
and select the first item from the data set.
Is there a way to limit the lookup initially to a certain amount of rows? I'm using MySQL as the base database.
You can do that using following:
Using SubSonic:
If you want to get limited records from subsonic function then you can use GetPaged method to get records. To learn more about querying visit this link.
Use GetPaged instead of GetAll function in your query.
Using LINQ
Use Skip and Take methods to get limited records. To learn more about linq visit this link.
My understanding is that the LinqToSql pseudolanguage describes a set using a syntax very similar to SQL and this will allow you to efficiently update a property on a collection of objects:
from b in BugsCollection where b.status = 'closed' set b.status = 'open'
This would update the underlying database using just one SQL statement.
Normally an ORM needs to retieve all of the rows as separate objects, update attributes on each of them and save them individually to the database (at least that's my understanding).
So, how does linq-to-sql avoid having to do this when other orms are not able to avoid it?
The syntax shown in your question is incorrect. LINQ is not intended to have side-effects; it is a query language. The proper way to accomplish what you're looking for is
var x = from b in dataContext.BugsCollection where b.status == "closed";
foreach (var y in x)
y.status = "open";
dataContext.SubmitChanges();
This would generate the single SQL statement that you're talking about. The reason it is able to accomplish this is because of deferred execution - the L2S engine doesn't actually talk to the database until it has to - in this case, because SubmitChanges() was called. L2S then sends the generated SQL statement to the database for execution.
Because LINQ to SQL uses Expression Trees to convert your Query Syntax to actual SQL...it then executes the SQL against the database (rather than pulling all of the data, executing against the in-memory data, and then writing the changes back to the database).
For example, the following Query Syntax:
var records = from r in Records
where r.Property == value
select r
Gets translated first to Lamda Syntax:
Records.Where(r => r.Property == value).Select();
And finally to SQL (via Expression Trees):
SELECT Property, Property2, Property3 FROM Record WHERE Property = #value
...granted, the example doesn't update anything...but the process would be the same for an update query as opposed to a simple select.
Recently we turned a set of complicate C# based scheduling logic into SQL CLR stored procedure (running in SQL Server 2005). We believed that our code is a great SQL CLR candidate because:
The logic involves tons of data from sQL Server.
The logic is complicate and hard to be done using TSQL
There is no threading or sychronization or accessing resources from outside of the sandbox.
The result of our sp is pretty good so far. However, since the output of our logic is in form of several tables of data, we can't just return a single rowset as the result of the sp. Instead, in our code we have a lot of "INSERT INTO ...." statements in foreach loops in order to save each record from C# generic collection into SQL tables. During code review, someone raised concern about whether the inline SQL INSERT approach within the SQL CLR can cause perforamnce problem, and wonder if there's other better way to dump data out (from our C# generic collections).
So, any suggestion?
I ran across this while working on an SQLite project a few months back and found it enlightening. I think it might be what you're looking for.
...
Fastest universal way to insert data
using standard ADO.NET constructs
Now that the slow stuff is out of the
way, lets talk about some hardcore
bulk loading. Aside from SqlBulkCopy
and specialized constructs involving
ISAM or custom bulk insert classes
from other providers, there is simply
no beating the raw power of
ExecuteNonQuery() on a parameterized
INSERT statement. I will demonstrate:
internal static void FastInsertMany(DbConnection cnn)
{
using (DbTransaction dbTrans = cnn.BeginTransaction())
{
using (DbCommand cmd = cnn.CreateCommand())
{
cmd.CommandText = "INSERT INTO TestCase(MyValue) VALUES(?)";
DbParameter Field1 = cmd.CreateParameter();
cmd.Parameters.Add(Field1);
for (int n = 0; n < 100000; n++)
{
Field1.Value = n + 100000;
cmd.ExecuteNonQuery();
}
}
dbTrans.Commit();
}
}
You could return a table with 2 columns (COLLECTION_NAME nvarchar(max), CONTENT xml) filled with as many rows as internal collections you have. CONTENT will be an XML representation of the data in the collection.
Then you can use the XML features of SQL 2005/2008 to parse each collection's XML into tables, and perform your INSERT INTO's or MERGE statements on the whole table.
That should be faster than individual INSERTS inside your C# code.
While searching for linq conditional where clause, I found this article, the way they use is like below:
var logs = from log in context.Logs
select log;
if (filterBySeverity)
logs = logs.Where(p => p.Severity == severity);
if (filterByUser)
logs = logs.Where(p => p.User == user);
but I was wondering is this method efficient? How many queries would linq perform?
Yes, I think this is efficient. No queries will actually be performed by this code, because it doesn't try to read anything from 'logs'. When it does, it should take both conditions into account in the same query (i.e. a WHERE clause that includes both conditions).
BUT, if you are using LINQ and worried about efficiency, you really have to check everything you write by using tools to look at what queries actually get run. You can do this using SQL Server Profiler.
You can use dynamic LINQ (ScottGu's Article)
So you can easily create your where clause in string and then pass it to the Where method:
public string GetWhereClause()
{
string whereClause = "";
....
return whereClause
}
var logs = context.Logs.Where( GetWhereClause() );
Hope this helps ;)