Is there a way to do a bulk update on a collection with LINQ? Currently if I have a List<myObject> and I want to update column1 to equal TEST for every row in the List I would setup a foreach loop and then for each individual object I would set the value and then save it. This works fine but I was just wondering if there was some LINQ method out there where I could do something like myOject.BulkUpdate(columnName, value)?
Your requirement here is entirely possible using Linq expressions and Terry Aney's excellent library on this topic.
Batch Updates and Deletes with LINQ to SQL
An update in the terms of the example you gave would be as follows:
using BTR.Core.Linq;
...
Context.myObjects.UpdateBatch
(
Context.myObjects.Where(x => x.columnName != value),
x => new myObject { columnName = value}
);
Edit (2017-01-20): It's worth nothing this is now available in the form of a NuGet package # https://www.nuget.org/packages/LinqPost/.
Install-Package LinqPost
Sounds like you're using LINQ To SQL, and you've got the basics laid out already.
LINQ To SQL is about abstracting tables into classes, and doesn't really provide the 'silver bullet' or one-liner you are looking for.
The only way to do that is to achieve your one-liner would be to make a stored proc to take that column name and new value, and implement that logic yourself.
db.MassUpdateTableColumn("Customer", "Name", "TEST");
....
CREATE PROC MassUpdateTableColumn
#TableName varchar(100), #ColumnName varchar(100), #NewVal varchar(100)
AS
/*your dynamic SQL to update a table column with a new val. */
Otherwise, it's as you describe:
List<Customer> myCusts = db.Customers.ToList();
foreach(Customer c in myCusts)
{
c.Name = "TEST";
}
db.SubmitChanges();
LINQ to SQL (or EF for that matter), is all about bringing objects into memory, manipulating them, and then updating them with separate database requests for each row.
In cases where you don't need to hydrate the entire object on the client, it is much better to use server side operations (stored procs, TSQL) instead of LINQ. You can use the LINQ providers to issue TSQL against the database. For example, with LINQ to SQL you can use context.ExecuteCommand("Update table set field=value where condition"), just watch out for SQL Injection.
EF Core 7.0 introduces Bulk Update and Bulk Delete.
For example, consider the following LINQ query terminated with a call to ExecuteUpdateAsync:
var priorToDateTime = new DateTime(priorToYear, 1, 1);
await context.Tags
.Where(t => t.Posts.All(e => e.PublishedOn < priorToDateTime))
.ExecuteUpdateAsync(s => s.SetProperty(t => t.Text, t => t.Text + " (old)"));
This generates SQL to immediately update the “Text” column of all tags for posts published before the given year:
UPDATE [t]
SET [t].[Text] = [t].[Text] + N' (old)'
FROM [Tags] AS [t]
WHERE NOT EXISTS (
SELECT 1
FROM [PostTag] AS [p]
INNER JOIN [Posts] AS [p0] ON [p].[PostsId] = [p0].[Id]
WHERE [t].[Id] = [p].[TagsId] AND [p0].[PublishedOn] < #__priorToDateTime_1)
Related
I'd like to update a table with Django - something like this in raw SQL:
update tbl_name set name = 'foo' where name = 'bar'
My first result is something like this - but that's nasty, isn't it?
list = ModelClass.objects.filter(name = 'bar')
for obj in list:
obj.name = 'foo'
obj.save()
Is there a more elegant way?
Update:
Django 2.2 version now has a bulk_update.
Old answer:
Refer to the following django documentation section
Updating multiple objects at once
In short you should be able to use:
ModelClass.objects.filter(name='bar').update(name="foo")
You can also use F objects to do things like incrementing rows:
from django.db.models import F
Entry.objects.all().update(n_pingbacks=F('n_pingbacks') + 1)
See the documentation.
However, note that:
This won't use ModelClass.save method (so if you have some logic inside it won't be triggered).
No django signals will be emitted.
You can't perform an .update() on a sliced QuerySet, it must be on an original QuerySet so you'll need to lean on the .filter() and .exclude() methods.
Consider using django-bulk-update found here on GitHub.
Install: pip install django-bulk-update
Implement: (code taken directly from projects ReadMe file)
from bulk_update.helper import bulk_update
random_names = ['Walter', 'The Dude', 'Donny', 'Jesus']
people = Person.objects.all()
for person in people:
r = random.randrange(4)
person.name = random_names[r]
bulk_update(people) # updates all columns using the default db
Update: As Marc points out in the comments this is not suitable for updating thousands of rows at once. Though it is suitable for smaller batches 10's to 100's. The size of the batch that is right for you depends on your CPU and query complexity. This tool is more like a wheel barrow than a dump truck.
Django 2.2 version now has a bulk_update method (release notes).
https://docs.djangoproject.com/en/stable/ref/models/querysets/#bulk-update
Example:
# get a pk: record dictionary of existing records
updates = YourModel.objects.filter(...).in_bulk()
....
# do something with the updates dict
....
if hasattr(YourModel.objects, 'bulk_update') and updates:
# Use the new method
YourModel.objects.bulk_update(updates.values(), [list the fields to update], batch_size=100)
else:
# The old & slow way
with transaction.atomic():
for obj in updates.values():
obj.save(update_fields=[list the fields to update])
If you want to set the same value on a collection of rows, you can use the update() method combined with any query term to update all rows in one query:
some_list = ModelClass.objects.filter(some condition).values('id')
ModelClass.objects.filter(pk__in=some_list).update(foo=bar)
If you want to update a collection of rows with different values depending on some condition, you can in best case batch the updates according to values. Let's say you have 1000 rows where you want to set a column to one of X values, then you could prepare the batches beforehand and then only run X update-queries (each essentially having the form of the first example above) + the initial SELECT-query.
If every row requires a unique value there is no way to avoid one query per update. Perhaps look into other architectures like CQRS/Event sourcing if you need performance in this latter case.
Here is a useful content which i found in internet regarding the above question
https://www.sankalpjonna.com/learn-django/running-a-bulk-update-with-django
The inefficient way
model_qs= ModelClass.objects.filter(name = 'bar')
for obj in model_qs:
obj.name = 'foo'
obj.save()
The efficient way
ModelClass.objects.filter(name = 'bar').update(name="foo") # for single value 'foo' or add loop
Using bulk_update
update_list = []
model_qs= ModelClass.objects.filter(name = 'bar')
for model_obj in model_qs:
model_obj.name = "foo" # Or what ever the value is for simplicty im providing foo only
update_list.append(model_obj)
ModelClass.objects.bulk_update(update_list,['name'])
Using an atomic transaction
from django.db import transaction
with transaction.atomic():
model_qs = ModelClass.objects.filter(name = 'bar')
for obj in model_qs:
ModelClass.objects.filter(name = 'bar').update(name="foo")
Any Up Votes ? Thanks in advance : Thank you for keep an attention ;)
To update with same value we can simply use this
ModelClass.objects.filter(name = 'bar').update(name='foo')
To update with different values
ob_list = ModelClass.objects.filter(name = 'bar')
obj_to_be_update = []
for obj in obj_list:
obj.name = "Dear "+obj.name
obj_to_be_update.append(obj)
ModelClass.objects.bulk_update(obj_to_be_update, ['name'], batch_size=1000)
It won't trigger save signal every time instead we keep all the objects to be updated on the list and trigger update signal at once.
IT returns number of objects are updated in table.
update_counts = ModelClass.objects.filter(name='bar').update(name="foo")
You can refer this link to get more information on bulk update and create.
Bulk update and Create
I have this form in C# with a listbox where I selected 4 items. Now I want to make single stored procedure using which I can find data from single table for all this selected item with single parameter.
As I am a beginner when it comes to SQL Server, I completely don't know this type of procedure
Thanks, but this is not my question's answer
I want a Single Stored Procedure for all Items which are selected in ListBox
Create Procedure procedureName
(
#ItemName varchar(50),
)
AS
BEGIN
(
Select * from item_master where item_name = #ItemName
)
END
by this Query i can find data for one ItemName, but i want for all selected Items in Listbox, even I don't know the C# code also,
so plz help me....
This is a very simple example that does what you want. You would not want to use hard-coded connection strings, especially in-line, and you would want error-handling, but I am going for as much clarity as possible. You would also probably want to make the column length greater than 50 characters, but I made it match your column definition.
Also, I would recommend a generic approach, passing keys (column names) and values, so as to be able to use it for any sort of criteria, but you asked that I keep it to exactly what you require, so I trimmed it down to the essential.
This example returns all the Employees with FirstName matching any in the list passed to the stored procedure (as a user-defined table type).
First, create a user-defined table type (to hold the values you want to pass to the stored procedure) in your SQL Server database as follows:
CREATE TYPE [dbo].[FilterValues] AS TABLE(
[Value] [varchar](50) NOT NULL,
PRIMARY KEY CLUSTERED
(
[Value] ASC
)
)
The stored procedure to return the Employees looks as follows (note that it has the user-defined table type as the type of the single parameter passed in):
CREATE PROCEDURE [dbo].[GetEmployees] (
#FirstNameFilterValues dbo.FilterValues READONLY
)
AS
BEGIN
SELECT * FROM Employees
INNER JOIN #FirstNameFilterValues fv ON fv.Value = Employees.FirstName;
END
That's the SQL Server side done. To call it from C#, you can create a DataTable with a single column matching the column name and populate it with the values you want. In this simple example, I populate it with two names, but it could be as many as you want.
var filterValuesDataTable = new DataTable();
filterValuesDataTable.Columns.Add(new DataColumn("Value", typeof(string)) { AllowDBNull = false });
filterValuesDataTable.Rows.Add("Frodo");
filterValuesDataTable.Rows.Add("Sam");
using (var connection = new SqlConnection("server=.;Initial Catalog=Test;Integrated Security=True;"))
{
connection.Open();
using (var command = connection.CreateCommand())
{
command.CommandText = "GetEmployees";
command.CommandType = CommandType.StoredProcedure;
command.Parameters.AddWithValue("#FirstNameFilterValues", filterValuesDataTable);
using (var reader = command.ExecuteReader())
{
while (reader.Read())
{
Console.WriteLine("{0} {1}", reader["FirstName"], reader["LastName"]);
}
reader.Close();
}
}
connection.Close();
}
I'm trying to select an object using values of another object in LINQ SQL,
I currently have this,
var result1 = (from s in pdc.ScanLogs
from ec in pdc.ExhibitsContacts
where s.ExhibitID == ec.ExhibitID
select ec.Contact);
I want to assign a value of ec.Contact.Note = ec.Comment;
Is there to a way to do this in LINQ SQL without writing multiple queries?
I read this blog article: http://blog.robvolk.com/2009/05/linq-select-object-but-change-some.html but it doesn't seem to work with LINQ SQL.
Basically you can't do this. LINQ is meant to be a query language, and what you want to do is mutate existing entities with your query. This means your query would have side effects and this is not something that is supported by LINQ to SQL.
While this won't work in a single query while returning LINQ to SQL entities, what will work is when you return simple DTO structues. For instance:
var result1 =
from s in pdc.ScanLogs
from ec in s.ExhibitsContacts
select new ContactDto
{
Id = ec.Contact.Id,
Note = ec.Comment,
SomeOtherFields = ec.Contact.SomeOtherFields
};
As a side note: also look at how I removed the where s.ExhibitID == ec.ExhibitID join from the query, by just using the ExhibitsContacts property of the ScanLog entity (which will be generated by LINQ to SQL for you when your database schema has the proper foreign keys defined).
Update:
When you need to return those DTO from several methods, you might consider centralizing the transformation from a collection of entities to a collection of DTO objects. What I tend to do is place this method on the DTO (which makes it easy to find). The code might look like this:
public class ContactDto
{
// Many public properties here
public static IQueryable<ContactDto> ToDto(
IQueryable<Contact> contacts)
{
return
from contact in contacts
select new ContactDto
{
Id = contact.Id,
Note = contact.ExhibitsContact.Comment,
ManyOtherFields = contact.ManyOtherFields
};
}
}
The trick with this static transformation method is that it takes an IQueryable and returns an IQueryable. This allows to to simply specify the transformation and let LINQ to SQL (or any other LINQ enabled O/RM) to efficiently execute that LINQ expression later on. The original code would now look like this:
IQueryable<Contact> contacts =
from s in pdc.ScanLogs
from ec in s.ExhibitsContacts
select ec.Contact;
IQuerable<ContactDto> result1 = ContactDto.ToDto(contacts);
the problem is that LINQ to SQL does not know how to interpret your extension method. The only way, other than using stored procedures from LINQ to SQL (which kind of defeats the ponit), is to get the object, update and then commit changes.
SQL Server 2008 Ent
ASP.NET MVC 2.0
Linq-to-SQL
I am building a gaming site, that tracks when a particular player (toon) had downed a particular monster (boss). Table looks something like:
int ToonId
int BossId
datetime LastKillTime
I use a 3d party service that gives me back latest information (toon,boss,time).
Now I want to update my database with that new information.
Brute force approach is to do line-by-line upsert. But It looks ugly (code-wise), and probably slow too.
I think better solution would be to insert new data (using temp table?) and then run MERGE statement.
Is it good idea? I know temp tables are "better-to-avoid". Should I create a permanent "temp" table just for this operation?
Or should I just read entire current set (100 rows at most), do merge and put it back from within application?
Any pointers/suggestions are always appreciated.
An ORM is the wrong tool for performing batch operations, and Linq-to-SQL is no exception. In this case I think you have picked the right solution: Store all entries in a temporary table quickly, then do the UPSERT using merge.
The fastest way to store the data to the temporary table is to use SqlBulkCopy to store all data to a table of your choice.
If you're using Linq-to-SQL, upserts aren't that ugly..
foreach (var line in linesFromService) {
var kill = db.Kills.FirstOrDefault(t=>t.ToonId==line.ToonId && t.BossId==line.BossId);
if (kill == null) {
kill = new Kills() { ToonId = line.ToonId, BossId = line.BossId };
db.Kills.InsertOnSubmit(kill);
}
kill.LastKillTime = line.LastKillTime;
}
db.SubmitChanges();
Not a work of art, but nicer than in SQL. Also, with only 100 rows, I wouldn't be too concerned about performance.
Looks like a straight-forward insert.
private ToonModel _db = new ToonModel();
Toon t = new Toon();
t.ToonId = 1;
t.BossId = 2;
t.LastKillTime = DateTime.Now();
_db.Toons.InsertOnSubmit(t);
_db.SubmitChanges();
To update without querying the records first, you can do the following. It will still hit the db once to check if record exists but will not pull the record:
var blob = new Blob { Id = "some id", Value = "some value" }; // Id is primary key (PK)
if (dbContext.Blobs.Contains(blob)) // if blob exists by PK then update
{
// This will update all columns that are not set in 'original' object. For
// this to work, Blob has to have UpdateCheck=Never for all properties except
// for primary keys. This will update the record without querying it first.
dbContext.Blobs.Attach(blob, original: new Blob { Id = blob.Id });
}
else // insert
{
dbContext.Blobs.InsertOnSubmit(blob);
}
dbContext.Blobs.SubmitChanges();
See here for an extension method for this.
I am trying to get the records from the 'many' table of a one-to-many relationship and add them as a list to the relevant record from the 'one' table.
I am also trying to do this in a single database request.
Code derived from Linq to Sql - Populate JOIN result into a List almost achieves the intended result, but makes one database request per entry in the 'one' table which is unacceptable. That failing code is here:
var res = from variable in _dc.GetTable<VARIABLE>()
select new { x = variable, y = variable.VARIABLE_VALUEs };
However if I do a similar query but loop through all the results, then only a single database request is made. This code achieves all goals:
var res = from variable in _dc.GetTable<VARIABLE>()
select variable;
List<GDO.Variable> output = new List<GDO.Variable>();
foreach (var v2 in res)
{
List<GDO.VariableValue> values = new List<GDO.VariableValue>();
foreach (var vv in v2.VARIABLE_VALUEs)
{
values.Add(VariableValue.EntityToGDO(vv));
}
output.Add(EntityToGDO(v2));
output[output.Count - 1].VariableValues = values;
}
However the latter code is ugly as hell, and it really feels like something that should be do-able in a single linq query.
So, how can this be done in a single linq query that makes only a single database query?
In both cases the table is set to preload using the following code:
_dc = _db.CreateLinqDataContext();
var loadOptions = new DataLoadOptions();
loadOptions.LoadWith<VARIABLE>(v => v.VARIABLE_VALUEs);
_dc.LoadOptions = loadOptions;
I am using .NET 3.5, and the database back-end was generated using SqlMetal.
This link may help
http://msdn.microsoft.com/en-us/vcsharp/aa336746.aspx
Look under join operators. You'll probably have to change from using extension syntax other syntax too. Like this,
var = from obj in dc.Table
from obj2 in dc.Table2
where condition
select