I'm using Spring data JPA like this:
public interface SampleRepository extends CrudRepository<Sample, Integer>{
#Query("SELECT s FROM Sample s WHERE ((:ids) IS NULL OR s.id IN (:ids))")
List<Sample> queryIn(#Param("ids") List<Integer> ids);
}
But when ids is not empty, I got (1,2,3) is null which is incorrect in mysql.
If I write like this:
public interface SampleRepository extends CrudRepository<Sample, Integer>{
#Query("SELECT s FROM Sample s WHERE s.id IN (:ids)")
List<Sample> queryIn(#Param("ids") List<Integer> ids);
}
When ids is empty, I got s.id in (null)
If I must use native query to do this. Any suggestions?
I guess you want to use ISNULL function to see if :ids has a value or not
SELECT s FROM Sample s WHERE s.id IN (:ids)) or ISNULL(:ids) = 1
I want to execute the follwoing query but when I used the object[] as return type it gives the error of "PersistentEntity must not be null!" What should be the return type of this query ?
public interface PropertyRepository extends JpaRepository<Property, Long>,JpaSpecificationExecutor {
#Query(value="select p.city.name, count(p) from Property p join p.city group by p.city.id")
Object[] countPropertyInCities();
}
Your query will return a List<Object[]>.
Try this
#Query(value="select p.city.name as name, count(p) as pCount from Property p join p.city c group by c.id")
List<Object[]> countPropertyInCities();
I suggest you try JPA Projections to only fetch entity properties that your actually need.
I'm trying to optimize some queries, and I have this crazy one. The basic idea is I get a bunch of rooms which has some corresponding meetings. I currently run a query to get all the rooms, then foreach room I need to get the meetings, where I do a query for each room. This opens up for a lot of database connections (i.e. 1000 rooms each having to open a connection to pull the meetings), and I'd like to do it as a batch instead. I am using dapper to map my queries to models and I'm trying to use the list parameters described here
SELECT
mm.id,
mm.organizer_name as Organizer,
mm.subject as Subject,
mm.start_time as StartTime,
mm.end_time as EndTime,
(mm.deleted_at IS NOT NULL) as WasCancelled,
(am.interactive = 0 AND am.cancelled_at IS NOT NULL) as WasNoShow,
c.name as name
FROM master_meeting mm
LEFT JOIN master_meeting__exchange mme ON mme.id=mm.id
LEFT JOIN master_meeting__forwarded_exchange mmfe ON mmfe.id=mm.id
LEFT JOIN meeting_instance__exchange mie ON mie.meeting_id=mm.id
LEFT JOIN meeting_instance__forwarded_exchange mife ON mife.meeting_id=mm.id
LEFT JOIN appointment_meta__exchange ame ON mie.item_id=ame.item_id
LEFT JOIN appointment_meta__exchange ame2 ON mife.item_id=ame2.item_id
LEFT JOIN appointment_meta am ON am.id=ame.id
LEFT JOIN appointment_meta am2 ON am2.id=ame2.id
LEFT JOIN calendar c on mie.calendar_id=c.id
WHERE mie.calendar_id = #Id OR mife.calendar_id=#Id
AND mm.start_time BETWEEN #StartTime AND #EndTime
Without going into details of the crazy long join sequence, I currently have to do this query, a lot. It has been written up initially as:
List<Result> resultSet = new List<Result>();
foreach(int id in idList){
resultSet.AddRange(
_queryHandler.Handle(
new MeetingQuery(id, "FixedStartTime", "FixedEndTime")
)
);
}
Which in turn calls this a bunch of times and runs the query:
_connection.Query<Meeting>(sql,
new {
Id = query.id,
StartTime = query.StartTime,
EndTime = query.EndTime
}
);
This obviously requires quite a few database connections, and I'd like to avoid this by having dapper doing multiple queries, but I get the following error if I try to add the parameters as a list which looks like this:
class Parameters {
int Id;
string StartTime;
string EndTime;
}
List<Parameters> parameters = new List<Parameters>();
foreach(int id in idList)
parameters.Add(new Parameters(id, "SameStartTime", "SameEndTime");
Then I would use the list of parameters as this:
_connection.Query<Meeting>(sql,parameters);
The error I get is:
dapper Additional information: An enumerable sequence of parameters (arrays, lists, etc) is not allowed in this context
Firstly, it's possible to reuse a single connection for multiple queries, so you could retrieve all of your data with multiple Dapper "Query" calls using the same connection.
Something like the following (which isn't the exact same query as you showed since I was testing this on my own computer with a local database; it should be easy enough to see how it could be altered to work with your query, though) -
private static IEnumerable<Record> UnbatchedRetrieval(IEnumerable<Parameters> parameters)
{
var allResults = new List<Record>();
using (var conn = GetConnection())
{
foreach (var parameter in parameters)
{
allResults.AddRange(
conn.Query<Record>(
"SELECT Id, Title FROM Posts WHERE Id = #id",
parameter
)
);
}
}
return allResults;
}
public class Parameters
{
public int Id { get; set; }
}
However, if it really is the number of queries that you want to reduce through batching then there isn't anything in Dapper that makes it very easy to do since each parameter must be uniquely named, which won't be the case if you provide multiple instances of a type as the "parameters" value (since there will be "n" Id values that are all called "Id", for example).
You could do something a bit hacky to produce a single query string that will return results from multiple parameter sets, such as the following -
private static IEnumerable<Record> BatchedRetrieval(IEnumerable<Parameters> parameters)
{
using (var conn = GetConnection)
{
var select = "SELECT Id, Title FROM Posts";
var where = "Id = {0}";
var sqlParameters = new DynamicParameters();
var combinedWheres =
"(" +
string.Join(
") OR (",
parameters.Select((parameter, index) =>
{
sqlParameters.Add("id" + index, parameter.Id);
return string.Format(where, "#id" + index);
})
) +
")";
return conn.Query<Record>(
select + " WHERE " + combinedWheres,
sqlParameters
);
}
}
public class Parameters
{
public int Id { get; set; }
}
.. but this feels a bit dirty. It might be an option to explore, though, if you are absolutely sure that performing those queries one-by-one is a performance bottleneck.
Another thing to consider - when you need the data for 1000 different ids, are the start and end times always the same for each of the 1000 queries? If so, then you could possibly change your query to the following:
private static IEnumerable<Record> EfficientBatchedRetrieval(
IEnumerable<int> ids,
DateTime startTime,
DateTime endTime)
{
using (var conn = GetConnection())
{
return conn.Query<Record>(
#"SELECT
mm.id,
mm.organizer_name as Organizer,
mm.subject as Subject,
mm.start_time as StartTime,
mm.end_time as EndTime,
(mm.deleted_at IS NOT NULL) as WasCancelled,
(am.interactive = 0 AND am.cancelled_at IS NOT NULL) as WasNoShow,
c.name as name
FROM master_meeting mm
LEFT JOIN master_meeting__exchange mme ON mme.id=mm.id
LEFT JOIN master_meeting__forwarded_exchange mmfe ON mmfe.id=mm.id
LEFT JOIN meeting_instance__exchange mie ON mie.meeting_id=mm.id
LEFT JOIN meeting_instance__forwarded_exchange mife ON mife.meeting_id=mm.id
LEFT JOIN appointment_meta__exchange ame ON mie.item_id=ame.item_id
LEFT JOIN appointment_meta__exchange ame2 ON mife.item_id=ame2.item_id
LEFT JOIN appointment_meta am ON am.id=ame.id
LEFT JOIN appointment_meta am2 ON am2.id=ame2.id
LEFT JOIN calendar c on mie.calendar_id=c.id
WHERE mie.calendar_id IN #Ids OR mife.calendar_id IN #Ids
AND mm.start_time BETWEEN #StartTime AND #EndTime",
new { Ids = ids, StartTime = startTime, EndTime = endTime }
);
}
}
There may be a problem with this if you call it with large numbers of ids, though, due to the way that Dapper converts the IN clause - as described in https://stackoverflow.com/a/19938414/3813189 (where someone warns against using it with large sets of values).
If that approach fails then it might be possible to do something similar to the temporary table bulk load suggested here: https://stackoverflow.com/a/9947259/3813189, where you get all of the keys that you want data for into a temporary table and then perform a query that joins on to that table for the keys (and then deletes it again after you have the data).
I have plenty of experience with SQL but am fairly new to LINQ and am struggling to convert the following MySQL query into LINQ.
Can anyone help convert the following to LINQ for use in an ASP.net MVC project with Entity framework?
SELECT
S.Submission_ID,
P.Photo_ID,
C2.Contract_Name,
J.Job_Number,
D.Device_Name,
A.`Display_Name`,
S.Submission_Status,
S.Submission_JobRef,
S.Created,
TRUE
FROM
Submission S
LEFT JOIN Job J ON S.`Job_ID` = J.`Job_ID`
LEFT JOIN Contract C2 ON J.`Contract_ID` = C2.`Contract_ID`
INNER JOIN Submission_Status SS ON S.`Submission_Status` = SS.`ID`
INNER JOIN Device D ON S.`Device_ID` = D.`Device_ID`
INNER JOIN ACTION A ON S.`Action_ID` = A.`Action_ID`
INNER JOIN (
SELECT
MIN(P.Photo_ID) AS Photo_ID,
P.Submission_ID
FROM
Photo P
GROUP BY
P.`Submission_ID`) P ON S.`Submission_ID` = P.Submission_ID
WHERE
S.`Submission_Status` <> 3 AND
(LOCATE(#Criteria, C2.`Contract_Name`) > 0 OR
LOCATE(#Criteria, J.`Job_Number`) > 0 OR
LOCATE(#Criteria, D.`Device_Name`) > 0 OR
LOCATE(#Criteria, A.`Display_Name`) > 0 OR
LOCATE(#Criteria, SS.`Value`) > 0 OR
LOCATE(#Criteria, S.`Submission_JobRef`) > 0)
ORDER BY
S.`Submission_ID` DESC
I have tried to get my head around the multiple joins and subquery but have since got stuck. This is what I have so far...Obviously, it is not working or complete!!
Dim results = From S In db.Submissions
Join P In db.Photos On S.Submission_ID Equals P.Submission_ID
Group Join J In db.Jobs On S.Job_ID Equals J.Job_ID
Into Job = Group
Join J In db.Jobs On S.Job_ID Equals J.Job_ID
Group By P.Submission_ID
Into SubmissionPhotoID = Min(P.Photo_ID)
Select New With {.Submission_ID = Submission_ID,
.Photo_ID = SubmissionPhotoID,
.Contract_Name = If(IsNothing(S.Job), "", S.Job.Contract.Contract_Name),
.Job_Number = If(IsNothing(S.Job), "", S.Job.Job_Number),
.Device_Name = S.Device.Device_Name,
.Action_Name = S.Action.Display_Name,
.Submission_Status = S.Submission_Status1.ID,
.Submission_JobRef = S.Submission_JobRef,
.Created = S.Created,
.CanEdit = bolCanEdit}
Order By S.Submission_ID
Skip param.iDisplayStart
Take param.iDisplayLength
Any help or guidance with the above would be greatly appreciated!
Edit
To aid things, here are the classes from the model defining the entities used in the above query. (I have omitted some field which have no relevance to the question).
Partial Public Class Submission
Public Property Submission_ID As Integer
Public Property Job_ID As Nullable(Of Integer)
Public Property Device_ID As Integer
Public Property Action_ID As Integer
Public Property Submission_Status As Nullable(Of Integer)
Public Property Submission_JobRef As String
Public Property Created As Nullable(Of Date)
Public Overridable Property Action As Action
Public Overridable Property Device As Device
Public Overridable Property Job As Job
Public Overridable Property Photos As ICollection(Of Photo) = New HashSet(Of Photo)
Public Overridable Property Submission_Status1 As Submission_Status
End Class
Partial Public Class Job
Public Property Job_ID As Integer
Public Property Contract_ID As Nullable(Of Integer)
Public Property Job_Number As String
Public Overridable Property Contract As Contract
Public Overridable Property Submissions As ICollection(Of Submission) = New HashSet(Of Submission)
End Class
Partial Public Class Contract
Public Property Contract_ID As Integer
Public Property Contract_Name As String
Public Overridable Property Jobs As ICollection(Of Job) = New HashSet(Of Job)
End Class
Partial Public Class Submission_Status
Public Property ID As Integer
Public Property Value As String
Public Overridable Property Submissions As ICollection(Of Submission) = New HashSet(Of Submission)
End Class
Partial Public Class Device
Public Property Device_ID As Integer
Public Property Device_Name As String
Public Overridable Property Submissions As ICollection(Of Submission) = New HashSet(Of Submission)
End Class
Partial Public Class Action
Public Property Action_ID As Integer
Public Property Display_Name As String
Public Overridable Property Submissions As ICollection(Of Submission) = New HashSet(Of Submission)
End Class
Partial Public Class Photo
Public Property Photo_ID As Integer
Public Property Submission_ID As Integer
Public Overridable Property Submission As Submission
End Class
That's a fairly complex piece of SQL, with a sub-select and mixture of left and inner joins.
Some quick suggestions:
Break it down into a sequence of linq statements, starting with your core objects and adding the related pieces in subsequent steps. If you keep the results as IQueryable, the compiler will put it all together for you and send as one query to the db (i.e. don't ToList() until the last step).
Personally, I do joins using two from's and a where extension method than using the join operator. I makes it easier to know that you're getting a left join or an inner join, for one thing.
For example:
FROM Submission S LEFT JOIN Job J ON S.`Job_ID` = J.`Job_ID`
I would do this as (sorry I'm c# so the syntax may not be quite correct for VB)
Dim results = from s in db.Submissions
from j in db.Jobs.Where(j=> j.Job_Id == s.Job_Id).DefaultIfEmpty()
So, the join criteria is inside the .Where() on Jobs and .DefaultIfEmpty() tells it to left-join (essentially, Job will be a default if the join fails).
FURTHER EDIT:
After experimenting, I got this code to return a result (is it the correct result is another question). Again, sorry for the c# syntax.
[TestMethod]
public void Query()
{
const string conStr = "Data Source=(local);Initial Catalog=ComplexSqlToLinq; Integrated Security=True";
var db = new MyDbContext(conStr);
const string criteria = "Contract1";
var minPhotos = from p in db.Photos
group p by p.SubmissionId
into g
select new {SubmissionId = g.Key, PhotoId = g.Min(p=>p.PhotoId)};
var query = from s in db.Submissions
from j in db.Jobs.Where(j => j.JobId == s.JobId).DefaultIfEmpty()
from c in db.Contracts.Where(c => c.ContractId == j.ContractId).DefaultIfEmpty()
from ss in db.SubmissionStatuses.Where(ss => ss.Id == s.SubmissionStatus)
from d in db.Devices.Where(d => d.DeviceId == s.DeviceId)
from a in db.Actions.Where(a => a.ActionId == s.ActionId)
from p in minPhotos.Where(p => p.SubmissionId == s.SubmissionId)
where s.SubmissionStatus != 3 &&
( c.ContractName.Contains(criteria) ||
j.JobNumber.Contains(criteria) ||
d.DeviceName.Contains(criteria) ||
a.DisplayName.Contains(criteria) ||
ss.Value.Contains(criteria) ||
s.SubmissionJobRef.Contains(criteria))
select new
{
s.SubmissionId,
p.PhotoId,
c.ContractName,
j.JobNumber,
d.DeviceName,
a.DisplayName,
s.SubmissionStatus,
s.SubmissionJobRef,
s.Created,
SomeBool = true
};
var result = query.ToList();
Assert.IsTrue(result.Any());
}
Obviously, you can vary the criteria constant in the test to apply to different items, I chose to match the Contract - I assume that only one of the tables will strike a match.
This query generates the following SQL, looks a bit hokey but is pretty similar in function to your original.
SELECT
[Filter1].[SubmissionId] AS [SubmissionId],
[GroupBy1].[A1] AS [C1],
[Filter1].[ContractName] AS [ContractName],
[Filter1].[JobNumber] AS [JobNumber],
[Filter1].[DeviceName] AS [DeviceName],
[Filter1].[DisplayName] AS [DisplayName],
[Filter1].[SubmissionStatus] AS [SubmissionStatus],
[Filter1].[SubmissionJobRef] AS [SubmissionJobRef],
[Filter1].[Created] AS [Created],
cast(1 as bit) AS [C2]
FROM
(
SELECT
[Extent1].[SubmissionId] AS [SubmissionId],
[Extent1].[SubmissionStatus] AS [SubmissionStatus],
[Extent1].[SubmissionJobRef] AS [SubmissionJobRef],
[Extent1].[Created] AS [Created],
[Extent2].[JobNumber] AS [JobNumber],
[Extent3].[ContractName] AS [ContractName],
[Extent4].[Value] AS [Value],
[Extent5].[DeviceName] AS [DeviceName],
[Extent6].[DisplayName] AS [DisplayName]
FROM
[dbo].[Submissions] AS [Extent1]
LEFT OUTER JOIN [dbo].[Jobs] AS [Extent2] ON [Extent2].[JobId] = [Extent1].[JobId]
LEFT OUTER JOIN [dbo].[Contracts] AS [Extent3] ON [Extent3].[ContractId] = [Extent2].[ContractId]
INNER JOIN [dbo].[SubmissionStatus] AS [Extent4] ON [Extent4].[Id] = [Extent1].[SubmissionStatus]
INNER JOIN [dbo].[Devices] AS [Extent5] ON [Extent5].[DeviceId] = [Extent1].[DeviceId]
INNER JOIN [dbo].[Actions] AS [Extent6] ON [Extent6].[ActionId] = [Extent1].[ActionId]
WHERE
3 <> [Extent1].[SubmissionStatus]
) AS [Filter1]
INNER JOIN (
SELECT
[Extent7].[SubmissionId] AS [K1],
MIN([Extent7].[PhotoId]) AS [A1]
FROM
[dbo].[Photos] AS [Extent7]
GROUP BY
[Extent7].[SubmissionId] ) AS [GroupBy1]
ON [GroupBy1].[K1] = [Filter1].[SubmissionId]
WHERE
(
[Filter1].[ContractName] LIKE #p__linq__0 ESCAPE N'~') OR
([Filter1].[JobNumber] LIKE #p__linq__1 ESCAPE N'~') OR
([Filter1].[DeviceName] LIKE #p__linq__2 ESCAPE N'~') OR
([Filter1].[DisplayName] LIKE #p__linq__3 ESCAPE N'~') OR
([Filter1].[Value] LIKE #p__linq__4 ESCAPE N'~') OR
([Filter1].[SubmissionJobRef] LIKE #p__linq__5 ESCAPE N'~')
)
To respond to Dave Johnson's comment in a word - scalability.
Recently I was trying to improve performance of an application and my first thought was to add some SQL similar in complexity to John Henry's sample - multiple joins and filters. After all, it performed like a rocket on my dev machine.
The architect flatly prohibited the use of complex SQL on the database server, on the basis that several large applications with 100's of users were hooked in to it. Much as I like building snappy SQL that rocks, I had to agree. Shifting the logic to to machine that consumes it is good architecture.
So for those of us proficient in declarative SQL, learning translation to linq skills is important.
Of course, the solution I gave earlier doesn't achieve this as the same SQL is sent to the server. But having a linq equivalent is a start that can be be further optimised.
After an awful lot of searching and reading various articles I have given up trying to write this query in LINQ query syntax and gone with method syntax instead.
A big thank you to Ackroydd for your suggestions and support with converting complex SQL to LINQ. When you know you can accomplish something in SQL in a matter of minutes but need to use LINQ for scalability and to keep with existing code, it can get rather frustrating!
Here is what I ended up with as I'm sure it will be useful to someone else:
Dim query As IQueryable(Of Submission)
' Initialise the new query
query = db.Submissions.Include(Function(s) s.Action) _
.Include(Function(s) s.Photos) _
.Include(Function(s) s.Device) _
.Include(Function(s) s.Job) _
.Include(Function(s) s.Submission_Status1) _
.Include(Function(s) s.Job.Contract) _
.Include(Function(s) s.Comments) _
.AsNoTracking
' Apply initial filters
query = query.Where(Function(S) Not S.Submission_Status1.ID.Equals(3))
' Apply search criteria if passed
If Not String.IsNullOrEmpty(param.sSearch) Then
query = query.Where(Function(S) S.Job.Contract.Contract_Name.Contains(param.sSearch) OrElse
S.Job.Job_Number.Contains(param.sSearch) OrElse
S.Device.Device_Name.Contains(param.sSearch) OrElse
S.Action.Display_Name.Contains(param.sSearch) OrElse
S.Submission_Status1.Value.Contains(param.sSearch) OrElse
S.Submission_JobRef.Contains(param.sSearch))
End If
' Order the results
query = query.OrderByDescending(Function(S) S.Submission_ID)
' Paginate the results
query = query.Skip(param.iDisplayStart).Take(param.iDisplayLength)
' Return only the required columns
Dim resultData = query.AsEnumerable.Select(Function(S) New AjaxSubmissionOverview With { _
.Submission_ID = S.Submission_ID,
.Photo_ID = S.Photos.First.Photo_ID,
.Contract_Name = If(IsNothing(S.Job), "", S.Job.Contract.Contract_Name),
.Job_Number = If(IsNothing(S.Job), "", S.Job.Job_Number),
.Device_Name = S.Device.Device_Name,
.Action_Name = S.Action.Display_Name,
.Submission_Status = S.Submission_Status,
.Submission_JobRef = S.Submission_JobRef,
.Latest_Comment = If(S.Comments.Count = 0, "", HtmlHelpers.Truncate(S.Comments.Last.Comment1, 100)),
.Created = S.Created,
.CanEdit = bolCanEdit})
How do i write a Linq to SQL query that translates into the following:
SELECT CAST(DATETEXT AS datetime) FROM mytable
var dates = from row in mytable
select DateTime.Parse(row.DATETEXT);
There are method overloads for DateTime.Parse that allow you to specify a format.
actually, you won't be needing that. just parse it as datatime when you will use the field value. here is an exaple;
var query = from c in mytable
select c;
then when you will use it;
DateTime _value = (DateTime)query.SingleOrDefault().DATETEXT
but if you wanna use it so much. here is an example;
NorthwindEntities _e = new NorthwindEntities();
public void poo() {
var query = from e in _e.Products
select DateTime.Parse(e.DateText);
}