I've got a task to fetch a set of entities for a specific period of time.
I'm using code first approach and MySql server.
With the code
IQueryable<Entity> query = dbContext.Set<Entity>().AsNoTracking();
Entity[] entities = (from e in query
let last = (from ee in query orderby ee.Id descending select ee).FirstOrDefault()
where e.Timestamp >= DbFunctions.AddHours(last.Timestamp, -hours)
select e).ToArray();
And the following SQL generated by EF 6.1.3
SELECT
`Extent1`.`Id`,
`Extent1`.`Timestamp`
FROM `Entities` AS `Extent1` LEFT OUTER JOIN (SELECT
`Extent2`.`Id`,
`Extent2`.`Timestamp`
FROM `Entities` AS `Extent2`
ORDER BY
`Extent2`.`Id` DESC LIMIT 1) AS `Limit1` ON 1 = 1
WHERE `Extent1`.`Timestamp` >= (AddHours(`Limit1`.`Timestamp`, -(#p__linq__0)))
I face an error
Failed in 6 ms with error: FUNCTION mydb.AddHours does not exist
And indeed I haven't got that function at DB side.
Does anybody know how those DbFunctions work?
UPD
Similar questions
EntityFramework 6.1.3 and MySQL DbFunctions.TruncateTime does not exist?
how to use canonical functions in Entity Framework and Mysql
A workaround for now - just added an implementation of the function in DbInitializer
CREATE FUNCTION AddHours(timeValue datetime, addValue int) RETURNS datetime
RETURN DATE_ADD(timeValue, INTERVAL addValue HOUR);
You are trying to call a stored function of your database, but that does not exist. The solution is to properly create the AddHours db function. More information can be found here.
Related
I have been plugging away at this and I am really at a loss for what I'm messing up. I am using My SQL workbench and sending queries to a mariadb. The original function I am working with, returns records just fine, however they pull in all of them (of which there are 1000+). Ideally the function would only return new records instead of all of them (hence no delete). I am trying to use WHERE NOT EXISTS however SQL is returning error code 1064. Sample code below:
SELECT WHERE NOT EXISTS
usr1.email
, case when ccm.course = 7 then 'CourseComplete'
when ccm.course = 10 then 'Course1' when ccm.course = 4 then 'Course1CourseComplete' else 'other' end as coursecompleted
, course
, date_add(from_unixtime(timecompleted), INTERVAL 1 HOUR) as CompletionDate
FROM moodledb.m_course_completions ccm
inner join m_user usr1
on ccm.userid = usr1.id
where timecompleted is not null and ccm.course in (4,7,10) order by 4 desc
In addition I've tried alternate placements for the where not exists function, but they all return the same error1064. Would a different operator work better? Can goku defeat this new foe? Any suggestions will be greatly appreciated
I have below code to get the customers whose term is ending today.
$customers = CompanyModel::where(("term_ending") ,"=", "CURDATE()")->get();
But when I run this it returns null.
Same query I run in Mysql it gives me results.
select * from `companies` where `term_ending` = CURDATE()
What is wrong I am doing?
I solved it using Carbon.
$customers = CompanyModel::where(("term_ending") ,"=", Carbon::today())->get();
Thanks :)
I am tying to run an update query with a subquery against a MySQL database using ruby. I am using ruby 1.9.3 and rails 4.1.
The query I am trying to create is as below:
UPDATE `items`
SET
`items`.`status_id` = 12
WHERE
`items`.`id` IN (SELECT DISTINCT
`items`.`id`
FROM
`items`
LEFT OUTER JOIN
`statuses` ON `items`.`status_id` = `statuses`.`id`
LEFT OUTER JOIN
`resources` ON `items`.`resource_id` = `resources`.`id`
WHERE
`statuses`.`title` LIKE 'On Loan'
AND `items`.`duedate` < '2015-04-24'
AND `items`.`return_date` IS NULL
ORDER BY `items`.`duedate`)
I can produce this query in ruby using AREL with the code shown below:
# Declare Arel objects
i = Item.arel_table
s = Status.arel_table
r = Resource.arel_table
# This is the AREL query that returns the data
overdues = i.project(i[:id]).
join(s, Arel::Nodes::OuterJoin).on(i[:status_id].eq(s[:id])).
join(r, Arel::Nodes::OuterJoin).on(i[:resource_id].eq(r[:id])).
where(s[:title].matches("On Loan").
and(i[:duedate].lt(DateTime.now.to_date)).
and(i[:return_date].eq(nil))
).
order(i[:duedate])
# Note: You can't chain distinct, otherwise "overdues" becomes a string with the value "DISTINCT".
overdues.distinct
# This creates the update...
u = Arel::UpdateManager.new i.engine
u.table(i)
u.set([[i[:status_id], 10]]).where(i[:id].in(overdues))
This does not work and returns an error message:
ActiveRecord::StatementInvalid: Mysql2::Error: You can't specify target table 'items' for update in FROM clause:
I tried using AR "update_all" but it produces the same SQL and hence the same error.
Item.where(i[:id].in(overdues)).update_all(:status_id => (Status.find_by(:title => "Overdue").id))
Having done some research I have found that you cannot run a update with a subquery that references the table you want to update in MySQL. I have seen a number of posts on this site and the wider internet that detail work arounds.
One suggestion says that the update should use a join instead of a sub query. Having looked at the code behind the update manager it has no "join" so I can't do that.
Another says run this in two parts but I can't see how to because AREL and AciveRecord both chain actions.
The only way I can see of doing this is by aliasing the table and adding an additional select (see below). This isn't great but it would be useful to see if it is possible to do.
UPDATE `items`
SET `status_id` = 10
WHERE `items`.`id` IN (
SELECT x.id
FROM
(SELECT DISTINCT `items`.`id`
FROM `items`
LEFT OUTER JOIN `statuses` ON `items`.`status_id` = `statuses`.`id`
LEFT OUTER JOIN `resources` ON `items`.`resource_id` = `resources`.`id`
WHERE `statuses`.`title` LIKE 'On Loan'
AND `items`.`duedate` < '2015-04-24'
AND `items`.`return_date` IS NULL
ORDER BY `items`.`duedate`) x
);
If I can't get this to work I could adopt two other approaches:
1) I could just hard-code the SQL but I want to use ActiveRecord and reference the models to keep it database agnostic.
2) The other way is to return an instance of all the records and loop through them doing individual updates. This will have a performance issue but I can accept this because its a background job that won't be updating more than a handful of records each day.
Update
I have the AREL query below that produces the subquery in the format I need.
x = Arel::Table.new('x')
overdues = Item.select(x[:id]).from(
Item.select(Item.arel_table[:id]).where(
Status.arel_table[:title].matches("On Loan").and(
Item.arel_table[:duedate].lt(DateTime.now.to_date).and(
Item.arel_table[:return_date].eq(nil))
)
).joins(
Item.arel_table.join(Status.arel_table, Arel::Nodes::OuterJoin).on(
Item.arel_table[:status_id].eq(Status.arel_table[:id])
).join_sources
).joins(
Item.arel_table.join(Resource.arel_table, Arel::Nodes::OuterJoin).on(
Item.arel_table[:resource_id].eq(Resource.arel_table[:id])
).join_sources
).order(Item.arel_table[:duedate]).uniq.as('x')
)
Sadly it returns an error when I use it in my update statement.
TypeError: Cannot visit Item::ActiveRecord_Relation
Having revisited this question I am at the conclusion that it's not possible to do this because of a limitation with MySQL:
ActiveRecord::StatementInvalid: Mysql2::Error: You can't specify target table 'items' for update in FROM clause:
It should be possible to do with other databases (although I haven't tested that).
I could create a temporary table, which is the copy of the original table, reference that and then drop the temporary table like this post suggests:
http://richtextblog.blogspot.co.uk/2007/09/mysql-temporary-tables-and-rails.html. That seems a lot of overhead to do a simple subquery.
What I am going to do is find all the ID's and loop through them and update the records that way (using a simple find and update). This has an overhead but it should only be updating a handful of records each run (no more than 100). The update will be running as a scheduled job outside user working hours so it won't impact performance.
I still find it bizarre that in all other flavours of SQL I have never encountered this problem before. Still you live and learn.
UPDATE:
Since updating my version of MySQL the select statement now works. I had to take out the order by for it to work.
ORDER BY `items`.`duedate`
I am now using version: 5.7.19.
I have quite a complex query to essentially select the cheapest delivery service price per delivery service.
In order to get unique records per delivery service, I utilise the DISTINCT function in SQL. This query provides correct results:
DeliveryServicePrice.active.select('DISTINCT ON (delivery_service_id) *').order('delivery_service_id, price ASC')
(only a part of the query)
However, this query only seems to work with PostgreSQL (which I think is strange considering PostgreSQL is a lot more strict with SQL standards); it does not work with MySQL and SQLite. I receive the following error:
Mysql2::Error: You have an error in your SQL syntax; check the manual
that corresponds to your MySQL server version for the right syntax to
use near 'ON (delivery_service_id) * FROM `delivery_service_prices`
WHERE `delivery_servi' at line 1: SELECT DISTINCT ON
(delivery_service_id) * FROM `delivery_service_prices` WHERE
`delivery_service_prices`.`active` = 1 AND (2808.0 >= min_weight AND
2808.0 <= max_weight AND 104.0 >= min_length AND 104.0 <= max_length AND 104.0 >= min_thickness AND 104.0 <= max_thickness) ORDER BY delivery_service_id, price ASC
The application I'm building is open source, so it's required to support all 3 database types.
How do I create DISTINCT ON queries for MySQL and SQLite in the Rails framework syntax?
I'm using Rails 4.1.
Resources
My previous problem for reference:
How to select unique records based on foreign key column in Rails?
File and line number for where the query is being used.
Finished answer
DeliveryServicePrice.select('delivery_service_prices.id').active.joins('LEFT OUTER JOIN delivery_service_prices t2 ON (delivery_service_prices.delivery_service_id = t2.delivery_service_id AND delivery_service_prices.price > t2.price)').where('t2.delivery_service_id IS NULL')
DISTINCT ON is a Postgres specific extension to the standard SQL DISTINCT. Neither of them is a "function", both are SQL key words - even though the parentheses required after DISTINCT ON make it look like a function.
There are a couple of techniques to rewrite this with standard-SQL, all of them more verbose, though. Since MySQL does not support window-functions row_number() is out.
Details and more possible query techniques:
Select first row in each GROUP BY group?
Fetch the row which has the Max value for a column
Rewritten with NOT EXISTS:
SELECT *
FROM delivery_service_prices d1
WHERE active = 1
AND 2808.0 BETWEEN min_weight AND max_weight
AND 104.0 BETWEEN min_length AND max_length
AND 104.0 BETWEEN min_thickness AND max_thickness
AND NOT EXISTS (
SELECT 1
FROM delivery_service_prices d2
WHERE active = 1
AND 2808.0 BETWEEN min_weight AND max_weight
AND 104.0 BETWEEN min_length AND max_length
AND 104.0 BETWEEN min_thickness AND max_thickness
AND d2.delivery_service_id = d1.delivery_service_id
AND d2.price < d1.price
AND d2.<some_unique_id> < d1.<some_unique_id> -- tiebreaker!
)
ORDER BY delivery_service_id
If there can be multiple rows with the same price for the same delivery_service_id, you need to add some unique tie-breaker to avoid multiple results per delivery_service_id. At least if you want a perfectly equivalent query. My example would select the row with the smallest <some_unique_id> from each set of dupes.
Unlike with DISTINCT ON, ORDER BY is optional here.
DeliveryServicePrice.active.select(:delivery_service_id).distinct.order('delivery_service_id, price ASC')
I have the following query in HQL:
update ProjectFile pf1
set pf1.validUntil.id =123
where pf1 = (
select pf from ProjectVersion pv, ProjectFile as pf
where pf.validFrom.sequence <= pv.sequence
and pf.validUntil.sequence >= pv.sequence
and pf.state <> 12
and pf.projectVersion.project.id = 1
and pv.project.id = 1
and pv.id = 12
and pf.id not in (2,3,4)
)
Hibernate parses the query correctly and generates SQL, but the database (MySQL) fails with error:
You can't specify target table 'ProjectFile' for update in FROM clause
The problem seems to be that the table to be updated is queried in the same context. Is there any way to rewrite the HQL query to produce SQL that can be executed in MySQL correctly? The other approach would be to create an intermediate table, which is what exactly I am trying to avoid.
I bumped into the same problem and posted a question here: MySQL/SQL: Update with correlated subquery from the updated table itself.
To solve your problem, you need to join at the UPDATE level, please take a look at the answer to my question.