I run a query, where I count several sums on the different fields of same database entity. My problem arises when I am running tests on the query and at the same time changing the query from Mysql to the native language used on tests in IntelliJ Idea tool (I don't know what it uses).
Problem is this: in the new environment all sum statements inside the single query return the value equal to the one that is got of the first sum statement.
I am using JPA and NativeQuery.
More information:
I have a code like this
List < Object [ ] > row = List < Object [ ] > em.createNativeQuery("select sum (e.field), sum (e.otherField) from entity e where somevalue = something").getResultList();
and then
return new MyResult ( ( Double ) row.get ( 0 ) [ 0 ] , ( Double )row.get ( 0 ) [ 1 ] );
Everything is inside a doInJPA function.
Real issue was identified to be so that my Idea used for testing HSQL and that somehow does not support sum in the same way that Mysql does.
Related
Following are the HDD column from the Computer model (I know it's not a good format to store data but it's already stored like that)
HDD
4x2TBSATA2
2x2TBSATA2
8x2TBSATA2
4x1TBSATA2
2x120GBSSD
4x480GBSSD
I want to fetch the range out of HDD column where storage is in a specific range, for example, fetch storage between 120GB to 1TB should output
4x1TBSATA2
2x120GBSSD
4x480GBSSD
I was wondering if it's possible to combine like and between in the same statement?
I tried the following which doesn't work.
select * from `server_details` where `HDD` between '%120GB%' and '%10TB%'
select * from `server_details` where `HDD` between "Like '%120GB%'" and "LIKE '%10TB%'"
If you need to do just in SQL, extract the size part, convert it to a number, and compare.
select *,
cast(`HDD` as unsigned)*
cast(substr(`HDD`,LOCATE('x',`HDD`)+1) as unsigned)*
(case when`HDD` LIKE '%TB%' then 1000 else 1 end) as GB
from `server_details`
where
cast(`HDD` as unsigned)*
cast(substr(`HDD`,LOCATE('x',`HDD`)+1) as unsigned)*
(case when`HDD` LIKE '%TB%' then 1000 else 1 end)
between 120 and 10000;
You can't use between with wildcard queries. You might be able to write a regular expression to match what you need, for example:
select * from `server_details` where `HDD` regexp '1[2-9]\dGB|[2-9]\d\dGB|\dTB|10TB'
but as you can see this is a very specific expression based on what you've written and each different limit will need a different expression.
There's some python code to generate such an expression but no PHP code that I could find (with some very basic googling)
Another solution (and what I would personally recommend) is to add the capacity as a separate column:
Migrate your current table:
class AddCapacityColumnMigration extends Migration {
public function up()
{
Schema::table('computers', function (Blueprint $table) {
$table->bigInt('capacityMB')->nullable();
});
Computer::chunk(100, function ($computers) {
foreach ($computers as $computer) {
if (preg_match('/(\d+)x(\d+)(M|G|T)B/',$computer->HDD,$m) {
$capacity = $m[1];
$capacity *= $m[3] === 'M' ? 1 : ($m[3] === 'G' ? 1000 : 1000000 );
$computer->capacityMB = $capacity * $m[2];
$computer->save();
}
}
});
}
Then you might want to add a creating and updating event in your model to ensure you always set the new capacityMB column. When all this is done your query is as simple as:
select * from `server_details` where `capacityMB` between 120000 and 10000000
In the database, in HDD column, you should not store alpha-numeric values like 120GB, 10TB, you should store numeric values like 120, 10000. Please try with the following query.
$hdds = DB::table('server_details')
->whereBetween('HDD', [120, 10000])
->get();
I have following query against EF whereby mysql was used:
var query = from r in context.myContext
where r.clmn1.CompareTo("2015-11-19 00:00:00") > 0)
orderby r.someColumn
select r;
return query;
The number of returned rows is as expected. however some values of the property r.clmn2 repeat itself in the result of the query. For example I could not find clmn2 == 220011 because it was "overwritten" by the value 220033 (The value 220033 is correct and expected but should not "overwrite" other values). Strangely enough, when I add this condition to the query I get it in the result (of course then only and only this value) which means that the first condition is also valid for clmn2:
var query = from r in context.myContext
where r.clmn1.CompareTo("2015-11-19 00:00:00") > 0) && r.clmn2.Equals("220011")
orderby r.someColumn
select r;
return query;
The same query (the first one) works at DB-level and returns all values (will not be overwritten)
SELECT * FROM myContext.myTable
WHERE r.clmn1 > ("2015-11-19 00:00:00")
ORDER BY r.someColumn
It should be a problem of EF. I hope someone could help me!
Thanks in Advance.
I have prefixed the column/property clmn2 with [key] atribute in the generated entity class so that it is now a part of the multiple key, i.e., with other columns/properties. It works and i get all values from DB. Maybe cus this property comes from a DB-view, Visual Studio could not recognize it as a primary key as done by other properties.
I have a table with rows containing the following numbers:
4191808.51
3035280.22
3437796.06
4013772.33
1740652.56
0
The sum of that table is 16419309.68.
When I do a Linq SUM query on that table, it returns a whole different number: 19876858.14
The code I use to do the sum is as follows:
IEnumerable<MyData> data = _myRepository.GetMatching(myValue);
decimal sumSales = data.Sum(x => x.Sales);
What could be causing this? I suspect some max decimal value but couldn't find info on that
can you please examine your
IEnumerable<MyData> data = _myRepository.GetMatching(myValue);
in the debugger after the execution. I'm suspecting that you are selecting some different set of data - not what you have shown in the sample. I attempted to recreated you situation LinqPad, but constantly getting the correct answer.
decimal[] data = {
4191808.51m
,3035280.22m
,3437796.06m
,4013772.33m
,1740652.56m
,0m
};
decimal sumSales = data.Sum();
sumSales.Dump();
and getting: 16419309.68
I've stumbled upon a very strange LINQ to SQL behaviour / bug, that I just can't understand.
Let's take the following tables as an example: Customers -> Orders -> Details.
Each table is a subtable of the previous table, with a regular Primary-Foreign key relationship (1 to many).
If I execute the follow query:
var q = from c in context.Customers
select (c.Orders.FirstOrDefault() ?? new Order()).Details.Count();
Then I get an exception: Could not format node 'Value' for execution as SQL.
But the following queries do not throw an exception:
var q = from c in context.Customers
select (c.Orders.FirstOrDefault() ?? new Order()).OrderDateTime;
var q = from c in context.Customers
select (new Order()).Details.Count();
If I change my primary query as follows, I don't get an exception:
var q = from r in context.Customers.ToList()
select (c.Orders.FirstOrDefault() ?? new Order()).Details.Count();
Now I could understand that the last query works, because of the following logic:
Since there is no mapping of "new Order()" to SQL (I'm guessing here), I need to work on a local list instead.
But what I can't understand is why do the other two queries work?!?
I could potentially accept working with the "local" version of context.Customers.ToList(), but how to speed up the query?
For instance in the last query example, I'm pretty sure that each select will cause a new SQL query to be executed to retrieve the Orders. Now I could avoid lazy loading by using DataLoadOptions, but then I would be retrieving thousands of Order rows for no reason what so ever (I only need the first row)...
If I could execute the entire query in one SQL statement as I would like (my first query example), then the SQL engine itself would be smart enough to only retrieve one Order row for each Customer...
Is there perhaps a way to rewrite my original query in such a way that it will work as intended and be executed in one swoop by the SQL server?
EDIT:
(longer answer for Arturo)
The queries I provided are purely for example purposes. I know they are pointless in their own right, I just wanted to show a simplistic example.
The reason your example works is because you have avoided using "new Order()" all together. If I slightly modify your query to still use it, then I still get an exception:
var results = from e in (from c in db.Customers
select new { c.CustomerID, FirstOrder = c.Orders.FirstOrDefault() })
select new { e.CustomerID, Count = (e.FirstOrder != null ? e.FirstOrder : new Order()).Details().Count() }
Although this time the exception is slightly different - Could not format node 'ClientQuery' for execution as SQL.
If I use the ?? syntax instead of (x ? y : z) in that query, I get the same exception as I originaly got.
In my real-life query I don't need Count(), I need to select a couple of properties from the last table (which in my previous examples would be Details). Essentially I need to merge values of all the rows in each table. Inorder to give a more hefty example I'll first have to restate my tabels:
Models -> ModelCategoryVariations <- CategoryVariations -> CategoryVariationItems -> ModelModuleCategoryVariationItemAmounts -> ModelModuleCategoryVariationItemAmountValueChanges
The -> sign represents a 1 -> many relationship. Do notice that there is one sign that is the other way round...
My real query would go something like this:
var q = from m in context.Models
from mcv in m.ModelCategoryVariations
... // select some more tables
select new
{
ModelId = m.Id,
ModelName = m.Name,
CategoryVariationName = mcv.CategoryVariation.Name,
..., // values from other tables
Categories = (from cvi in mcv.CategoryVariation.CategoryVariationItems
let mmcvia = cvi.ModelModuleCategoryVariationItemAmounts.SingleOrDefault(mmcvia2 => mmcvia2.ModelModuleId == m.ModelModuleId) ?? new ModelModuleCategoryVariationItemAmount()
select new
{
cvi.Id,
Amount = (mmcvia.ModelModuleCategoryVariationItemAmountValueChanges.FirstOrDefault() ?? new ModelModuleCategoryVariationItemAmountValueChange()).Amount
... // select some more properties
}
}
This query blows up at the line let mmcvia =.
If I recall correctly, by using let mmcvia = new ModelModuleCategoryVariationItemAmount(), the query would blow up at the next ?? operand, which is at Amount =.
If I start the query with from m in context.Models.ToList() then everything works...
Why are you looking into only the individual count without selecting anything related to the customer.
You can do the following.
var results = from e in
(from c in db.Customers
select new { c.CustomerID, FirstOrder = c.Orders.FirstOrDefault() })
select new { e.CustomerID, DetailCount = e.FirstOrder != null ? e.FirstOrder.Details.Count() : 0 };
EDIT:
OK, I think you are over complicating your query.
The problem is that you are using the new WhateverObject() in your query, T-SQL doesnt know anyting about that; T-SQL knows about records in your hard drive, your are throwing something that doesn't exist. Only C# knows about that. DON'T USE new IN YOUR QUERIES OTHER THAN IN THE OUTER MOST SELECT STATEMENT because that is what C# will receive, and C# knows about creating new instances of objects.
Of course is going to work if you use ToList() method, but performance is affected because now you have your application host and sql server working together to give you the results and it might take many calls to your database instead of one.
Try this instead:
Categories = (from cvi in mcv.CategoryVariation.CategoryVariationItems
let mmcvia =
cvi.ModelModuleCategoryVariationItemAmounts.SingleOrDefault(
mmcvia2 => mmcvia2.ModelModuleId == m.ModelModuleId)
select new
{
cvi.Id,
Amount = mmcvia != null ?
(mmcvia.ModelModuleCategoryVariationItemAmountValueChanges.Select(
x => x.Amount).FirstOrDefault() : 0
... // select some more properties
}
Using the Select() method allows you to get the first Amount or its default value. I used "0" as an example only, I dont know what is your default value for Amount.
I'm looking for an explanation of the result of the following query with Active Record:
date_range = (Date.today - 7)..(Time.now.to_datetime)
r = Report.find(:all, :conditions => {:created_at => date_range},
:group => 'date(created_at)',
:select => 'date(created_at) as day, count(id) as counter')
Basically I'm just counting the results in a table "reports," but I got different types of values for the named field "counter", array of double quote strings vs array of only numbers.
MySQL
If using MySQL,
r.map(&:counter)
returns:
=> ["3", "3", "5", "4", "4"]
SQLite
If using SQLite,
r.map(&:counter)
returns:
=> [3, 3, 5, 4, 4]
Is it correct that MySQL returns the numbers with quotes (Strings) and SQLite numbers?
I expected that both return just integers. Or am I missing some configuration with on MySQL side?
Edit:
Just in case, both DBs where created using normal migrations, so all fields are equivalent types.
This is a bug in the SQLite ActiveRecord adapter.
When you are synthesizing new column values using "SELECT expr AS col_name" col_name is cast to a String instead of the proper data type (int in this case).
See related Rails bug ticket: https://rails.lighthouseapp.com/projects/8994-ruby-on-rails/tickets/4544-rails3-activerecord-sqlite3-lost-column-type-when-using-views#ticket-4544-2
Apparently SQLite can't return proper column type values for views, which is the mechanism used when you're synthesizing attributes like this.
I've been having the same issue using the underlying connection to execute my SQL statements.
I have noticed, however, that if you are using binds in exec_sql statement, the data types come back as expected.
E.g.
ActiveRecord::Base.connection.exec_sql("select * from users where id = 35")
results in
ActiveRecord::Result:0x7fa2c6ede200 #columns=["id", "name"], #hash_rows=nil, #rows=[["35", "FOO"]]
However,
ActiveRecord::Base.connection.exec_sql("select * from users where id = ?", nil, [[nil,35]])
results in
ActiveRecord::Result:0x7fa2c6ede200 #columns=["id", "name"],
#hash_rows=nil, #rows=[[35, "FOO"]]
Note that datatype of 35 is integer!
Hope that helps.