I do want to join two table in HTML5 indexed db.
I found lot samples to add, Update, Delete and listing record but can't found any samples to join multiple table.
Sample URL:
*http://users.telenet.be/kristofdegrave/*
Was wondering the same question when just generally looking at these technologies. Did not check your particular example, but here is an example from Mozilla on how to do a join if you need it:
https://hacks.mozilla.org/2010/06/comparing-indexeddb-and-webdatabase/
It is quite a lot of code compared to SQL, but I think the idea is that abstracting these using JavaScript later is quite easy.
There is no join, but you can open multiple object stores and join during cursor iterations.
Using my own ydn-db library, you can query SELECT * FROM Supplier, Part WHERE Supplier.CITY = Part.CITY by
var iter_supplier = new ydn.db.IndexValueIterator('Supplier', 'CITY');
var iter_part = new ydn.db.IndexValueIterator('Part', 'CITY');
var req = db.scan(function(keys, values) {
var SID = keys[0];
var PID = keys[1];
console.log(SID, PID);
if (!SID || !PID) {
return []; // done
}
var cmp = ydn.db.cmp(SID, PID); // compare keys
if (cmp == 0) {
console.log(values[0], values[1]);
return [true, true]; // advance both
} else if (cmp == 1) {
return [undefined, SID]; // jump PID cursor to match SID
} else {
return [PID, undefined]; // jump SID cursor to match PID
}
}, [iter_supplier, iter_part]);
See more detail on Join query article.
As far as I know, IndexedDB does not have an API for doing JOINs, yet. The solutions I adopted involved opening a cursor, looping through the results and doing the join manually. It's the dreadful RBAR approach, but I couldn't find a better solution.
Instead of trying to join, redesign how you store the data so that a join is not required. You do not need to follow the same normalization constraints as SQL when you are using a no-sql approach. No-sql supports storing data redundantly when it is appropriate.
IndexedDB is in fact an object store and normaly in an object store there is less need for joins because you can just save the nested structure.
In the case you show, data from a codetable gets joined with real data. The case to solve this without a join is just fetch your code table into memory (normally this data never changes) and make the join in code.
Related
any hint on what's wrong with the below query?
return new ItemPricesViewModel()
{
Source = (from o in XpoSession.Query<PRICE>()
select new ItemPriceViewModel()
{
ID = o.ITEM_ID.ITEM_ID,
ItemCod = o.ITEM_ID.ITEM_COD,
ItemModifier = o.ITEM_MODIFIER_ID.ITEM_MODIFIER_COD,
ItemName = o.ITEM_ID.ITEM_COD,
ItemID = o.ITEM_ID.ITEM_ID,
ItemModifierID = o.ITEM_MODIFIER_ID.ITEM_MODIFIER_ID,
ItemPrices = (from d in o
where d.ITEM_ID.ITEM_ID == o.ITEM_ID.ITEM_ID && d.ITEM_MODIFIER_ID.ITEM_MODIFIER_ID == o.ITEM_MODIFIER_ID.ITEM_MODIFIER_ID
select new Price()
{
ID = o.PRICE_ID,
PriceList = o.PRICELIST_ID.PRICELIST_,
Price = o.PRICE_
}).ToList()
}).ToList()
};
o in subquery is in read and I got the message "Could not find an implementation of the query pattern for source type . 'Where' not found."
I would like to have distinct ItemID, ItemModifier: should I create a custom IEqualityComparer to do it?
Thank you!
It seems like XPO it's not able to respond to this scenario. For reference this is what you could do with DbContext.
It sounds like maybe you want a GroupBy. Try something like this.
var result = dbContext.Prices
.GroupBy(p => new {p.ItemName, p.ItemTypeName)
.Select(g => new Item
{
ItemName = g.Key.ItemName,
ItemTypeName = g.Key.ItemTypeName,
Prices = g.Select(p => new Price
{
Price = p.Price
}
).ToList()
})
.Skip(x)
.Take(y)
.ToList();
Probable cause
In general, XPO does not support "free joins" in most of the cases. It was explicitelly written somewhere in their knowledgebase or Q/A site. If I hit that article again, I'll include a link to it.
In your original code example, you were trying to perform a "free join" in the INNER query. The 'WHERE' clause was doing a join-by-key, probably navigational, but also it contained an extra filter by "modifier" which probably is not a part of the definition of the relation.
Moreover, the query tried to reuse the IQueryable<PRICE> o in the inner query - what actually seems somewhat supported by XPO - but if you ever add any prefiltering ('where') to the toplevel 'o', it would have high odds of breaking again.
The docs state that XPO supports only navigational joins, along paths formed by properties and/or xpcollections defined in your XPObjects. This applies to XPO as whole, so XPQuery too. All other kinds of joins are called "free joins" and either:
are silently emulated by XPO by fetching related objects, extracting key values from them and rewriting the query into a multiple roundtrips with a series of partial queries that fetch full objects with WHERE-id-IN-(#p0,#p1,#p2,...) - but this happens only in the some simpliest cases
or are "not fully supported", meaning they throw exceptions and require you to manually split the query or rephrase it
Possible direct solution schemes
If ITEM_ID is a relation and XPCollection in PRICE class, then you could rewrite your query so that it fetches a PRICE object then builds up a result object and initializes its fields with PRICE object's properties. Something like:
return new ItemPricesViewModel()
{
Source = (from o in XpoSession.Query<PRICE>().AsEnumerable()
select new ItemPriceViewModel()
{
ID = o.ITEM_ID.ITEM_ID,
ItemCod = o.ITEM_ID.ITEM_COD,
....
ItemModifierID = o.ITEM_MODIFIER_ID.ITEM_MODIFIER_ID,
ItemPrices = (from d in o
where d.ITEM_ID.ITEM_ID == ....
select new Price()
.... .... ....
};
Note the 'AsEnumerable' that breaks the query and ensures that PRICE objects are first fetched instead of just trying to translate the query. Very probable that this would "just work".
Also, splitting the query into explicit stages sometimes help the XPO to analyze it:
return new ItemPricesViewModel()
{
Source = (from o in XpoSession.Query<PRICE>()
select new
{
id = o.ITEM_ID.ITEM_ID,
itemcod = o.ITEM_ID.ITEM_COD,
....
}
).AsEnumerable()
.Select(temp =>
select new ItemPriceViewModel()
{
ID = temp.id
ItemCod = temp.itemcod,
....
ItemPrices = (from d in XpoSession.Query<PRICE>()
where d.ITEM_ID.ITEM_ID == ....
select new Price()
.... .... ....
};
Here, note that I first fetch the item-data from server, and then conctruct the item on the 'client', and then build the required groupings. Note that I could not refer to the variable o anymore. In these precise case and examples, unsuprisingly, the second one (splitted) would be probably even slower than the first one, since it would fetch all PRICEs and then refetch the groupings through additional queries, while the first one would just fetch all PRICEs and then would calculate the groups in-memory basing on the PRICEs already fetched. This is not an a sideeffect of my laziness, but it is a common pitfall when rewriting the LINQ queries, so I included it as a warning :)
Both of these code examples are NOT RECOMMENDED for your case, as they would probably have very poor performace, especially if you have many PRICEs in the table, which is highly likely. I included them to present as only an example of how you could rewrite the query to siplify its structure so the XPO can eat it without choking. However, you have to be really careful and pay attention to little details, as you can very easily spoil the performance.
observations and real solution
However, it is worth noting that they are not that much worse than your original query. It was itself quite poor, since it tried to perform something near O(N^2) row-fetches from the table just to perform to group te rows by "ITEM_ID" and then formatting the results as separate objects. Properly done, it would be something like O(N lg N)+O(N), so regardless of being supported or not, your alternate attempt with GroupBy is surely a much better approach, and I'm really glad you found it yourself.
Very often when you are trying to split/simplify the XPQuery expressions as I did above, you implicitely rethink the problem and find an easier and simplier way to express the query that was initially not-supported or just were crashing.
Unfortunatelly, your query was in fact quite simple. For a really complex queries that cannot be "just rephrased", splitting into stages and making some of the join-filter work at 'client' is unavoidable.. But again, doing them on XPCollections or XPViews with CritieriaOperators is impossible too, so either we have to bear with it or use plain direct handcrafted SQL..
Sidenote:
Whole XPO has problems with "free joins", they are "not fully supported" not only in XPQuery, but also there's not much for them in XPCollection, XPView, CriteriaOperators, etc, too. But, it is worth noting that at least in "my version" of DX11, the XPQuery has very poor LINQ support at all.
I've hit many cases where a proper LINQ query was:
throwing "NotSupportedException", mostly in FreeJoins, but also very often with complex GroupBy or Select-with-Projection, GroupJoin, and many others - sometimes even Distinct(!) seemed to malfunction
throwing "NullReferenceExceptions" at some proper type conversions (XPO tried to interprete a column that held INT/NULL as an object..), often I had to write some completely odd and artificial expressions like foo!=null && foo.bar!=123 instead of foo = 123 despite the 'foo' being an public int Foo {get;set;}, all because the DX could not cope properly with NULLs in the database (because XPO created nullable-INT column for this property.. but that's another story)
throwing other random ArgumentException/InvalidOperation exceptions from other constructs
or even analyzing the query structure improperly, for example this one is usually valid:
session.Query<ABC>()
.Where( abc => abc.foo == "somefilter" )
.Select( abc => new { first = abc, b = abc } )
.ToArray();
but things like this one usually throws:
session.Query<ABC>()
.Select( abc => new { first = abc, b = abc } )
.Where ( temp => temp.first.foo == "somefilter" )
.ToArray();
but this one is valid:
session.Query<ABC>()
.Select( abc => new { first = abc, b = abc } )
.ToArray()
.Where ( temp => temp.first.foo == "somefilter" )
.ToArray();
The middle code example usually throws with an error that reveals that XPO layer were trying to find ".first.foo" path inside the ABC class, which is obviously wrong since at that point the element type isn't ABC anymore but instead a' anonymous class.
disclaimer
I've already noted that, but let me repeat: these observations are related to DX11 and most probably also earlier. I do not know what of that was fixed in DX12 and above (if anything at all was!).
SQL Server 2008 Ent
ASP.NET MVC 2.0
Linq-to-SQL
I am building a gaming site, that tracks when a particular player (toon) had downed a particular monster (boss). Table looks something like:
int ToonId
int BossId
datetime LastKillTime
I use a 3d party service that gives me back latest information (toon,boss,time).
Now I want to update my database with that new information.
Brute force approach is to do line-by-line upsert. But It looks ugly (code-wise), and probably slow too.
I think better solution would be to insert new data (using temp table?) and then run MERGE statement.
Is it good idea? I know temp tables are "better-to-avoid". Should I create a permanent "temp" table just for this operation?
Or should I just read entire current set (100 rows at most), do merge and put it back from within application?
Any pointers/suggestions are always appreciated.
An ORM is the wrong tool for performing batch operations, and Linq-to-SQL is no exception. In this case I think you have picked the right solution: Store all entries in a temporary table quickly, then do the UPSERT using merge.
The fastest way to store the data to the temporary table is to use SqlBulkCopy to store all data to a table of your choice.
If you're using Linq-to-SQL, upserts aren't that ugly..
foreach (var line in linesFromService) {
var kill = db.Kills.FirstOrDefault(t=>t.ToonId==line.ToonId && t.BossId==line.BossId);
if (kill == null) {
kill = new Kills() { ToonId = line.ToonId, BossId = line.BossId };
db.Kills.InsertOnSubmit(kill);
}
kill.LastKillTime = line.LastKillTime;
}
db.SubmitChanges();
Not a work of art, but nicer than in SQL. Also, with only 100 rows, I wouldn't be too concerned about performance.
Looks like a straight-forward insert.
private ToonModel _db = new ToonModel();
Toon t = new Toon();
t.ToonId = 1;
t.BossId = 2;
t.LastKillTime = DateTime.Now();
_db.Toons.InsertOnSubmit(t);
_db.SubmitChanges();
To update without querying the records first, you can do the following. It will still hit the db once to check if record exists but will not pull the record:
var blob = new Blob { Id = "some id", Value = "some value" }; // Id is primary key (PK)
if (dbContext.Blobs.Contains(blob)) // if blob exists by PK then update
{
// This will update all columns that are not set in 'original' object. For
// this to work, Blob has to have UpdateCheck=Never for all properties except
// for primary keys. This will update the record without querying it first.
dbContext.Blobs.Attach(blob, original: new Blob { Id = blob.Id });
}
else // insert
{
dbContext.Blobs.InsertOnSubmit(blob);
}
dbContext.Blobs.SubmitChanges();
See here for an extension method for this.
I am trying to get the records from the 'many' table of a one-to-many relationship and add them as a list to the relevant record from the 'one' table.
I am also trying to do this in a single database request.
Code derived from Linq to Sql - Populate JOIN result into a List almost achieves the intended result, but makes one database request per entry in the 'one' table which is unacceptable. That failing code is here:
var res = from variable in _dc.GetTable<VARIABLE>()
select new { x = variable, y = variable.VARIABLE_VALUEs };
However if I do a similar query but loop through all the results, then only a single database request is made. This code achieves all goals:
var res = from variable in _dc.GetTable<VARIABLE>()
select variable;
List<GDO.Variable> output = new List<GDO.Variable>();
foreach (var v2 in res)
{
List<GDO.VariableValue> values = new List<GDO.VariableValue>();
foreach (var vv in v2.VARIABLE_VALUEs)
{
values.Add(VariableValue.EntityToGDO(vv));
}
output.Add(EntityToGDO(v2));
output[output.Count - 1].VariableValues = values;
}
However the latter code is ugly as hell, and it really feels like something that should be do-able in a single linq query.
So, how can this be done in a single linq query that makes only a single database query?
In both cases the table is set to preload using the following code:
_dc = _db.CreateLinqDataContext();
var loadOptions = new DataLoadOptions();
loadOptions.LoadWith<VARIABLE>(v => v.VARIABLE_VALUEs);
_dc.LoadOptions = loadOptions;
I am using .NET 3.5, and the database back-end was generated using SqlMetal.
This link may help
http://msdn.microsoft.com/en-us/vcsharp/aa336746.aspx
Look under join operators. You'll probably have to change from using extension syntax other syntax too. Like this,
var = from obj in dc.Table
from obj2 in dc.Table2
where condition
select
So, I'm trying to return a collection of People whose ID is contained within a locally created collection of ids ( IQueryable)
When I specify "locally created collection", I mean that the Ids collection hasnt come from a LinqToSql query and has been programatically created (based upon user input).
My query looks like this:
var qry = from p in DBContext.People
where Ids.Contains(p.ID)
select p.ID;
This causes the following exception...
"queries with local collections are not supported"
How can I find all the People with an id that is contained within my locally created Ids collection?
Is it possible using LinqToSql?
If Ids is a List, array or similar, L2S will translate into a contains.
If Ids is a IQueryable, just turn it into a list before using it in the query. E.g.:
List<int> listOfIDs = IDs.ToList();
var query =
from st in dc.SomeTable
where listOfIDs.Contains(st.ID)
select .....
I was struggling with this problem also. Solved my problem with using Any() instead
people.Where(x => ids.Any(id => id == x.ID))
As the guys mentioned above, converting the ids, which is of type IQueryable to List or Array will solve the issue, this will be translated to "IN" operator in SQL.But be careful because if the count of ids >= 2100 this will cause another issue which is "The server supports a maximum of 2100 parameters" and that is the maximum number of parameters(values) you can pass to "IN" in SQL server.
Another alternative would be keeping ids as IQueryable and using LINQ "Any" operator instead of "Contains", this will be translated to "EXISTS" in SQL server.
I'm sorry but the answers here didn't work for me as I'm doing dynamic types further along.
What I did was to use "UNION" in a loop which works great. Here's how:
var firstID = cityList.First().id;
var cities = dc.zs_Cities.Where(c => c.id == firstID);
foreach(var c in cityList)
{
var tempCity = c;
cities = cities.Union(dc.zs_Cities.Where(cty => cty.id == tempCity.id));
}
We have followed the approach below to get the data from multiple results using LINQ To SQL
CREATE PROCEDURE dbo.GetPostByID
(
#PostID int
)
AS
SELECT *
FROM Posts AS p
WHERE p.PostID = #PostID
SELECT c.*
FROM Categories AS c
JOIN PostCategories AS pc
ON (pc.CategoryID = c.CategoryID)
WHERE pc.PostID = #PostID
The calling method in the class the inherits from DataContext should look like:
[Database(Name = "Blog")]
public class BlogContext : DataContext
{
...
[Function(Name = "dbo.GetPostByID")]
[ResultType(typeof(Post))]
[ResultType(typeof(Category))]
public IMultipleResults GetPostByID(int postID)
{
IExecuteResult result =
this.ExecuteMethodCall(this,
((MethodInfo)(MethodInfo.GetCurrentMethod())),
postID);
return (IMultipleResults)(result.ReturnValue);
}
}
Notice that the method is decorated not only with the Function attribute that maps to the stored procedure name, but also with the ReturnType attributes with the types of the result sets that the stored procedure returns. Additionally, the method returns an untyped interface of IMultipleResults:
public interface IMultipleResults : IFunctionResult, IDisposable
{
IEnumerable<TElement> GetResult<TElement>();
}
so the program can use this interface in order to retrieve the results:
BlogContext ctx = new BlogContext(...);
IMultipleResults results = ctx.GetPostByID(...);
IEnumerable<Post> posts = results.GetResult<Post>();
IEnumerable<Category> categories = results.GetResult<Category>();
In the above stored procedures we had two select queries
1. Select query without join
2. Select query with Join
But in the above second select query the data which is displayed is from one of the table i.e. from Categories table. But we have used join and want to display the data table with the results from both the tables i.e. from Categories as well as PostCategories.
Please if anybody can let me know how to achieve this using LINQ to SQL
What is the performance trade-off if we use the above approach vis-à-vis implement the above approach with simple SQL
Scott Guthrie (the guy who runs the .Net dev teams at MS) covered how to do this on his blog some months ago much better than I ever could, link here. On that page there is a section titled "Handling Multiple Result Shapes from SPROCs". That explains how to handle multiple results from stored procs of different shapes (or the same shape).
I highly recommend subscribing to his RSS feed. He is pretty much THE authoritative source on all things .Net.
Heya dude - does this work?
IEnumerable<Post> posts;
IEnumerable<Category> categories;
using (BlogContext ctx = new BlogContext(...))
{
ctx.DeferredLoadingEnabled = false; // THIS IS IMPORTANT.
IMultipleResults results = ctx.GetPostByID(...);
posts = results.GetResult<Post>().ToList();
categories = results.GetResult<Category>().ToList();
}
// Now we need to associate each category to the post.
// ASSUMPTION: Each post has only one category (1-1 mapping).
if (posts != null)
{
foreach(var post in posts)
{
int postId = post.PostId;
post.Category = categories
.Where(p => p.PostId == postId)
.SingleOrDefault();
}
}
Ok. lets break this down.
First up, a nice connection inside a using block (so it's disposed of nicely).
Next, we make sure DEFERRED LOADING is off. Otherwise, when u try and do the set (eg. post.Category == blah) it will see that it's null, lazy-load the data (eg. do a rountrip the database) set the data and THEN override the what was just dragged down from the db, with the result of there Where(..) method. phew! Summary: make sure deferred loading is off for the scope of the query.
Last, for each post, iterate and set the category from the second list.
does that help?
EDIT
Fixed it so that it doesn't throw an enumeration error by calling the ToList() methods.
Just curious, if a Post have have one or many Categories, is it possible to instead of using the for loop, to load the Post.PostCategories with the list of Categories (one to many), all in one shot, using a JOIN?
var rslt = from p in results.GetResult<Post>()
join c in results.GetResult<Category>() on p.PostId = c.PostID
...
p.Categories.Add(c)