I have a simple JSON deepmerge function in my database. It doesn’t try to merge arrays, only objects. The code in SQL is pretty unreadable, so in pseudocode:
define deepmerge(source, delta):
result = {}
for key union(source.keys, delta.keys):
if key not in source:
result[key] = delta[key]
if key not in delta:
result[key] = source[key]
else if type(result[key]) is object and type(delta[key]) = object:
result[key] = deepmerge(source[key], result[key])
else:
result[key] = delta[key]
What I’m interested is, whether the function is associative. I.e. whether I can always be sure that deepmerge(a, deepmerge(b, c)) is equal to deepmerge(deepmerge(a, b), c). This assurance would allow me to optimize some database operations. It seems intuitive for me that it is associative, but I fail to produce a proof that would make me sure of it. How can I formulate a good proof of this property?
Related
Is there any way to initialize a Sequential value not in one fellow swoop?
Like, can I declare it, then use a for loop to populate it, step by step?
As this could all happen inside a class body, the true immutability of the Sequential value could then kick in once the class instance construction phase has been completed.
Example:
Sequential<String> strSeq;
for (i in span(0,10)) {
strSeq[i] = "hello";
}
This code doesn't work, as I get this error:
Error:(12, 9) ceylon: illegal receiving type for index expression:
'Sequential' is not a subtype of 'KeyedCorrespondenceMutator' or
'IndexedCorrespondenceMutator'
So what I can conclude is that sequences must be assigned in one statement, right?
Yes, several language guarantees hinge on the immutability of sequential objects, so that immutability must be guaranteed by the language – it can’t just trust you that you won’t mutate it after the initialization is done :)
Typically, what you do in this situation is construct some sort of collection (e. g. an ArrayList from ceylon.collection), mutate it however you want, and then take its .sequence() when you’re done.
Your specific case can also be written as a comprehension in a sequential literal:
String[] strSeq = [for (i in 0..10) "hello"];
The square brackets used to create a sequence literal accept not only a comma-separated list of values, but also a for-comprehension:
String[] strSeq = [for (i in 0..10) "hello"];
You can also do both at the same time, as long as the for-comprehension comes last:
String[] strSeq = ["hello", "hello", for (i in 0..8) "hello"];
In this specific case, you could also do this:
String[] strSeq = ["hello"].repeat(11);
You can also get a sequence of sequences via nesting:
String[][] strSeqSeq = [for (i in 0..2) [for (j in 0..2) "hello"]];
And you can do the cartesian product (notice that the nested for-comprehension here isn't in square brackets):
[Integer, Character][] pairs = [for (i in 0..2) for (j in "abc") [i, j]];
Foo[] is an abbreviation for Sequential<Foo>, and x..y translates to span(x, y).
If you know upfront the size of the sequence you want to create, then a very efficient way is to use an Array:
value array = Array.ofSize(11, "");
for (i in 0:11) {
array[i] = "hello";
}
String[] strSeq = array.sequence();
On the other hand, if you don't know the size upfront, then, as described by Lucas, you need to use either:
a comprehension, or
some sort of growable array, like ArrayList.
I have two ways in which i can send data(json string) to server.
1)map<key,list>
eg. {"components":["ab","bc","cd"],"values":[1,2,3]}
This represents value of component "ab" is 1.
2) map<key,values>
eg. {"ab":1,"bc":2,"cd":3}
Now above example is of three components but I want to send thousands of components so which one is more scalable in terms of size and performance.
If you feel I missed some details feel free to ask.
Server side you will have to do this if you want value for key "bc":
Option 1: {"components":["ab","bc","cd"],"values":[1,2,3]}
String json = [...] // got json from client
MyJsonObject jo = new Gson().fromJson(json, MyJsonObject.class);
int index = jo.getComponents().indexOf("bc"); // O(n)
int value = jo.getValuess().get(index); // O(1)
Option 2: {"ab":1,"bc":2,"cd":3}
String json = [...] // got json from client
MyJsonObject jo = new Gson().fromJson(json, MyJsonObject.class);
int value = jo.get("bc"); // O(1) ~ O(n)
I believe the option 2 is better because of better performance. Moreover it is simpler and more readable than option 1. Differences in the bandwidth can be neglected.
But you should decide what solution is better for your application.
I'm trying to develop a function which performs math on two values which have the same key:
property = {a=120, b=50, c=85}
operator = {has = {a, b}, coefficient = {a = 0.45}}
function Result(x) return operator.has.x * operator.coefficient.x end
print (Result(a))
error: attempt to perform arithmetic on field 'x' (a nil value)
The problem is that the function is attempting math on literally
"operator.has.x" instead of "operator.has.a".
I'm able to call a function (x) return x.something end, but if I try function (x) something.x i get an error. I need to improve my understanding of functions in Lua, but I can't find this in the manuals.
I'm not exactly sure what you're trying to do, but here is some working code that is based on your code:
property = {a=120, b=50, c=85}
operator = {has = {a=2, b=3}, coefficient = {a = 0.45}}
function Result(x) return operator.has[x] * operator.coefficient[x] end
print (Result('a'))
Prints '0.9'
This is a common gotcha for newcomers to the language. Buried in the Lua manual somewhere:
To represent records, Lua uses the field name as an index. The
language supports this representation by providing a.name as syntactic
sugar for a["name"].
This explains why your function Result(x) is failing. If you translate the syntactic sugar, your function becomes:
function Result(x)
return operator.has['x'] * operator.coefficient['x']
end
Geary already offered a solution to this so I won't reiterate it here.
At the beginning of a method I want to check if the method is called with these exact parameters before, and if so, return the result that was returned back then.
At first, with one parameter, I used a Dictionary, but now I need to check 3 parameters (a String, an Object and a boolean).
I tried making a custom Object like so:
var cacheKey:Object = { identifier:identifier, type:type, someBoolean:someBoolean };
//if key already exists, return it (not working)
if (resultCache[cacheKey]) return resultCache[cacheKey];
//else: create result ...
//and save it in the cache
resultCache[cacheKey] = result;
But this doesn't work, because the seccond time the function is called, the new cacheKey is not the same object as the first, even though it's properties are the same.
So my question is: is there a datatype that will check the properties of the object used as key for a matching key?
And what else is my best option? Create a cache for the keys as well? :/
Note there are two aspects to the technical solution: equality comparison and indexing.
The Cliff Notes version:
It's easy to do custom equality comparison
In order to perform indexing, you need to know more than whether one object is equal to another -- you need to know which is object is "bigger" than the other.
If all of your properties are primitives you should squash them into a single string and use an Object to keep track of them (NOT a Dictionary).
If you need to compare some of the individual properties for reference equality you're going to have a write a function to determine which set of properties is bigger than the other, and then make your own collection class that uses the output of the comparison function to implement its own a binary search tree based indexing.
If the number of unique sets of arguments is in the several hundreds or less AND you do need reference comparison for your Object argument, just use an Array and the some method to do a naive comparison to all cached keys. Only you know how expensive your actual method is, so it's up to you to decide what lookup cost (which depends on the number of unique arguments provided to the function) is acceptable.
Equality comparison
To address equality comparison it is easy enough to write some code to compare objects for the values of their properties, rather than for reference equality. The following function enforces strict set comparison, so that both objects must contain exactly the same properties (no additional properties on either object allowed) with the same values:
public static propsEqual(obj1:Object, obj2:Object):Boolean {
for(key1:* in obj1) {
if(obj2[key1] === undefined)
return false;
if(obj2[key1] != obj2[key1])
return false;
}
for(key2:* in obj2)
if(obj1[key2] === undefined)
return false;
return true;
}
You could speed it up by eliminating the second for loop with the tradeoff that {A:1, B:2} will be deemed equal to {A:1, B:2, C:'An extra property'}.
Indexing
The problem with this in your case is that you lose the indexing that a Dictionary provides for reference equality or that an Object provides for string keys. You would have to compare each new set of function arguments to the entire list of previously seen arguments, such as using Array.some. I use the field currentArgs and the method to avoid generating a new closure every time.
private var cachedArgs:Array = [];
private var currentArgs:Object;
function yourMethod(stringArg:String, objArg:Object, boolArg:Boolean):* {
currentArgs = { stringArg:stringArg, objArg:objArg, boolArg:boolArg };
var iveSeenThisBefore:Boolean = cachedArgs.some(compareToCurrent);
if(!iveSeenThisBefore)
cachedArgs.push(currentArgs);
}
function compareToCurrent(obj:Object):Boolean {
return someUtil.propsEqual(obj, currentArgs);
}
This means comparison will be O(n) time, where n is the ever increasing number of unique sets of function arguments.
If all the arguments to your function are primitive, see the very similar question In AS3, where do you draw the line between Dictionary and ArrayCollection?. The title doesn't sound very similar but the solution in the accepted answer (yes I wrote it) addresses the exact same techinical issue -- using multiple primitive values as a single compound key. The basic gist in your case would be:
private var cachedArgs:Object = {};
function yourMethod(stringArg:String, objArg:Object, boolArg:Boolean):* {
var argKey:String = stringArg + objArg.toString() + (boolArg ? 'T' : 'F');
if(cachedArgs[argKey] === undefined)
cachedArgs[argKey] = _yourMethod(stringArg, objArg, boolArg);
return cachedArgs[argKey];
}
private function _yourMethod(stringArg:String, objArg:Object, boolArg:Boolean):* {
// Do stuff
return something;
}
If you really need to determine which reference is "bigger" than another (as the Dictionary does internally) you're going to have to wade into some ugly stuff, since Adobe has not yet provided any API to retrieve the "value" / "address" of a reference. The best thing I've found so far is this interesting hack: How can I get an instance's "memory location" in ActionScript?. Without doing a bunch of performance tests I don't know if using this hack to compare references will kill the advantages gained by binary search tree indexnig. Naturally it would depend on the number of keys.
any hint on what's wrong with the below query?
return new ItemPricesViewModel()
{
Source = (from o in XpoSession.Query<PRICE>()
select new ItemPriceViewModel()
{
ID = o.ITEM_ID.ITEM_ID,
ItemCod = o.ITEM_ID.ITEM_COD,
ItemModifier = o.ITEM_MODIFIER_ID.ITEM_MODIFIER_COD,
ItemName = o.ITEM_ID.ITEM_COD,
ItemID = o.ITEM_ID.ITEM_ID,
ItemModifierID = o.ITEM_MODIFIER_ID.ITEM_MODIFIER_ID,
ItemPrices = (from d in o
where d.ITEM_ID.ITEM_ID == o.ITEM_ID.ITEM_ID && d.ITEM_MODIFIER_ID.ITEM_MODIFIER_ID == o.ITEM_MODIFIER_ID.ITEM_MODIFIER_ID
select new Price()
{
ID = o.PRICE_ID,
PriceList = o.PRICELIST_ID.PRICELIST_,
Price = o.PRICE_
}).ToList()
}).ToList()
};
o in subquery is in read and I got the message "Could not find an implementation of the query pattern for source type . 'Where' not found."
I would like to have distinct ItemID, ItemModifier: should I create a custom IEqualityComparer to do it?
Thank you!
It seems like XPO it's not able to respond to this scenario. For reference this is what you could do with DbContext.
It sounds like maybe you want a GroupBy. Try something like this.
var result = dbContext.Prices
.GroupBy(p => new {p.ItemName, p.ItemTypeName)
.Select(g => new Item
{
ItemName = g.Key.ItemName,
ItemTypeName = g.Key.ItemTypeName,
Prices = g.Select(p => new Price
{
Price = p.Price
}
).ToList()
})
.Skip(x)
.Take(y)
.ToList();
Probable cause
In general, XPO does not support "free joins" in most of the cases. It was explicitelly written somewhere in their knowledgebase or Q/A site. If I hit that article again, I'll include a link to it.
In your original code example, you were trying to perform a "free join" in the INNER query. The 'WHERE' clause was doing a join-by-key, probably navigational, but also it contained an extra filter by "modifier" which probably is not a part of the definition of the relation.
Moreover, the query tried to reuse the IQueryable<PRICE> o in the inner query - what actually seems somewhat supported by XPO - but if you ever add any prefiltering ('where') to the toplevel 'o', it would have high odds of breaking again.
The docs state that XPO supports only navigational joins, along paths formed by properties and/or xpcollections defined in your XPObjects. This applies to XPO as whole, so XPQuery too. All other kinds of joins are called "free joins" and either:
are silently emulated by XPO by fetching related objects, extracting key values from them and rewriting the query into a multiple roundtrips with a series of partial queries that fetch full objects with WHERE-id-IN-(#p0,#p1,#p2,...) - but this happens only in the some simpliest cases
or are "not fully supported", meaning they throw exceptions and require you to manually split the query or rephrase it
Possible direct solution schemes
If ITEM_ID is a relation and XPCollection in PRICE class, then you could rewrite your query so that it fetches a PRICE object then builds up a result object and initializes its fields with PRICE object's properties. Something like:
return new ItemPricesViewModel()
{
Source = (from o in XpoSession.Query<PRICE>().AsEnumerable()
select new ItemPriceViewModel()
{
ID = o.ITEM_ID.ITEM_ID,
ItemCod = o.ITEM_ID.ITEM_COD,
....
ItemModifierID = o.ITEM_MODIFIER_ID.ITEM_MODIFIER_ID,
ItemPrices = (from d in o
where d.ITEM_ID.ITEM_ID == ....
select new Price()
.... .... ....
};
Note the 'AsEnumerable' that breaks the query and ensures that PRICE objects are first fetched instead of just trying to translate the query. Very probable that this would "just work".
Also, splitting the query into explicit stages sometimes help the XPO to analyze it:
return new ItemPricesViewModel()
{
Source = (from o in XpoSession.Query<PRICE>()
select new
{
id = o.ITEM_ID.ITEM_ID,
itemcod = o.ITEM_ID.ITEM_COD,
....
}
).AsEnumerable()
.Select(temp =>
select new ItemPriceViewModel()
{
ID = temp.id
ItemCod = temp.itemcod,
....
ItemPrices = (from d in XpoSession.Query<PRICE>()
where d.ITEM_ID.ITEM_ID == ....
select new Price()
.... .... ....
};
Here, note that I first fetch the item-data from server, and then conctruct the item on the 'client', and then build the required groupings. Note that I could not refer to the variable o anymore. In these precise case and examples, unsuprisingly, the second one (splitted) would be probably even slower than the first one, since it would fetch all PRICEs and then refetch the groupings through additional queries, while the first one would just fetch all PRICEs and then would calculate the groups in-memory basing on the PRICEs already fetched. This is not an a sideeffect of my laziness, but it is a common pitfall when rewriting the LINQ queries, so I included it as a warning :)
Both of these code examples are NOT RECOMMENDED for your case, as they would probably have very poor performace, especially if you have many PRICEs in the table, which is highly likely. I included them to present as only an example of how you could rewrite the query to siplify its structure so the XPO can eat it without choking. However, you have to be really careful and pay attention to little details, as you can very easily spoil the performance.
observations and real solution
However, it is worth noting that they are not that much worse than your original query. It was itself quite poor, since it tried to perform something near O(N^2) row-fetches from the table just to perform to group te rows by "ITEM_ID" and then formatting the results as separate objects. Properly done, it would be something like O(N lg N)+O(N), so regardless of being supported or not, your alternate attempt with GroupBy is surely a much better approach, and I'm really glad you found it yourself.
Very often when you are trying to split/simplify the XPQuery expressions as I did above, you implicitely rethink the problem and find an easier and simplier way to express the query that was initially not-supported or just were crashing.
Unfortunatelly, your query was in fact quite simple. For a really complex queries that cannot be "just rephrased", splitting into stages and making some of the join-filter work at 'client' is unavoidable.. But again, doing them on XPCollections or XPViews with CritieriaOperators is impossible too, so either we have to bear with it or use plain direct handcrafted SQL..
Sidenote:
Whole XPO has problems with "free joins", they are "not fully supported" not only in XPQuery, but also there's not much for them in XPCollection, XPView, CriteriaOperators, etc, too. But, it is worth noting that at least in "my version" of DX11, the XPQuery has very poor LINQ support at all.
I've hit many cases where a proper LINQ query was:
throwing "NotSupportedException", mostly in FreeJoins, but also very often with complex GroupBy or Select-with-Projection, GroupJoin, and many others - sometimes even Distinct(!) seemed to malfunction
throwing "NullReferenceExceptions" at some proper type conversions (XPO tried to interprete a column that held INT/NULL as an object..), often I had to write some completely odd and artificial expressions like foo!=null && foo.bar!=123 instead of foo = 123 despite the 'foo' being an public int Foo {get;set;}, all because the DX could not cope properly with NULLs in the database (because XPO created nullable-INT column for this property.. but that's another story)
throwing other random ArgumentException/InvalidOperation exceptions from other constructs
or even analyzing the query structure improperly, for example this one is usually valid:
session.Query<ABC>()
.Where( abc => abc.foo == "somefilter" )
.Select( abc => new { first = abc, b = abc } )
.ToArray();
but things like this one usually throws:
session.Query<ABC>()
.Select( abc => new { first = abc, b = abc } )
.Where ( temp => temp.first.foo == "somefilter" )
.ToArray();
but this one is valid:
session.Query<ABC>()
.Select( abc => new { first = abc, b = abc } )
.ToArray()
.Where ( temp => temp.first.foo == "somefilter" )
.ToArray();
The middle code example usually throws with an error that reveals that XPO layer were trying to find ".first.foo" path inside the ABC class, which is obviously wrong since at that point the element type isn't ABC anymore but instead a' anonymous class.
disclaimer
I've already noted that, but let me repeat: these observations are related to DX11 and most probably also earlier. I do not know what of that was fixed in DX12 and above (if anything at all was!).