Remove layers (keys) from heavily nested JSON in JSCRIPT/VBA - json

I am parsing a heavily nested JSON in VBA, using scriptcontrol/jscript.
The resulting JSON object is super nested, and has recurring 'useless' levels / layers called 'buckets'.
Is there a way I can remove these collectively from either my json string or the parsed json object?
Imagine it something like this:
responses.0.buckets.0.aggregations.0.10.buckets.0.5.buckets.0.9.buckets.0.20.buckets.0.8.buckets.0.13.buckets.0.14.buckets.0.15.buckets.0.16.buckets.0.19.buckets.0.18.buckets.0.21.doc_count_error_upper_bound
I'd only need the 'doc_count_error_upper_bound' value, and could essentially do without all the 0s and without all the buckets, making it less nested into:
responses.aggregations.10.5.9.20.8.13.14.15.16.19.18.21.doc_count_error_upper_bound
This would still be pretty heavily nested, but saves me a lot of headaches already.
I just do not know how I could do this with jscript/scriptcontrol in VBA (es3).
The source data is coming from a Kibana dashboard (examples on http://demo.elastic.co/ )
Thanks for any help!
Jasper
UPDATE:
Question regarding VBA code - the VBA code I have is irrelevant, as it's the standard way of loading a json string into an object via scriptcontrol.
I do not use EVAL, but for example purposes, it would be something like the below:
Dim Scr as Object, Json as Object
Set Scr = CreateObject("Scriptcontrol")
Scr.Language = "Jscript"
Set Json = Scr.Eval("(" & WinHTTP.ResponseText & ")")
I cannot share an example of the JSON string, as it contains sensitive data.
But ultimately, that's beside the question.
Consider example https://adobe.github.io/Spry/data/json/donuts.js
On the top there, is "batter" as key in between "batters" and the different IDs. If I'd want to remove that key, but keep the underlying ID data - how would I do that, through a js scrip that works in scriptcontrol in VBA?
UPDATE:
omegastripes answer worked very well, however, I failed to realize that a number of the keys I wanted to remove (the 'buckets' and '0' etc) had keys and values under them.
Let's take the example of the donuts, just altered a bit - see here:
https://pastebin.com/WxYir7vK
now I would want to remove the '0', '1', '2' etc keys without losing the underlying sub-keys.
However, for omegastripes code to work, I'd have to delete keys 'sequence', 'variant', 'name', and 'ppu' from all layers / throughout the json.
I can do that for one of them, for one layer with the function below:
function unseat(obj, prop) { for(var k in obj[prop]) obj[k] = obj[prop][k]; delete obj[prop]; return obj; }
And then calling the functio 'unseat (JSONObj, "variant")' - this works, but only for one of the four variables at a time and only for one layer.
How can I alter this so that I can remove it throughout the object, for all four at once, so that afterwards I can use omegastripes code to unwrap.
Summary
1) I take this json string: https://pastebin.com/WxYir7vK
2) parse it into script control into VBA
3) loop through it and remove all 'sequence', 'variant', 'name' and 'ppu' key/value pairs
4) unwrap it via omegastripes code.
Step 1 / 2 and 4 are taken care of - but how to do 3?
Thanks!

Using ScriptControl for parsing JSON has the following shortcomings (check this answer for details):
System environment is exposed to malware code injections received within response.
ScriptControl is not available on 64-bit MS Office.
Anyway if you are confident that operating in JScript environment is the only way, you may unwrap excessive nesting of objects and arrays structure using the below functions:
function gParse(sample) {
return eval('(' + sample + ')');
};
function gUnwrap(sample) {
for (var key in sample) {
sample[key] = gUnwrap(sample[key]);
};
var count = 0;
for (var key in sample) {
count++;
if (count == 2) break;
};
if (count == 1) {
var type = gGetType(sample);
if (type == 'Array' || type == 'Object') {
var type = gGetType(sample[key]);
if (type == 'Array' || type == 'Object') {
return sample[key];
}
}
};
return sample;
};
function gGetType(sample) {
return {}.toString.call(sample).slice(8, -1);
};
That could be done in VBA as shown below:
Option Explicit
Sub Test()
Dim sJSON As String
Dim ParseJSON As Object
Dim UnwrapJSON As Object
Dim oJSON As Object
With CreateObject("MSXML2.XMLHTTP")
.Open "GET", "https://adobe.github.io/Spry/data/json/donuts.js", False
.send
sJSON = .responseText
End With
With CreateObject("htmlfile")
With .parentWindow
.execScript "function gParse(sample) {return eval('(' + sample + ')')};"
.execScript "function gUnwrap(sample) {for (var key in sample) {sample[key] = gUnwrap(sample[key]);}; var count = 0; for (var key in sample) {count++; if (count == 2) break;}; if (count == 1) {var type = gGetType(sample); if (type == 'Array' || type == 'Object') {var type = gGetType(sample[key]); if (type == 'Array' || type == 'Object') {return sample[key];}}}; return sample;};"
.execScript "function gGetType(sample) {return {}.toString.call(sample).slice(8, -1)};"
Set ParseJSON = .gParse
Set UnwrapJSON = .gUnwrap
End With
End With
Set oJSON = UnwrapJSON(ParseJSON(sJSON))
End Sub
The locals window shows JSON object for the sample you provided as follows:
And unwrapped JSON object:

Related

how to convert type 'system.linq.iqueryable<byte[]>' to 'byte[]'

In Linq to SQL asp.net, I want to fetch a specific image from database and display on an aspx page. I declared returned type as "var" on the code. So after fetching the image, it assigns image to var type.
Here is the code:
var db = from c in context.Images
where c.imageId == iID
select c.imageData;
return new MemoryStream((byte[])db); ---->> this line of code gives error
It gives this compile error: Cannot convert type 'system.linq.iqueryable' to 'byte[]'.
How can I convert 'system.linq.iqueryable type to byte[] ?
Your problem is that you're selecting a collection of byte[] rather than a single byte[].
Depending on the behavior you want for your application, you can use Single, SingleOrDefault, First, or FirstOrDeault:
var image = context.Images.FirstOrDefault(i => i.imageId == ID);
if(image == null)
{
// Handle the case where no image was found.
//
// First or Single would throw Exceptions in this case
// if that's what you want.
}
else
{
return new MemoryStream(image.imageData);
}
try below line this work for me
var db = (context.Images.FirstOrDefault(i => i.imageId == ID).ImageData

as3 dictionary of arrays

My code:
var dict = new Dictionary();
dict[[1, 2]] = 1;
var dbg = dict[[1, 2]];
trace(dbg);
Output:
undefined
Why?
How can I get
set<pair<int, int> >
from C++?
Note: I would not call [1,2] "pair of int", it is "array with 2 elements". { item1:1, item2:2 } would look more like pair to me.
According to documentation Dictionary uses strict equals (===) to compare keys, so passing new array as key every time will not work for indexing (arrays are strictly equal only when it is exactly the same array, not an array with the same content).
Depending on your needs converting key to a string maybe good option:
var keyPair = [1,2];
var key = keyPair[0] + "_" + keyPair[1];

Why does iterating over a generic object in ActionScript give me values instead of keys?

I may be doing something really stupid, but I don't get why the below code doesn't work...
Here, I create a generic Object called spellbook:
// A list of all the player's spells
public var spellBook:Object = {};
Here, I add a key-value pair to the spellbook:
spellBook["bubble"] = new BubbleSpell(spellBook);
And here I try to output the contents of the spellbook:
trace("Spells initialised. Available spells are:");
for each (var _spell:String in spellBook)
{
trace(" ", _spell, " : ", spellBook[_spell]);
}
But this is the output I get:
Spells initialised. Available spells are:
[object BubbleSpell] : undefined
What I don't get is why it's not outputting:
Spells initialised. Available spells are:
bubble : [object BubbleSpell]
??
It's as if I'm iterating over the values in spellbook, rather than the keys... is that what's happening? All the docs I've seen so far seem to indicate that this is the correct way to iterate over a dictionary (which a generic object kind of is...) Do I need to call a method to get keys instead of values for generic objects?
So confused!
for each is used to iterate over values, you want to use a for loop which iterates over keys, eg:
var mySpells = {
speedy: new SpeedySpell(),
sleepy: new SleepySpell()
};
for (var key : String in mySpells) {
trace("key: " + key + " value: " + mySpells[key]);
}
Note that when iterating over the keys in a Dictionary you should leave the key value untyped (*) as a Dictionary's keys can be of any type (whereas an Object's keys can only be of type String).
A common trick would be to create a utility function which extracts an Object's keys to an Array for easier iteration:
function getKeys(source : Object) : Array {
const result : Array = [];
for (var key : * in source) {
result.push(key);
}
return result;
}
You can then make use of this helper function:
trace(getKeys(mySpells));
Because you're using a for each loop. Use a for loop instead:
for (var key:* in spellBook) {
trace(key + ': ' + spellBook[key]);
}

LINQ variable to list of string without using column names?

In an C# ASP.Net MVC project, I'm trying to make a List<string> from a LINQ variable.
Now this might be a pretty basic thing, but I just cannot get that to work without using the actual column names for the data in that variable. The thing is that in the interests of trying to make the program as dynamic as possible, I'm leaving it up to a stored procedure to get the data out. There can be any amount of any which way named columns depending on where the data is fetched from. All I care about is taking all of their values into a List<string>, so that I can compare user-input values with them in program.
Pointing to the columns by their names in the code means I'd have to make dozens of overloaded methods that all just basically do the same thing. Below is false non-functioning code. But it should open up the idea of what I mean.
// call for stored procedure
var courses = db.spFetchCourseInformation().ToList();
// if the data fails a check on a single row, it will not pass the check
bool passed = true;
foreach (var i in courses)
{
// each row should be cast into a list of string, which can then be validated
// on a row-by-row basis
List courseRow = new List();
courseRow = courses[i]; // yes, obviously this is wrong syntax
int matches = 0;
foreach (string k in courseRow)
{
if (validator.checkMatch(courseRow[k].ToString()))
{
matches++;
}
}
if (matches == 0)
{
passed = false;
break;
}
}
Now below is an example of how I currently have to do it because I need to use the names for the columns
for (int i = 0; i < courses.Count; i++)
{
int matches = 0;
if (validator.checkMatch(courses[i].Name))
matches++;
if (validator.checkMatch(courses[i].RandomOtherColumn))
matches++;
if (validator.checkMatch(courses[i].RandomThirdColumn))
matches++;
if (validator.checkMatch(courses[i].RandomFourthColumn))
matches++;
/* etc...
* etc...
* you get the point
* and one of these for each and every possible variation from the stored procedure, NOT good practice
* */
Thanks for help!
I'm not 100% sure what problem you are trying to solve (matching user data to a particular record in the DB?), but I'm pretty sure you're going about this in slightly the wrong fashion by putting the data in a List. I
t should be possible to get your user input in an IDictionary with the key being used for the column name, and the object as the input data field.
Then when you get the data from the SP, you can get the data back in a DataReader (a la http://msmvps.com/blogs/deborahk/archive/2009/07/09/dal-access-a-datareader-using-a-stored-procedure.aspx).
DataReaders are indexed on column name, so if you run through the keys in the input data IDictionary, you can check the DataReader to see if it has matching data.
using (SqlDataReader reader = Dac.ExecuteDataReader("CustomerRetrieveAll", null))
{
while (reader.Read())
{
foreach(var key in userInputDictionary.AllKeys)
{
var data = reader[key];
if (data != userInputDictionary[key]) continue;
}
}
}
Still not sure about the problem you are solving but, I hope this helps!
A little creative reflection should do the trick.
var courses = db.spFetchCourseInformation()
var values = courses.SelectMany(c => c.GetType().GetProperties() // gets the properties for your object
.Select(property => property.GetValue(c, null))); // gets the value of each property
List<string> stringValues = new List<string>(
values.Select(v => v == null ? string.Empty : v.ToString()) // some of those values will likely be null
.Distinct()); // remove duplicates

exception in Linq to sql

my query is :
var ReadAndUnreadMessages =
(from m in MDB.Messages
orderby m.Date descending
where m.ID_Receive == (Guid)USER.ProviderUserKey && m.Delete_Admin == false
select new AllMessages()
{
id = (LoadMessageChildren(m.ID_Message)[LoadMessageChildren(m.ID_Message).Count - 1] as Message).ID_Message,
parent = (Guid)(LoadMessageChildren(m.ID_Message)[LoadMessageChildren(m.ID_Message).Count - 1] as Message).ID_Message_Parent,
sender = (LoadMessageChildren(m.ID_Message)[LoadMessageChildren(m.ID_Message).Count - 1] as Message).ID_Sender,
receiver = (Guid)USER.ProviderUserKey,
subject = (LoadMessageChildren(m.ID_Message)[LoadMessageChildren(m.ID_Message).Count - 1] as Message).Subject.Subject1.ToString() == "Other" ?
(LoadMessageChildren(m.ID_Message)[LoadMessageChildren(m.ID_Message).Count - 1] as Message).Other_Subject
:
(LoadMessageChildren(m.ID_Message)[LoadMessageChildren(m.ID_Message).Count - 1] as Message).Subject.Subject1.ToString(),
body = (LoadMessageChildren(m.ID_Message)[LoadMessageChildren(m.ID_Message).Count - 1] as Message).Body.Length > 26 ?
(LoadMessageChildren(m.ID_Message)[LoadMessageChildren(m.ID_Message).Count - 1] as Message).Body.Substring(0, 25) + "..."
:
(LoadMessageChildren(m.ID_Message)[LoadMessageChildren(m.ID_Message).Count - 1] as Message).Body,
date = (LoadMessageChildren(m.ID_Message)[LoadMessageChildren(m.ID_Message).Count - 1] as Message).Date.ToShortDateString(),
read =(LoadMessageChildren(m.ID_Message)[LoadMessageChildren(m.ID_Message).Count - 1] as Message).IsRead,
finished = (LoadMessageChildren(m.ID_Message)[LoadMessageChildren(m.ID_Message).Count - 1] as Message).IsFinished,
count = MessageClass.LoadAll(m.ID_Message).Count
}).ToList();
and exception is :
The argument 'value' was the wrong type. Expected 'Message'. Actual 'System.Object'.
what does meaning it?
LoadMessageChildren :
public static ArrayList LoadMessageChildren(Guid Parent)
{
ArrayList arr = new ArrayList();
Guid id = Parent;
while (id != Guid.Empty)
{
arr.Add(LoadMessage(id));
try
{
id = (Guid)MDB.Messages.Single(a => a.ID_Message_Parent == id).ID_Message;
}
catch
{
id = Guid.Empty;
}
}
return arr;
}
LoadMessage :
public static Message LoadMessage(Guid id)
{
var mess = from m in MDB.Messages
where m.ID_Message == id
select m;
return mess.Single();
}
The code is unreadable, and as a bad case of code repetition (and multiple executions of LoadMessageChildren).
For starters, consider the following:
from m in MDB.Messages
orderby m.Date descending
where m.ID_Receive == (Guid)USER.ProviderUserKey && m.Delete_Admin == false
let children = LoadMessageChildren(m.ID_Message)
let lastChildMessage = children.Last()
select new AllMessages()
{
id = lastChildMessage.ID_Message,
...
}
This may solve your problem, as it is might be caused by using the [] indexer.
Aside from that, it is not clear the posted code is causing the exception.
The only thing I see you using LoadChildMessages() for in the end is to get the child message count... Unless I am wrong I would think you could write it as a join. You doing a lot of queries with in queries that don't seem necessary and are probably causing multiple hits to the database. My question to that would be why isn't there a relationship in your dmbl/sql database so that LinqToSql knows to create a property as a List<Message> ChildMessages
But here is my take:
var query = from message in MDB.Messges
join childmessage in MDB.Messages.Where(child => child.ID_Message_Parent == message.ID_Message) into childMessages
from childMessage in childMessages.DefaultIfEmpty() // This creates a
// left outer join so you get parent messages that don't have any children
where message.ID_Receive == (Guid)USER.ProviderUserKey && message.Delete_Admin == false
select new AllMessages()
{
id = message.ID_Message,
parent = message.ID_Message_Parent,
sender = message.ID_Sender,
receiver = (Guid)USER.ProviderUserKey,
subject = message.Subject.Subject1.ToString() == "Other" ?
message.Other_Subject
:
message.Subject.Subject1.ToString(),
body = message.Body.Length > 26 ?
message.Body.Substring(0, 25) + "..."
:
message.Body,
date = message.Date.ToShortDateString(),
read =message.IsRead,
finished = message.IsFinished,
count = childMessage.Count() // This might have to be this
//count = childMessage == null ? 0 : childMessage.Count()
};
var ReadAndUnreadMessages = query.ToList();
But it's hard to say because I can't run the code... Please respond and let me know if this works.
Note: May I suggest using a class that links to your DataContext.Log property that writes the generated TSQL code to the debugger window. Here is an article on writing your own. It has really help me know when I am making unnecessary calls to the database.
The error is most likely caused by the use of the ArrayList.
The problem is that LINQ was designed to work with generic collections that implement the System.Collections.Generic.IEnumerable<T> interface. The ArrayList is a nongeneric collection that internally stores everything as an Object. So when you retrieve something from the ArrayList you need to cast it to a Message.
Looking at your error message it looks like somewhere a Message object is expected, but the instance in your ArrayList (an Object) is not casted to a Message object when that reference occurs. Also, the ArrayList does not implement the IEnumerable<T> interface which might get you into trouble in certain situations also.
How to fix it?
I suggest changing the implementation of your LoadMessageChildren to use a generic list (List<Message>):
public static List<Message> LoadMessageChildren(Guid Parent)
{
List<Message> arr = new List<Message>();
Guid id = Parent;
while (id != Guid.Empty)
{
arr.Add(LoadMessage(id));
try
{
id = (Guid)MDB.Messages.Single(a => a.ID_Message_Parent == id).ID_Message;
}
catch
{
id = Guid.Empty;
}
}
return arr;
}
You will have to make also change the code that interacts with the generic list in terms of retrieving/referencing items. But that is just syntax. Since equivalent methods for dealist with lists and items exist.
There are also advantages in terms of performance and compile-time validation for switching from ArrayList to List<T>. The ArrayList is basically an inheritance from version 1.0 of the .Net Framework when there was no support for generics and it just get kept in the framework probably for compatibility reasons.
There are greater benefits for using generics.
UPDATED ANSWER:
The "Method 'System.Collections.Generic.List'1[Message] LoadMessageChildren(System.Guid)' has no supported translation to SQL" exception that you are getting is caused by the fact that your LoadMessageChildren method is not mapping to a stored procedure or a user defined function in your database.
You cannot have any regular C# method call inside your LINQ to SQL queries. The LINQ to SQL object model interprets a method found inside your query as either a stored procedure or a user defined function. So the engine is basically looking for a method called LoadMessageChildren that maps to a stored procedure or a user defined function in your database. Because there are no mappings, it tells you that no supported translation to SQL was found. The LINQ to SQL object model link shows you how to use method attributes to map a method that executes a stored procedure.
You have a few choices now:
create stored procedures of your regular C# method calls
rewrite your LINQ query to use joins to select child messages