Sticking to Excel, I am parsing response text to custom objects using vba-json. Sometimes there are thousands of objects in the json string. Using application.ontime recursively to run some less expensive status requests in intervals of only a few seconds so efficiency is important.
Simply parsing the data to an array is fast but gets expensive and complex the more the data needs to be manipulated.
I’m looking for outside the box efficiencies beyond the standard “speed up excel” stuff.
An example would be parsing the json objects to a class property with getters pointing to the property inside the json object.
Private jStr
Public property let JSON(value)
Set jStr = value
End property
Public property get Name() as string
Name = jStr(“name”)
End property
Public property get ID() as string
ID = jStr(“id”)
End property
Instead of assigning variables, I just pass in the parsed string.
I have tried various combinations of collections, nested collections/arrays/arraylists and while the differences in some cases are seemingly insignificant, the right combination of types will help a lot when repeated thousands of times.
If anyone has suggestions please chime in. If I’m going about this the wrong way and someone needs to wear me out about it then by all means... be my guest. I just need some creative thinking. Obviously if I step outside of VBA then this isn’t a problem but here we are...
Related
I'm trying to access the roomName, but so far I am unable. I don't get how to get past the barriar of the info.[long ID with dashes].roomName.
At most I can get back the object of the long id or undefined.
I have tried info[0].roomName. Trying to get the first object in the info and then go on from there. The long id number is also in list.id, I don't know if that can help.
I would have set the info as an array like list is, but this is not my JSON, only one that I am working with.
{
"list":[ IGNORE, can access code here ],
"info":{
"e5eb1ccf-bd45-4d01-8e2a":{
"id":"e5eb1ccf-bd45-4d01-8e2a",
"name":"Lucy",
"roomName":"Arts" <<I need to get to this.
}
}
}
I hope this makes sense, first post and this is just a boiled down version of what I have. Putting in the id number in the info.e5eb1ccf-bd45-4d01-8e2a.roomName breaks after the first -
Once you've parsed the JSON (assuming you even have JSON*) and you have an object, you'd use brackets notation:
var room = theObject.info["e5eb1ccf-bd45-4d01-8e2a"].roomName;
console.log(room); // Arts
* Remember, JSON is a textual notation for data exchange. (More here.) If you're dealing with JavaScript source code, and not dealing with a string, you're not dealing with JSON.
I have a Typescript app. I use the localstorage for development purpose to store my objects and I have the problem at the deserialization.
I have an object meeting of type MeetingModel:
export interface MeetingModel {
date: moment.Moment; // from the library momentjs
}
I store this object in the localStorage using JSON.stringify(meeting).
I suppose that stringify call moment.toJson(), that returns an iso string, hence the value stored is: {"date":"2016-12-26T15:03:54.586Z"}.
When I retrieve this object, I do:
const stored = window.localStorage.getItem("meeting");
const meeting: MeetingModel = JSON.parse(stored);
The problem is: meeting.date contains a string instead of a moment !
So, first I'm wondering why TypeScript let this happen ? Why can I assign a string value instead of a Moment and the compiler agree ?
Second, how can I restore my objects from plain JSON objects (aka strings) into Typescript types ?
I can create a factory of course, but when my object database will grow up it will be a pain in the *** to do all this work.
Maybe there is a solution for better storing in the local storage in the first place?
Thank you
1) TypeScript is optionally typed. That means there are ways around the strictness of the type system. The any type allows you to do dynamic typing. This can come in very handy if you know what you are doing, but of course you can also shoot yourself in the foot.
This code will compile:
var x: string = <any> 1;
What is happening here is that the number 1 is casted to any, which to TypeScript means it will just assume you as a developer know what it is and how you to use it. Since the any type is then assigned to a string TypeScript is absolutely fine with it, even though you are likely to get errors during run-time, just like when you make a mistake when coding JavaScript.
Of course this is by design. TypeScript types only exist during compile time. What kind of string you put in JSON.parse is unknowable to TypeScript, because the input string only exists during run-time and can be anything. Hence the any type. TypeScript does offer so-called type guards. Type guards are bits of code that are understood during compile-time as well as run-time, but that is beyond the scope of your question (Google it if you're interested).
2) Serializing and deserializing data is usually not as simple as calling JSON.stringify and JSON.parse. Most type information is lost to JSON and typically the way you want to store objects (in memory) during run-time is very different from the way you want to store them for transfer or storage (in memory, on disk, or any other medium). For instance, during run-time you might need lookup tables, user/session state, private fields, library specific properties, while in storage you might want version numbers, timestamps, metadata, different types of normalization, etc. You can JSON.stringify anything you want in JavaScript land, but that does necessarily mean it is a good idea. You might want to design how you actually store data. For example, an iso string looks pretty, but takes a lot of bytes. If you have just a few that does not matter, but when you are transferring millions a second you might want to consider another format.
My advise to you would be to define interfaces for the objects you want to save and like moment create a .toJson method on your model object, which will return the DTO (Data Transfer Object) that you can simply serialize with JSON.stringify. Then on the way back you cast the any output of JSON.parse to your DTO and then convert it back to your model with a factory function or constructor of your creation. That might seem like a lot of boilerplate, but in my experience it is totally worth it, because now you are in control of what gets stored and that gives you a lot of flexility to change your model without getting deserialization problems.
Good luck!
You could use the reviver feature of JSON.parse to convert the string back to a moment:
JSON.parse(input, (key, value) => {
if (key == "date") {
return parseStringAsMoment(value);
} else {
return value;
});
Check browser support for reviver, though, as it's not the same as basic JSON.parse
So I've read all the posts on String Based Enums in Typescript, but I couldn't find a solution that meets my requirements. Those would be:
Enums that provide code completion
Enums that can be iterated over
Not having to specify an element twice
String based
The possibilities I've seen so far for enums in typescript are:
enum MyEnum {bla, blub}: This fails at being string based, so I can't simply read from JSONs which are string based...
type MyEnum = 'bla' | 'blub': Not iterable and no code completion
Do it yourself class MyEnum { static get bla():string{return "bla"} ; static get blub():string{return "blub"}}: Specifies elements twice
So here come the questions:
There's no way to satisfy those requirements simultaneously? If no, will it be possible in the future?
Why didn't they make enums string based?
Did someone experience similar problems and how did you solve them?
I think that implementing Enum in a C-like style with numbers is fine, because an Enum (similar to a Symbol) is usually used to declare a value that is uniquely identifiable on development time. How the machine represents that value on run time doesn't really concern the developer.
But what we developer sometimes want (because we're all lazy and still want to have all the benefits!), is to use the Enum as an API or with an API that does not share that Enum with us, even though the API is essentially an Enum because the valid value of a property only is foo and bar.
I guess this is the reason why some languages have string based Enums :)
How TypeScript handles Enums
If you look at the transpiled JavaScript you can see that TypeScript just uses a plain JavaScript Object to implement an Enum. For example:
enum Color {
Red,
Green,
Blue
}
will be transpiled to:
{
0: "Red",
1: "Green",
2: "Blue",
Blue: 2,
Green: 1,
Red: 0
}
This means you can access the string value like Color[Color.Red]. You will still have code completion and you do not have to specify the values twice. But you can not just do Object.keys(Color) to iterate over the Enum, because the values exist "twice" on the object.
Why didn't they make enums string based
To be clear Enums are both number and string based in that direct access is number and reverse map is string (more on this).
Meeting your requirement
You key reason for ruling out raw enums is
so I can't simply read from JSONs which are string based...
You will experience the same thing e.g. when reading Dates cause JSON has no date data type. You would new Date("someServerDateTime") to convert these.
You would use the same strategy to go from server side enum (string) to TS enum (number). Easy done thanks to the reverse lookup MyEnum["someServerString"]
Hydration
This process of converting server side data to client side active data is sometimes called Hydration. My favorite lib for this at the moment is https://github.com/pleerock/class-transformer
I personally handle this stuff myself at the server access level i.e. hand write an API that makes the XHR + does the serialization.
At my last job we automated this with code generation that did even more than that (supported common validation patterns between server and client code).
My server returns a list of objects in JSON. They might be Cats or Dogs, for example.
When I know that they'll all be Cats, I can set the AutoBeanCodex to work easily. When I don't know what types they are, though... what should I do?
I could give all of my entities a type field, but then I'd have to parse each entity before passing it to the AutoBeanCodex, which borders on defeating the point. What other options do I have?
Just got to play with this the other day, and fought it for a few hours, trying #Category methods and others, until I found this: You can create a property of type Splittable, which represents the underlying transport type that has some encoding for booleans/Strings/Lists/Maps. In my case, I know some enveloping type that goes over the wire at design time, and based on some other property, some other field can be any number of other autobeans.
You don't even need to know the type of the other bean at compile time, you could get values out using Splittable's methods, but if using autobeans anyway, it is nice to define the data that is wrapped.
interface Envelope {
String getStatus();
String getDataType();
Splittable getData();
}
(Setters might be desired if you sending data as well as recieving - encoding a bean into a `Splittable to send it in an envelope is even easier than decoding it)
The JSON sent over the wire is decoded (probably using AutoBeanCodex) into the Envelope type, and after you've decided what type must be coming out of the getData() method, call something like this to get the nested object out
SpecificNestedBean bean = AutoBeanCodex.decode(factory,
SpecificNestedBean.class,
env.getData()).as();
The Envelope type and the nested types (in factory above) don't even need to be the same AutoBeanFactory type. This could allow you to abstract out the reading/writing of envelopes from the generic transport instance, and use a specific factory for each dataType string property to decode the data's model (and nested models).
I have an interesting problem to solve that would be helped by successfully casting objects created by LINQ to SQL into a single master object that I could pass around. Here is the scenario at a high level.
I have a number of stored procedures that fetch data and then all return the exact same columns. The params into the procs and the logic vary greatly, so a single proc will not work. Then Linq creates a strongly typed object which is used throughout my application as parameter and return values.
I am using these strongly typed objects as noted above as parameters and return values in a series of filters used to analyze stocks. My client would like to change the order the order of the filters. The issue is that each succeeding filter will only work on what passed the last filter.
Currently I am hard coding my parameters, and if I could create a master object that I could cast any of these Linq objects to, I could then always pass and return the master object.
I have read the materials available on the internet about casting between different types such as static to anonymous types or a list of integers and an array list containing objects representing integers, but I need to actually cast one object into another.
What general direction would I take to solve this problem of converting strongly typed objects generated by linq that are exactly the same into a single master object?
Thank you for any of your thoughts.
If all your linq objects have the same fields, you could have them implement an interface defined with those common fields. Then the calls to your filter methods can depend on an interface rather than a specific implementation. In other words, the parameters in the filter methods will be of the interface type rather than a linq class type.
e.g.: Where ICommonFields is an interface you define with all the common fields in each l2s class -
public class Filterer
{
public ICommonFields filterStuff(ICommonFields x)
{
//do stuff
}
}
or -
public class Filterer
{
public T filterStuff<T>(T x)
where T: class, ICommonFields, new()
{
//do stuff
}
}
I'd prefer the generic version, as T becomes the actual type rather than a reference through an interface - linq-to-sql has issues when using a type through an interface with query expressions.
Edit: sorry, it was late when i first wrote this response (likely excuse! :). Fixed my obvious mistake with the example code :)
Although there might be a way to do this with casting, I'm going to offer you a quick and dirty solution - and I'm assuming that your resultant objects are collection-based:
Given that all of your child objects
all share the same columns, go ahead
and pick one of them to act as your
master object - then simply iterate
through the rows of your other LINQ
objects and add them to the collection
of your master object. If your
resultant object is a strongly typed
data table, then all you'd do is Add
to the .Rows collection.
Additionally, you might be able to just add the elements retrieved some subsequent LINQ queries directly to your master object depending upon how you write your SELECT causes in LINQ.